Learning pharmacometric covariate model structures with symbolic regression networks

Efficiently finding covariate model structures that minimize the need for random effects to describe pharmacological data is challenging. The standard approach focuses on identification of relevant covariates, and present methodology lacks tools for automatic identification of covariate model structures. Although neural networks could potentially be used to approximate covariate-parameter relationships, such approximations are not human-readable and come at the risk of poor generalizability due to high model complexity.In the present study, a novel methodology for the simultaneous selection of covariate model structure and optimization of its parameters is proposed. It is based on symbolic regression, posed as an optimization problem with a smooth loss function. This enables training of the model through back-propagation using efficient gradient computations.Feasibility and effectiveness are demonstrated by application to a clinical pharmacokinetic data set for propofol, containing infusion and blood sample time series from 1031 individuals. The resulting model is compared to a published state-of-the-art model for the same data set. Our methodology finds a covariate model structure and corresponding parameter values with a slightly better fit, while relying on notably fewer covariates than the state-of-the-art model. Unlike contemporary practice, finding the covariate model structure is achieved without an iterative procedure involving manual interactions.

First-principles, machine learning and symbolic regression modelling for organic molecule adsorption on two-dimensional CaO surface

Data-driven methods are receiving significant attention in recent years for chemical and materials researches; however, more works should be done to leverage the new paradigm to model and analyze the adsorption of the organic molecules on low-dimensional surfaces beyond using the traditional simulation methods. In this manuscript, we employ machine learning and symbolic regression method coupled with DFT calculations to investigate the adsorption of atmospheric organic molecules on a low-dimensional metal oxide mineral system. The starting dataset consisting of the atomic structures of the organic/metal oxide interfaces are obtained via the density functional theory (DFT) calculation and different machine learning algorithms are compared, with the random forest algorithm achieving high accuracies for the target output. The feature ranking step identifies that the polarizability and bond type of the organic adsorbates are the key descriptors for the adsorption energy output. In addition, the symbolic regression coupled with genetic programming automatically identifies a series of hybrid new descriptors displaying improved relevance with the target output, suggesting the viability of symbolic regression to complement the traditional machine learning techniques for the descriptor design and fast modeling purposes. This manuscript provides a framework for effectively modeling and analyzing the adsorption of the organic molecules on low-dimensional surfaces via comprehensive data-driven approaches.

SNR: Symbolic network-based rectifiable learning framework for symbolic regression

Symbolic regression (SR) can be utilized to unveil the underlying mathematical expressions that describe a given set of observed data. At present, SR can be categorized into two methods: learning-from-scratch and learning-with-experience. Compared to learning-from-scratch, learning-with-experience yields results that are comparable to those of several benchmarks and incurs significantly lower time costs for obtaining expressions. However, the learning-with-experience model performs poorly in terms of unseen data distributions and lacks a rectification tool, apart from constant optimization, which exhibits limited performance. In this study, we propose a Symbolic Network-based Rectifiable Learning Framework (SNR) that possesses the ability to correct errors. SNR adopts Symbolic Network (SymNet) to represent an expression, and the encoding of SymNet is designed to provide supervised information, with numerous self-generated expressions, to train a policy net (PolicyNet). The training of PolicyNet can offer prior knowledge to guide effective searches. Subsequently, the incorrectly predicted expressions are revised via a rectification mechanism. This rectification mechanism endows SNR with broader applicability. Experimental results demonstrate that our proposed method achieves the highest averaged coefficient of determination on self-generated datasets when compared with other state-of-the-art methods and yields more accurate results in public datasets.

Exploring the factors effecting on carbon emissions in each province in China: A comprehensive study based on symbolic regression, LMDI and Tapio models

Carbon emission (CE) has led to increasingly severe climate problems. The key to reducing CE is to identify the dominant influencing factors and explore their influence degree. The CE data of 30 provinces from 1997 to 2020 in China were calculated by IPCC method. Based on this, the importance order of six factors included GDP, Industrial Structure (IS), Total Population (TP), Population Structure (PS), Energy Intensity (EI) and Energy Structure (ES) affecting the CE of China's provinces were obtained by using symbolic regression, then the LMDI and the Tapio models were established to deeply explore the influence degree of different factors on CE. The results showed that the 30 provinces were divided into five categories according to the primary factor, GDP was the most important factor, followed by ES and EI, then IS, and the least TP and PS. The growth of per capita GDP promoted the increase of CE, while reduced EI inhibited the increase of CE. The increase of ES promoted CE in some provinces but inhibited in others. The increase of TP weakly promoted the increase of CE. These results can provide some references for governments to formulate relevant CE reduction policies under dual carbon goal.

Energy loss and contraction coefficients-based vertical sluice gate's discharge coefficient under submerged flow using symbolic regression

Accurate calculation of discharge is a critical task in terms of environmental and operational regulations. In the current study, a new approach for determining vertical sluice gates' flow discharge with a minor bias is proposed. Energy-momentum equations are used to characterize the physical expression of the phenomena intended for generation of the coefficient of discharge. The coefficient of discharge is then expressed according to coefficients of energy loss and contraction. Following that, the coefficient of discharge, coefficient of contraction, and coefficient of energy loss are calculated using an optimization approach. Then, dimensional analysis is conducted and regression equations for quantifying the coefficient of energy loss is produced using symbolic regression method. The derived contraction coefficient and energy loss coefficient formulas are accordingly utilized to compute the coefficient of discharge in the vertical sluice gate and also to determine flow discharge. For computing discharge, five different scenarios are considered. The developed approaches' performance is examined against selected benchmarks from the literature. The results show that the symbolic regression method can compute discharge more accurate than its alternatives.

Towards highly economical and accurate wastewater sensors by reduced parts of the LED-visible spectrum

Interest is growing in simple, fast and inexpensive systems to analyze urban wastewater quality in real time. In this research project, a methodology is presented for the characterization of COD, BOD₅, TSS, TN, and TP of wastewater samples, without the need to alter the samples or use chemical reagents, from a few wavelengths, belonging to the different color groups that compose the visible spectrum in isolation: (380-700 nm): violet (380-427 nm), blue (427-476 nm), cyan (476-497 nm), green (497-570 nm), yellow (570-581 nm), orange (581-618 nm), and red (618-700 nm). In this study, about 650 raw and treated urban wastewater samples from over 43 WWTPs and a total of 36 estimation models based on genetic algorithms have been calculated. Seven models were calculated for each pollutant parameter; one model for each color group of the visible spectrum, except for TN, which includes an additional model combining the wavelengths of the violet and red region of the spectrum. All the calculated models showed high accuracy, with an R² between 80 and 85 % for COD, BOD₅ and TSS, and 66-74 % for TN and TP. The tests carried out have shown the accuracy of the models of the different color groups to be very close to each other. However, it is noted that the models making use of the wavelengths between 497 and 570 nm (green) were the ones that showed the best performance in all the parameters under study. This research work lays the foundations for the development of cheaper, faster, and simpler wastewater monitoring and characterization equipment.

Symbolic expression generation via variational auto-encoder

There are many problems in physics, biology, and other natural sciences in which symbolic regression can provide valuable insights and discover new laws of nature. Widespread deep neural networks do not provide interpretable solutions. Meanwhile, symbolic expressions give us a clear relation between observations and the target variable. However, at the moment, there is no dominant solution for the symbolic regression task, and we aim to reduce this gap with our algorithm. In this work, we propose a novel deep learning framework for symbolic expression generation via variational autoencoder (VAE). We suggest using a VAE to generate mathematical expressions, and our training strategy forces generated formulas to fit a given dataset. Our framework allows encoding apriori knowledge of the formulas into fast-check predicates that speed up the optimization process. We compare our method to modern symbolic regression benchmarks and show that our method outperforms the competitors under noisy conditions. The recovery rate of SEGVAE is 65% on the Ngyuen dataset with a noise level of 10%, which is better than the previously reported SOTA by 20%. We demonstrate that this value depends on the dataset and can be even higher.

Exploratory research on influential factors of China's sulfur dioxide emission based on symbolic regression

The amount of China's sulfur dioxide emission remains significantly large in recent years. To further reduce sulfur dioxide emission, the key is to find out the leading factors affecting sulfur dioxide emission and then take measures to control it accordingly. In order to investigate the influential factors of sulfur dioxide emission of various provinces, the data of sulfur dioxide emission of 30 provinces in China from 2001 to 2020 were collected. We established the symbolic regression model to explore the relationship between the GDP (x₁), total population (x₂), total energy consumption (x₃), thermal power installed capacity (x₄), and sulfur dioxide emission (dependent variable) for each province. The results show that the amount of China's total sulfur dioxide emission and sulfur dioxide emission in most provinces meet the environmental Kuznets curve (EKC). The influential degree of the factors affecting China's sulfur dioxide emission are GDP, total energy consumption, thermal power installed capacity, and total population. The provinces with the primary factor of GDP have the lowest average total energy consumption and average thermal power installed capacity, and their average sulfur dioxide emissions are also relatively low. The provinces with the primary factor of GDP do not show obvious geographical characteristics, but the provinces with the primary factor of total energy consumption are all distributed in southern China. Based on the research results, some control measures are also put forward.

Combining symbolic regression with the Cox proportional hazards model improves prediction of heart failure deaths

BACKGROUND

Heart failure is a clinical syndrome characterised by a reduced ability of the heart to pump blood. Patients with heart failure have a high mortality rate, and physicians need reliable prognostic predictions to make informed decisions about the appropriate application of devices, transplantation, medications, and palliative care. In this study, we demonstrate that combining symbolic regression with the Cox proportional hazards model improves the ability to predict death due to heart failure compared to using the Cox proportional hazards model alone.

METHODS

We used a newly invented symbolic regression method called the QLattice to analyse a data set of medical records for 299 Pakistani patients diagnosed with heart failure. The QLattice identified non-linear mathematical transformations of the available covariates, which we then used in a Cox model to predict survival.

RESULTS

An exponential function of age, the inverse of ejection fraction, and the inverse of serum creatinine were identified as the best risk factors for predicting heart failure deaths. A Cox model fitted on these transformed covariates had improved predictive performance compared with a Cox model on the same covariates without mathematical transformations.

CONCLUSION

Symbolic regression is a way to find transformations of covariates from patients' medical records which can improve the performance of survival regression models. At the same time, these simple functions are intuitive and easy to apply in clinical settings. The direct interpretability of the simple forms may help researchers gain new insights into the actual causal pathways leading to deaths.

Implementing Early Warning Systems in WWTP. An investigation with cost-effective LED-VIS spectroscopy-based genetic algorithms

Measuring how the pollution load evolves in real time along sewer networks is key for proper management of water resources and protecting the environment. The technique of molecular spectroscopy for water characterization has increasingly widespread use, as it is a non-invasive technique that leads to the correlation of the physical-chemical conditions of wastewater with spectroscopic surrogates by a series of mathematical estimation models. In the present research work, different symbolic regression models obtained with evolutive genetic algorithms are evaluated for the estimation of chemical oxygen demand (COD); five-day biochemical oxygen demand (BOD₅); total suspended solids (TSS); total phosphorus (TP); and total nitrogen (TN), from the spectral response of samples measured between 380 and 700 nm and without the use of chemicals or pre-treatment. Around 650 wastewater samples were used in the campaign, from 43 different wastewater treatment plants (WWTP) in which both, raw/influent and treated/effluent, were examined through 18 models composed of Classical Genetic Algorithm (CGA), the Age-Layered Population Structure (ALPS), and Offspring Selection (OS) by mean of HeuristicLab software, to make a comparison among them and to determine which models and wavelengths are most suitable for the correlation. Models are proposed considering both raw and treated samples together (15) and only with tertiary treated wastewater reclaimed for agriculture irrigation effluent (3). The Pearson correlation coefficients were in the range of 67-91% for the test data in the case of the combined models. The results conform the first steps for a real-time monitoring of WWTP.

Shape-Constrained Symbolic Regression-Improving Extrapolation with Prior Knowledge

We investigate the addition of constraints on the function image and its derivatives for the incorporation of prior knowledge in symbolic regression. The approach is called shape-constrained symbolic regression and allows us to enforce, for example, monotonicity of the function over selected inputs. The aim is to find models which conform to expected behavior and which have improved extrapolation capabilities. We demonstrate the feasibility of the idea and propose and compare two evolutionary algorithms for shape-constrained symbolic regression: (i) an extension of tree-based genetic programming which discards infeasible solutions in the selection step, and (ii) a two-population evolutionary algorithm that separates the feasible from the infeasible solutions. In both algorithms we use interval arithmetic to approximate bounds for models and their partial derivatives. The algorithms are tested on a set of 19 synthetic and four real-world regression problems. Both algorithms are able to identify models which conform to shape constraints which is not the case for the unmodified symbolic regression algorithms. However, the predictive accuracy of models with constraints is worse on the training set and the test set. Shape-constrained polynomial regression produces the best results for the test set but also significantly larger models.

What is the appropriate approach in sex determination of hyoid bones?

The hyoid bone is characterized by sexually dimorphic features, enabling it to occasionally be used in the sex determination aspect of establishing the biological profile in skeletal remains. Based on a sample of 298 fused and non-fused hyoid bones, the present paper compares several methodological approaches to sexing human hyoid bones in order to test the legitimacy of osteometrics-based linear discriminant equations and to explore the potentials of symbolic regression and methods of geometric morphometrics. In addition, two sets of published predictive models, one of which originated in an indigenous population, were validated on the studied sample. The results showed that the hyoid shape itself is a moderate sex predictor and a combination of linear measurements is a better representation of sex-related differences. The symbolic regression was shown to exceed the predictive powers of linear discriminant function analysis when two models based on a logistic and step regression reached 96% of correctly classified cases. There was a positive correlation between discriminant scores and an individual's age as the sex assessment was highly skewed in favour of males. This suggests that the human hyoid undergoes age-related modifications which facilitates determination of male bones and complicates determination of females in older individuals. The validation of discriminant equations by Komenda and Černý (1990) and Kindschud et al. (2010) revealed that there are marked inter-population and inter-sample differences which lessened the power to correctly determine female hyoid bones.