Machine learning in computational chemistry: interplay between (non)linearity, basis sets, and dimensionality

Accelerated Development of Novel Biomass-Based Polyurethane Adhesives via Machine Learning

2-Pyrone-4,6-dicarboxylic acid (PDC) can be produced on a large scale from lignin by transformed bacteria, and its use as a bifunctional monomer to synthesize biomass-based polymers has been reported. Recently, excellent adhesive properties of the resulting biomass-based polymers were also reported, but their performance has not yet been optimized. In this study, we focus on improving the adhesive properties of PDC-based polyurethanes (PUs) by combining experiments and machine learning (ML). We synthesized an initial data set of 25 adhesive samples from different polyols and isocyanates with different isocyanate-to-hydroxyl ratios (r). Adhesive strengths were measured after hot-pressing at varying temperatures (Theat, °C) and durations (theat, h), following a Taguchi L25 orthogonal design. Gaussian process-based Bayesian optimization (BO) was employed to identify an optimal PDC-based PU adhesive as a function of polyol type, isocyanate type, r ratio, heating temperature, and time with an improved adhesive strength of 10.04 ± 1.26 MPa after only five iterations. This approach highlights the effectiveness of BO in guiding experimental conditions for an enhanced performance. Random Forest regression was also used as an alternative ML approach and supported the conclusions. Overall, this study demonstrates the potential of the BO in accelerating the development and optimization of novel adhesive materials.

The Analysis of Electron Densities: From Basics to Emergent Applications

The electron density determines all properties of a system of nuclei and electrons. It is both computable and observable. Its topology allows gaining insight into the mechanisms of bonding and other phenomena in a way that is complementary to and beyond that available from the molecular orbital picture and the formal oxidation state (FOS) formalism. The ability to derive mechanistic insight from electron density is also important with methods where orbitals are not available, such as orbital-free density functional theory (OF-DFT). While density topology-based analyses such as QTAIM (quantum theory of atoms-in-molecules) have been widely used, novel, vector-based techniques recently emerged such as next-generation (NG) QTAIM. Density-dependent quantities are also actively used in machine learning (ML)-based methods, in particular, for ML DFT functional development, including machine-learnt kinetic energy functionals. We review QTAIM and its recent extensions such as NG-QTAIM and localization-delocalization matrices (LDM) and their uses in the analysis of bonding, conformations, mechanisms of redox reactions excitations, as well as ultrafast phenomena. We review recent research showing that direct density analysis can circumvent certain pitfalls of the FOS formalism, in particular in the description of anionic redox, and of the widely used (spherically) projected density of states analysis. We discuss uses of density-based quantities for the construction of DFT functionals and prospects of applications of analyses of density topology to get mechanistic insight with OF-DFT and recently developed time-dependent OF-DFT.

Developing Machine Learning-Based Predictive Models for Hallux Valgus Recurrence Based on Measurements From Radiographs

BACKGROUND

Machine learning (ML) is increasingly used to predict the prognosis of numerous diseases. This retrospective analysis aimed to develop a prediction model using ML algorithms and to identify predictors associated with the recurrence of hallux valgus (HV) following surgery.

METHODS

A total of 198 symptomatic feet that underwent chevron osteotomy combined with a distal soft tissue procedure were enrolled and analyzed from 2 independent medical centers. The feet were grouped according to nonrecurrence or recurrence based on 1-year follow-up outcomes. Preoperative weightbearing radiographs and immediate postoperative nonweightbearing radiographs were obtained for each HV foot. Radiographic measurements (eg, HV angle and intermetatarsal angle) were acquired and used for ML model training. A total of 9 commonly used ML models were trained on the data obtained from one institute (108 feet), and tested on the other data set from another independent institute (90 feet) for external validation. Optimal feature sets for each model were identified based on a 2000-resample bootstrap-based internal validation via an exhaustive search. The performance of each model was then tested on the external validation set. The area under the curve (AUC), classification accuracy, sensitivity, and specificity of each model were calculated to evaluate the performance of each model.

RESULTS

The support vector machine (SVM) model showed the highest predictive accuracy compared to other methods, with an AUC of 0.88 and an accuracy of 75.6%. Preoperative hallux valgus angle, tibial sesamoid position, postoperative intermetatarsal angle, and postoperative tibial sesamoid position were identified as the most selected features by several ML models.

CONCLUSION

ML classifiers such as SVM could predict the recurrence of HV (an HVA >20 degrees) at a 1-year follow-up while identifying associated predictors in a multivariate manner. This study holds the potential for foot and ankle surgeons to effectively identify individuals at higher risk of HV recurrence postsurgery.

An Unsupervised Machine Learning Approach for the Automatic Construction of Local Chemical Descriptors

Condensing the many physical variables defining a chemical system into a fixed-size array poses a significant challenge in the development of chemical Machine Learning (ML). Atom Centered Symmetry Functions (ACSFs) offer an intuitive featurization approach by means of a tedious and labor-intensive selection of tunable parameters. In this work, we implement an unsupervised ML strategy relying on a Gaussian Mixture Model (GMM) to automatically optimize the ACSF parameters. GMMs effortlessly decompose the vastness of the chemical and conformational spaces into well-defined radial and angular clusters, which are then used to build tailor-made ACSFs. The unsupervised exploration of the space has demonstrated general applicability across a diverse range of systems, spanning from various unimolecular landscapes to heterogeneous databases. The impact of the sampling technique and temperature on space exploration is also addressed, highlighting the particularly advantageous role of high-temperature Molecular Dynamics (MD) simulations. The reliability of the resulting features is assessed through the estimation of the atomic charges of a prototypical capped amino acid and a heterogeneous collection of CHON molecules. The automatically constructed ACSFs serve as high-quality descriptors, consistently yielding typical prediction errors below 0.010 electrons bound for the reported atomic charges. Altering the spatial distribution of the functions with respect to the cluster highlights the critical role of symmetry rupture in achieving significantly improved features. More specifically, using two separate functions to describe the lower and upper tails of the cluster results in the best performing models with errors as low as 0.006 electrons. Finally, the effectiveness of finely tuned features was checked across different architectures, unveiling the superior performance of Gaussian Process (GP) models over Feed Forward Neural Networks (FFNNs), particularly in low-data regimes, with nearly a 2-fold increase in prediction quality. Altogether, this approach paves the way toward an easier construction of local chemical descriptors, while providing valuable insights into how radial and angular spaces should be mapped. Finally, this work opens the possibility of encoding many-body information beyond angular terms into upcoming ML features.

Degeneration of kernel regression with Matern kernels into low-order polynomial regression in high dimension

Kernel methods such as kernel ridge regression and Gaussian process regression with Matern-type kernels have been increasingly used, in particular, to fit potential energy surfaces (PES) and density functionals, and for materials informatics. When the dimensionality of the feature space is high, these methods are used with necessarily sparse data. In this regime, the optimal length parameter of a Matern-type kernel may become so large that the method effectively degenerates into a low-order polynomial regression and, therefore, loses any advantage over such regression. This is demonstrated theoretically as well as numerically in the examples of six- and fifteen-dimensional molecular PES using squared exponential and simple exponential kernels. The results shed additional light on the success of polynomial approximations such as PIP for medium-size molecules and on the importance of orders-of-coupling-based models for preserving the advantages of kernel methods with Matern-type kernels of on the use of physically motivated (reproducing) kernels.

Developing predictive models for surgical outcomes in patients with degenerative cervical myelopathy: a comparison of statistical and machine learning approaches

BACKGROUND CONTEXT

Machine learning (ML) is widely used to predict the prognosis of numerous diseases.

PURPOSE

This retrospective analysis aimed to develop a prognostic prediction model using ML algorithms and identify predictors associated with poor surgical outcomes in patients with degenerative cervical myelopathy (DCM).

STUDY DESIGN

A retrospective study.

PATIENT SAMPLE

A total of 406 symptomatic DCM patients who underwent surgical decompression were enrolled and analyzed from three independent medical centers.

OUTCOME MEASURES

We calculated the area under the curve (AUC), classification accuracy, sensitivity, and specificity of each model.

METHODS

The Japanese Orthopedic Association (JOA) score was obtained before and 1 year following decompression surgery, and patients were grouped into good and poor outcome groups based on a cut-off value of 60% based on a previous study. Two datasets were fused for training, 1 dataset was held out as an external validation set. Optimal feature-subset and hyperparameters for each model were adjusted based on a 2,000-resample bootstrap-based internal validation via exhaustive search and grid search. The performance of each model was then tested on the external validation set.

RESULTS

The Support Vector Machine (SVM) model showed the highest predictive accuracy compared to other methods, with an AUC of 0.82 and an accuracy of 75.7%. Age, sex, disease duration, and preoperative JOA score were identified as the most commonly selected features by both the ML and statistical models. Grid search optimization for hyperparameters successfully enhanced the predictive performance of each ML model, and the SVM model still had the best performance with an AUC of 0.93 and an accuracy of 86.4%.

CONCLUSIONS

Overall, the study demonstrated that ML classifiers such as SVM can effectively predict surgical outcomes for patients with DCM while identifying associated predictors in a multivariate manner.

Machine learning of kinetic energy densities with target and feature smoothing: Better results with fewer training data

Machine learning (ML) of kinetic energy functionals (KEFs), in particular kinetic energy density (KED) functionals, is a promising way to construct KEFs for orbital-free density functional theory (DFT). Neural networks and kernel methods including Gaussian process regression (GPR) have been used to learn Kohn-Sham (KS) KED from density-based descriptors derived from KS DFT calculations. The descriptors are typically expressed as functions of different powers and derivatives of the electron density. This can generate large and extremely unevenly distributed datasets, which complicates effective application of ML techniques. Very uneven data distributions require many training datapoints, can cause overfitting, and can ultimately lower the quality of an ML KED model. We show that one can produce more accurate ML models from fewer data by working with smoothed density-dependent variables and KED. Smoothing palliates the issue of very uneven data distributions and associated difficulties of sampling while retaining enough spatial structure necessary for working within the paradigm of KEDF. We use GPR as a function of smoothed terms of the fourth order gradient expansion and KS effective potential and obtain accurate and stable (with respect to different random choices of training points) kinetic energy models for Al, Mg, and Si simultaneously from as few as 2000 samples (about 0.3% of the total KS DFT data). In particular, accuracies on the order of 1% in a measure of the quality of energy-volume dependence B'=EV0-ΔV-2EV0+E(V0+ΔV)ΔV/V02 (where V0 is the equilibrium volume and ΔV is a deviation from it) are obtained simultaneously for all three materials.