Target-specific novel molecules with their recipe: Incorporating synthesizability in the design process

Application of Artificial intelligence (AI) in drug discovery has led to several success stories in recent times. While traditional methods mostly relied upon screening large chemical libraries for early-stage drug-design, de novo design can help identify novel target-specific molecules by sampling from a much larger chemical space. Although this has increased the possibility of finding diverse and novel molecules from previously unexplored chemical space, this has also posed a great challenge for medicinal chemists to synthesize at least some of the de novo designed novel molecules for experimental validation. To address this challenge, in this work, we propose a novel forward synthesis-based generative AI method, which is used to explore the synthesizable chemical space. The method uses a structure-based drug design framework, where the target protein structure and a target-specific seed fragment from co-crystal structures can be the initial inputs. A random fragment from a purchasable fragment library can also be the input if a target-specific fragment is unavailable. Then a template-based forward synthesis route prediction and molecule generation is performed in parallel using the Monte Carlo Tree Search (MCTS) method where, the subsequent fragments for molecule growth can again be obtained from a purchasable fragment library. The rewards for each iteration of MCTS are computed using a drug-target affinity (DTA) model based on the docking pose of the generated reaction intermediates at the binding site of the target protein of interest. With the help of the proposed method, it is now possible to overcome one of the major obstacles posed to the AI-based drug design approaches through the ability of the method to design novel target-specific synthesizable molecules.

Accelerated Sequence Design of Star Block Copolymers: An Unbiased Exploration Strategy via Fusion of Molecular Dynamics Simulations and Machine Learning

Star block copolymers (s-BCPs) have potential applications as novel surfactants or amphiphiles for emulsification, compatibilization, chemical transformations, and separations. s-BCPs have chain architectures where three or more linear diblock copolymer arms comprised of two chemically distinct linear polymers, e.g., solvophobic and solvophilic chains, are covalently joined at one point. The chemical composition of each of the subunit polymer chains comprising the arms, their molecular weights, and the number of arms can be varied to tailor the surface and interfacial activity of these architecturally unique molecules. This makes identification of the optimal s-BCP design nontrivial as the total number of plausible s-BCP architectures is experimentally or computationally intractable. In this work, we use molecular dynamics (MD) simulations coupled with a reinforcement learning-based Monte Carlo tree search (MCTS) to identify s-BCP designs that minimize the interfacial tension between polar and nonpolar solvents. We first validate the MCTS approach for the design of small- and medium-sized s-BCPs and then use it to efficiently identify sequences of copolymer blocks for large-sized s-BCPs. The structural origins of interfacial tension in these systems are also identified by using the configurations obtained from MD simulations. Chemical insights into the arrangement of copolymer blocks that promote lower interfacial tension were mined using machine learning (ML) techniques. Overall, this work provides an efficient approach to solve design problems via fusion of simulations and ML and provides important groundwork for future experimental investigation of s-BCPs for various applications.

Deep reinforcement learning in chemistry: A review

Reinforcement learning (RL) has been applied to various domains in computational chemistry and has found wide-spread success. In this review, we first motivate the application of RL to chemistry and list some broad application domains, for example, molecule generation, geometry optimization, and retrosynthetic pathway search. We set up some of the formalism associated with reinforcement learning that should help the reader translate their chemistry problems into a form where RL can be used to solve them. We then discuss the solution formulations and algorithms proposed in recent literature for these problems, the advantages of one over the other, together with the necessary details of the RL algorithms they employ. This article should help the reader understand the state of RL applications in chemistry, learn about some relevant actively-researched open problems, gain insight into how RL can be used to approach them and hopefully inspire innovative RL applications in Chemistry.

Exploring Chemical Reaction Space with Machine Learning Models: Representation and Feature Perspective

Chemical reactions serve as foundational building blocks for organic chemistry and drug design. In the era of large AI models, data-driven approaches have emerged to innovate the design of novel reactions, optimize existing ones for higher yields, and discover new pathways for synthesizing chemical structures comprehensively. To effectively address these challenges with machine learning models, it is imperative to derive robust and informative representations or engage in feature engineering using extensive data sets of reactions. This work aims to provide a comprehensive review of established reaction featurization approaches, offering insights into the selection of representations and the design of features for a wide array of tasks. The advantages and limitations of employing SMILES, molecular fingerprints, molecular graphs, and physics-based properties are meticulously elaborated. Solutions to bridge the gap between different representations will also be critically evaluated. Additionally, we introduce a new frontier in chemical reaction pretraining, holding promise as an innovative yet unexplored avenue.

Adaptive habitat biogeography-based optimizer for optimizing deep CNN hyperparameters in image classification

Deep Convolutional Neural Networks (DCNNs) have shown remarkable success in image classification tasks, but optimizing their hyperparameters can be challenging due to their complex structure. This paper develops the Adaptive Habitat Biogeography-Based Optimizer (AHBBO) for tuning the hyperparameters of DCNNs in image classification tasks. In complicated optimization problems, the BBO suffers from premature convergence and insufficient exploration. In this regard, an adaptable habitat is presented as a solution to these problems; it would permit variable habitat sizes and regulated mutation. Better optimization performance and a greater chance of finding high-quality solutions across a wide range of problem domains are the results of this modification's increased exploration and population diversity. AHBBO is tested on 53 benchmark optimization functions and demonstrates its effectiveness in improving initial stochastic solutions and converging faster to the optimum. Furthermore, DCNN-AHBBO is compared to 23 well-known image classifiers on nine challenging image classification problems and shows superior performance in reducing the error rate by up to 5.14%. Our proposed algorithm outperforms 13 benchmark classifiers in 87 out of 95 evaluations, providing a high-performance and reliable solution for optimizing DNNs in image classification tasks. This research contributes to the field of deep learning by proposing a new optimization algorithm that can improve the efficiency of deep neural networks in image classification.

Computational aptamer design for spike glycoprotein (S) (SARS CoV-2) detection with an electrochemical aptasensor

A new bioinformatic platform (APTERION) was used to design in a short time and with high specificity an aptamer for the detection of the spike protein, a structural protein of SARS-CoV-2 virus, responsible for the COVID-19 pandemic. The aptamer concentration on the carbon electrode surface was optimized using static contact angle and fluorescence method, while specificity was tested using differential pulse voltammetry (DPV) associated to carbon screen-printed electrodes. The data obtained demonstrated the good features of the aptamer which could be used to create a rapid method for the detection of SARS-CoV-2 virus. In fact, it is specific for spike also when tested against bovine serum albumin and lysozyme, competitor proteins if saliva is used as sample to test for the virus presence. Spectrofluorometric characterization allowed to measure the amount of aptamer present on the carbon electrode surface, while DPV measurements proved the affinity of the aptamer towards the spike protein and gave quantitative results. The acquired data allowed to conclude that the APTERION bioinformatic platform is a good method for aptamer design for rapidity and specificity. KEY POINTS: • Spike protein detection using an electrochemical biosensor • Aptamer characterization by contact angle and fluorescent measurements on electrode surface • Computational design of specific aptamers to speed up the aptameric sequence time.

VGAE-MCTS: A New Molecular Generative Model Combining the Variational Graph Auto-Encoder and Monte Carlo Tree Search

Molecular generation is crucial for advancing drug discovery, materials science, and chemical exploration. It expedites the search for new drug candidates, facilitates tailored material creation, and enhances our understanding of molecular diversity. By employing artificial intelligence techniques such as molecular generative models based on molecular graphs, researchers have tackled the challenge of identifying efficient molecules with desired properties. Here, we propose a new molecular generative model combining a graph-based deep neural network and a reinforcement learning technique. We evaluated the validity, novelty, and optimized physicochemical properties of the generated molecules. Importantly, the model explored uncharted regions of chemical space, allowing for the efficient discovery and design of new molecules. This innovative approach has considerable potential to revolutionize drug discovery, materials science, and chemical research for accelerating scientific innovation. By leveraging advanced techniques and exploring previously unexplored chemical spaces, this study offers promising prospects for the efficient discovery and design of new molecules in the field of drug development.

Adaptive algorithms for shaping behavior

Dogs and laboratory mice are commonly trained to perform complex tasks by guiding them through a curriculum of simpler tasks ('shaping'). What are the principles behind effective shaping strategies? Here, we propose a machine learning framework for shaping animal behavior, where an autonomous teacher agent decides its student's task based on the student's transcript of successes and failures on previously assigned tasks. Using autonomous teachers that plan a curriculum in a common sequence learning task, we show that near-optimal shaping algorithms adaptively alternate between simpler and harder tasks to carefully balance reinforcement and extinction. Based on this intuition, we derive an adaptive shaping heuristic with minimal parameters, which we show is near-optimal on the sequence learning task and robustly trains deep reinforcement learning agents on navigation tasks that involve sparse, delayed rewards. Extensions to continuous curricula are explored. Our work provides a starting point towards a general computational framework for shaping animal behavior.

Student of Games: A unified learning algorithm for both perfect and imperfect information games

Games have a long history as benchmarks for progress in artificial intelligence. Approaches using search and learning produced strong performance across many perfect information games, and approaches using game-theoretic reasoning and learning demonstrated strong performance for specific imperfect information poker variants. We introduce Student of Games, a general-purpose algorithm that unifies previous approaches, combining guided search, self-play learning, and game-theoretic reasoning. Student of Games achieves strong empirical performance in large perfect and imperfect information games-an important step toward truly general algorithms for arbitrary environments. We prove that Student of Games is sound, converging to perfect play as available computation and approximation capacity increases. Student of Games reaches strong performance in chess and Go, beats the strongest openly available agent in heads-up no-limit Texas hold'em poker, and defeats the state-of-the-art agent in Scotland Yard, an imperfect information game that illustrates the value of guided search, learning, and game-theoretic reasoning.

Gargoyles: An Open Source Graph-Based Molecular Optimization Method Based on Deep Reinforcement Learning

Automatic optimization methods for compounds in the vast compound space are important for drug discovery and material design. Several machine learning-based molecular generative models for drug discovery have been proposed, but most of these methods generate compounds from scratch and are not suitable for exploring and optimizing user-defined compounds. In this study, we developed a compound optimization method based on molecular graphs using deep reinforcement learning. This method searches for compounds on a fragment-by-fragment basis and at high density by generating fragments to be added atom by atom. Experimental results confirmed that the quantum electrodynamics (QED), the optimization target set in this study, was enhanced by searching around the starting compound. As a use case, we successfully enhanced the activity of a compound by targeting dopamine receptor D2 (DRD2). This means that the generated compounds are not structurally dissimilar from the starting compounds, as well as increasing their activity, indicating that this method is suitable for optimizing molecules from a given compound. The source code is available at https://github.com/sekijima-lab/GARGOYLES.

A Systematic Review on Password Guessing Tasks

Recently, many password guessing algorithms have been proposed, seriously threatening cyber security. In this paper, we systematically review over thirty methods for password guessing published between 2016 and 2023. First, we introduce a taxonomy for classifying the existing methods into trawling guessing and targeted guessing. Second, we present an extensive benchmark dataset that can assist researchers and practitioners in successive works. Third, we conduct a bibliometric analysis to present trends in this field and cross-citation between reviewed papers. Further, we discuss the open challenges of password guessing in terms of diverse application scenarios, guessing efficiency, and the combination of traditional and deep learning methods. Finally, this review presents future research directions to guide successive research and development of password guessing.

Hybrid Reinforcement Learning for Power Transmission Network Self-Healing Considering Wind Power

Transmission network self-healing considering uncertain wind power becomes crucial with increasing penetration of wind power. A hybrid reinforcement learning (HRL) method combining offline self-learning with online Monte Carlo tree search (MCTS) is designed to deal with the strong uncertainty induced by wind power restoration. The HRL method trains a policy network with offline self-learning based on historical wind and transmission system data. It then applies the policy network to guide MCTS to realize step-by-step transmission network self-healing based on real-time and forecast data in different wind power scenarios. Besides, a model predictive control method for active power dispatch is proposed to improve wind power generation credibility during self-healing. Simulation results of both test and real-life power systems demonstrate that the proposed method can realize online transmission system self-healing reliably. Comparisons among different reinforcement learning methods indicate that the number of scenarios dominated by HRL is more than twice that dominated by MCTS and a dozen times that dominated by deep Q-network. Meanwhile, the online method is more flexible in uncertain wind power scenarios than optimization methods.

PepScaf: Harnessing Machine Learning with In Vitro Selection toward De Novo Macrocyclic Peptides against IL-17C/IL-17RE Interaction

The combination of library-based screening and artificial intelligence (AI) has been accelerating the discovery and optimization of hit ligands. However, the potential of AI to assist in de novo macrocyclic peptide ligand discovery has yet to be fully explored. In this study, an integrated AI framework called PepScaf was developed to extract the critical scaffold relative to bioactivity based on a vast dataset from an initial in vitro selection campaign against a model protein target, interleukin-17C (IL-17C). Taking the generated scaffold, a focused macrocyclic peptide library was rationally constructed to target IL-17C, yielding over 20 potent peptides that effectively inhibited IL-17C/IL-17RE interaction. Notably, the top two peptides displayed exceptional potency with IC50 values of 1.4 nM. This approach presents a viable methodology for more efficient macrocyclic peptide discovery, offering potential time and cost savings. Additionally, this is also the first report regarding the discovery of macrocyclic peptides against IL-17C/IL-17RE interaction.

Survey of Time Series Data Generation in IoT

Nowadays, with the rapid growth of the internet of things (IoT), massive amounts of time series data are being generated. Time series data play an important role in scientific and technological research for conducting experiments and studies to obtain solid and convincing results. However, due to privacy restrictions, limited access to time series data is always an obstacle. Moreover, the limited available open source data are often not suitable because of a small quantity and insufficient dimensionality and complexity. Therefore, time series data generation has become an imperative and promising solution. In this paper, we provide an overview of classical and state-of-the-art time series data generation methods in IoT. We classify the time series data generation methods into four major categories: rule-based methods, simulation-model-based methods, traditional machine-learning-based methods, and deep-learning-based methods. For each category, we first illustrate its characteristics and then describe the principles and mechanisms of the methods. Finally, we summarize the challenges and future directions of time series data generation in IoT. The systematic classification and evaluation will be a valuable reference for researchers in the time series data generation field.

Artificial intelligence inspired design of non-isothermal aging for γ-γ' two-phase, Ni-Al alloys

In this paper, a state-of-the-art Artificial Intelligence (AI) technique is used for a precipitation hardening of Ni-based alloy to predict more flexible non-isothermal aging (NIA) and to examine the possible routes for the enhancement in strength that may be practically achieved. Additionally, AI is used to integrate with Materials Integration by Network Technology, which is a computational workflow utilized to model the microstructure evolution and evaluate the 0.2% proof stress for isothermal aging and NIA. As a result, it is possible to find enhanced 0.2% proof stress for NIA for a fixed time of 10 min compared to the isothermal aging benchmark. The entire search space for aging scheduling was ~ 3 billion. Out of 1620 NIA schedules, we succeeded in designing the 110 NIA schedules that outperformed the isothermal aging benchmark. Interestingly, it is found that early-stage high-temperature aging for a shorter time increases the γ' precipitate size up to the critical size and later aging at lower temperature increases the γ' fraction with no anomalous change in γ' size. Therefore, employing this essence from AI, we designed an optimum aging route in which we attained an outperformed 0.2% proof stress to AI-designed NIA routes.

Artificial intelligence for video game visualization, advancements, benefits and challenges

In recent years, the field of artificial intelligence (AI) has witnessed remarkable progress and its applications have extended to the realm of video games. The incorporation of AI in video games enhances visual experiences, optimizes gameplay and fosters more realistic and immersive environments. In this review paper, we systematically explore the diverse applications of AI in video game visualization, encompassing machine learning algorithms for character animation, terrain generation and lighting effects following the PRISMA guidelines as our review methodology. Furthermore, we discuss the benefits, challenges and ethical implications associated with AI in video game visualization as well as the potential future trends. We anticipate that the future of AI in video gaming will feature increasingly sophisticated and realistic AI models, heightened utilization of machine learning and greater integration with other emerging technologies leading to more engaging and personalized gaming experiences.

Molecular Graph Generation by Decomposition and Reassembling

Designing molecular structures with desired chemical properties is an essential task in drug discovery and materials design. However, finding molecules with the optimized desired properties is still a challenging task due to combinatorial explosion of the candidate space of molecules. Here we propose a novel decomposition-and-reassembling-based approach, which does not include any optimization in hidden space, and our generation process is highly interpretable. Our method is a two-step procedure: In the first decomposition step, we apply frequent subgraph mining to a molecular database to collect a smaller size of subgraphs as building blocks of molecules. In the second reassembling step, we search desirable building blocks guided via reinforcement learning and combine them to generate new molecules. Our experiments show that our method not only can find better molecules in terms of two standard criteria, the penalized log P and druglikeness, but also can generate drug molecules showing the valid intermediate molecules.

A Comprehensive Survey on Multi-Agent Reinforcement Learning for Connected and Automated Vehicles

Connected and automated vehicles (CAVs) require multiple tasks in their seamless maneuverings. Some essential tasks that require simultaneous management and actions are motion planning, traffic prediction, traffic intersection management, etc. A few of them are complex in nature. Multi-agent reinforcement learning (MARL) can solve complex problems involving simultaneous controls. Recently, many researchers applied MARL in such applications. However, there is a lack of extensive surveys on the ongoing research to identify the current problems, proposed methods, and future research directions in MARL for CAVs. This paper provides a comprehensive survey on MARL for CAVs. A classification-based paper analysis is performed to identify the current developments and highlight the various existing research directions. Finally, the challenges in current works are discussed, and some potential areas are given for exploration to overcome those challenges. Future readers will benefit from this survey and can apply the ideas and findings in their research to solve complex problems.

Navigation between initial and desired community states using shortcuts

Ecological management problems often involve navigating from an initial to a desired community state. We ask whether navigation without brute-force additions and deletions of species is possible via: adding/deleting a small number of individuals of a species, changing the environment, and waiting. Navigation can yield direct paths (single sequence of actions) or shortcut paths (multiple sequences of actions with lower cost than a direct path). We ask (1) when is non-brute-force navigation possible?; (2) do shortcuts exist and what are their properties?; and (3) what heuristics predict shortcut existence? Using a state diagram framework applied to several empirical datasets, we show that (1) non-brute-force navigation is only possible between some state pairs, (2) shortcuts exist between many state pairs; and (3) changes in abundance and richness are the strongest predictors of shortcut existence, independent of dataset and algorithm choices. State diagrams thus unveil hidden strategies for manipulating species coexistence and efficiently navigating between states.

A Study on Optimization of Noise Reduction of Powered Vehicle Seat Movement Using Brushless Direct-Current Motor

In this paper, an optimal design model was developed to reduce noise and secure the torque performance of a brushless direct-current motor used in the seat of an autonomous vehicle. An acoustic model using finite elements was developed and verified through the noise test of the brushless direct-current motor. In order to reduce noise in the brushless direct-current motor and obtain a reliable optimization geometry of noiseless seat motion, parametric analysis was performed through the design of experiments and Monte Carlo statistical analysis. The slot depth, stator tooth width, slot opening, radial depth, and undercut angle of the brushless direct-current motor were selected as design parameters for design parameter analysis. Then, a non-linear prediction model was used to determine the optimal slot depth and stator tooth width to maintain the drive torque and minimize the sound pressure level at 23.26 dB or lower. The Monte Carlo statistical method was used to minimize the deviation of the sound pressure level caused by the production deviation of the design parameters. The result is that the SPL was 23.00-23.50 dB with a confidence level of approximately 99.76% when the level of production quality control was set at 3σ.

Data-Driven Robotic Manipulation of Cloth-like Deformable Objects: The Present, Challenges and Future Prospects

Manipulating cloth-like deformable objects (CDOs) is a long-standing problem in the robotics community. CDOs are flexible (non-rigid) objects that do not show a detectable level of compression strength while two points on the article are pushed towards each other and include objects such as ropes (1D), fabrics (2D) and bags (3D). In general, CDOs' many degrees of freedom (DoF) introduce severe self-occlusion and complex state-action dynamics as significant obstacles to perception and manipulation systems. These challenges exacerbate existing issues of modern robotic control methods such as imitation learning (IL) and reinforcement learning (RL). This review focuses on the application details of data-driven control methods on four major task families in this domain: cloth shaping, knot tying/untying, dressing and bag manipulation. Furthermore, we identify specific inductive biases in these four domains that present challenges for more general IL and RL algorithms.

Difficulty-skill balance does not affect engagement and enjoyment: a pre-registered study using artificial intelligence-controlled difficulty

How does the difficulty of a task affect people's enjoyment and engagement? Intrinsic motivation and flow theories posit a 'goldilocks' optimum where task difficulty matches performer skill, yet current work is confounded by questionable measurement practices and lacks scalable methods to manipulate objective difficulty-skill ratios. We developed a two-player tactical game test suite with an artificial intelligence (AI)-controlled opponent that uses a variant of the Monte Carlo Tree Search algorithm to precisely manipulate difficulty-skill ratios. A pre-registered study (n = 311) showed that our AI produced targeted difficulty-skill ratios without participants noticing the manipulation, yet different ratios had no significant impact on enjoyment or engagement. This indicates that difficulty-skill balance does not always affect engagement and enjoyment, but that games with AI-controlled difficulty provide a useful paradigm for rigorous future work on this issue.

Artificial Intelligence and Machine Learning Technology Driven Modern Drug Discovery and Development

The discovery and advances of medicines may be considered as the ultimate relevant translational science effort that adds to human invulnerability and happiness. But advancing a fresh medication is a quite convoluted, costly, and protracted operation, normally costing USD ~2.6 billion and consuming a mean time span of 12 years. Methods to cut back expenditure and hasten new drug discovery have prompted an arduous and compelling brainstorming exercise in the pharmaceutical industry. The engagement of Artificial Intelligence (AI), including the deep-learning (DL) component in particular, has been facilitated by the employment of classified big data, in concert with strikingly reinforced computing prowess and cloud storage, across all fields. AI has energized computer-facilitated drug discovery. An unrestricted espousing of machine learning (ML), especially DL, in many scientific specialties, and the technological refinements in computing hardware and software, in concert with various aspects of the problem, sustain this progress. ML algorithms have been extensively engaged for computer-facilitated drug discovery. DL methods, such as artificial neural networks (ANNs) comprising multiple buried processing layers, have of late seen a resurgence due to their capability to power automatic attribute elicitations from the input data, coupled with their ability to obtain nonlinear input-output pertinencies. Such features of DL methods augment classical ML techniques which bank on human-contrived molecular descriptors. A major part of the early reluctance concerning utility of AI in pharmaceutical discovery has begun to melt, thereby advancing medicinal chemistry. AI, along with modern experimental technical knowledge, is anticipated to invigorate the quest for new and improved pharmaceuticals in an expeditious, economical, and increasingly compelling manner. DL-facilitated methods have just initiated kickstarting for some integral issues in drug discovery. Many technological advances, such as "message-passing paradigms", "spatial-symmetry-preserving networks", "hybrid de novo design", and other ingenious ML exemplars, will definitely come to be pervasively widespread and help dissect many of the biggest, and most intriguing inquiries. Open data allocation and model augmentation will exert a decisive hold during the progress of drug discovery employing AI. This review will address the impending utilizations of AI to refine and bolster the drug discovery operation.

Directive Explanations for Actionable Explainability in Machine Learning Applications

In this paper, we show that explanations of decisions made by machine learning systems can be improved by not only explaining why a decision was made but also by explaining how an individual could obtain their desired outcome. We formally define the concept of directive explanations (those that offer specific actions an individual could take to achieve their desired outcome), introduce two forms of directive explanations (directive-specific and directive-generic), and describe how these can be generated computationally. We investigate people’s preference for and perception towards directive explanations through two online studies, one quantitative and the other qualitative, each covering two domains (the credit scoring domain and the employee satisfaction domain). We find a significant preference for both forms of directive explanations compared to non-directive counterfactual explanations. However, we also find that preferences are affected by many aspects, including individual preferences and social factors. We conclude that deciding what type of explanation to provide requires information about the recipients and other contextual information. This reinforces the need for a human-centred and context-specific approach to explainable AI.

Probing Conformational Dynamics of Antibodies with Geometric Simulations

This chapter describes the application of constrained geometric simulations for prediction of antibody structural dynamics. We utilize constrained geometric simulations method FRODAN, which is a low computational complexity alternative to molecular dynamics (MD) simulations that can rapidly explore flexible motions in protein structures. FRODAN is highly suited for conformational dynamics analysis of large proteins, complexes, intrinsically disordered proteins, and dynamics that occurs on longer biologically relevant time scales that are normally inaccessible to classical MD simulations. This approach predicts protein dynamics at an all-atom scale while retaining realistic covalent bonding, maintaining dihedral angles in energetically good conformations while avoiding steric clashes in addition to performing other geometric and stereochemical criteria checks. In this chapter, we apply FRODAN to showcase its applicability for probing functionally relevant dynamics of IgG2a, including large-amplitude domain-domain motions and motions of complementarity determining region (CDR) loops. As was suggested in previous experimental studies, our simulations show that antibodies can explore a large range of conformational space.

SanJeeVni: Secure UAV-Envisioned Massive Vaccine Distribution for COVID-19 Underlying 6G Network

Recently, unmanned aerial vehicles (UAVs) are deployed in Novel Coronavirus Disease-2019 (COVID-19) vaccine distribution process. To address issues of fake vaccine distribution, real-time massive UAV monitoring and control at nodal centers (NCs), the authors propose SanJeeVni, a blockchain (BC)-assisted UAV vaccine distribution at the backdrop of sixth-generation (6G) enhanced ultra-reliable low latency communication (6G-eRLLC) communication. The scheme considers user registration, vaccine request, and distribution through a public Solana BC setup, which assures a scalable transaction rate. Based on vaccine requests at production setups, UAV swarms are triggered with vaccine delivery to NCs. An intelligent edge offloading scheme is proposed to support UAV coordinates and routing path setups. The scheme is compared against fifth-generation (5G) uRLLC communication. In the simulation, we achieve and 86% improvement in service latency, 12.2% energy reduction of UAV with 76.25% more UAV coverage in 6G-eRLLC, and a significant improvement of [Formula: see text]% in storage cost against the Ethereum network, which indicates the scheme efficacy in practical setups.

Planning spatial networks with Monte Carlo tree search

We tackle the problem of goal-directed graph construction: given a starting graph, finding a set of edges whose addition maximally improves a global objective function. This problem emerges in many transportation and infrastructure networks that are of critical importance to society. We identify two significant shortcomings of present reinforcement learning methods: their exclusive focus on topology to the detriment of spatial characteristics (which are known to influence the growth and density of links), as well as the rapid growth in the action spaces and costs of model training. Our formulation as a deterministic Markov decision process allows us to adopt the Monte Carlo tree search framework, an artificial intelligence decision-time planning method. We propose improvements over the standard upper confidence bounds for trees (UCT) algorithm for this family of problems that addresses their single-agent nature, the trade-off between the cost of edges and their contribution to the objective, and an action space linear in the number of nodes. Our approach yields substantial improvements over UCT for increasing the efficiency and attack resilience of synthetic networks and real-world Internet backbone and metro systems, while using a wall clock time budget similar to other search-based algorithms. We also demonstrate that our approach scales to significantly larger networks than previous reinforcement learning methods, since it does not require training a model.

MetaGlyph: Automatic Generation of Metaphoric Glyph-based Visualization

Glyph-based visualization achieves an impressive graphic design when associated with comprehensive visual metaphors, which help audiences effectively grasp the conveyed information through revealing data semantics. However, creating such metaphoric glyph-based visualization (MGV) is not an easy task, as it requires not only a deep understanding of data but also professional design skills. This paper proposes MetaGlyph, an automatic system for generating MGVs from a spreadsheet. To develop MetaGlyph, we first conduct a qualitative analysis to understand the design of current MGVs from the perspectives of metaphor embodiment and glyph design. Based on the results, we introduce a novel framework for generating MGVs by metaphoric image selection and an MGV construction. Specifically, MetaGlyph automatically selects metaphors with corresponding images from online resources based on the input data semantics. We then integrate a Monte Carlo tree search algorithm that explores the design of an MGV by associating visual elements with data dimensions given the data importance, semantic relevance, and glyph non-overlap. The system also provides editing feedback that allows users to customize the MGVs according to their design preferences. We demonstrate the use of MetaGlyph through a set of examples, one usage scenario, and validate its effectiveness through a series of expert interviews.

Stochastic optimization for vaccine and testing kit allocation for the COVID-19 pandemic

We present a formal mathematical modeling framework for a multi-agent sequential decision problem during an epidemic. The problem is formulated as a collaboration between a vaccination agent and learning agent to allocate stockpiles of vaccines and tests to a set of zones under various types of uncertainty. The model is able to capture passive information processes and maintain beliefs over the uncertain state of the world. We designed a parameterized direct lookahead approximation which is robust and scalable under different scenarios, resource scarcity, and beliefs about the environment. We design a test allocation policy designed to capture the value of information and demonstrate that it outperforms other learning policies when there is an extreme shortage of resources (information is scarce). We simulate the model with two scenarios including a resource allocation problem to each state in the United States and another for the nursing homes in Nevada. The US example demonstrates the scalability of the model and the nursing home example demonstrates the robustness under extreme resource shortages.

Chemical reaction networks and opportunities for machine learning

Chemical reaction networks (CRNs), defined by sets of species and possible reactions between them, are widely used to interrogate chemical systems. To capture increasingly complex phenomena, CRNs can be leveraged alongside data-driven methods and machine learning (ML). In this Perspective, we assess the diverse strategies available for CRN construction and analysis in pursuit of a wide range of scientific goals, discuss ML techniques currently being applied to CRNs and outline future CRN-ML approaches, presenting scientific and technical challenges to overcome.

Erato: Cooperative Data Story Editing via Fact Interpolation

As an effective form of narrative visualization, visual data stories are widely used in data-driven storytelling to communicate complex insights and support data understanding. Although important, they are difficult to create, as a variety of interdisciplinary skills, such as data analysis and design, are required. In this work, we introduce Erato, a human-machine cooperative data story editing system, which allows users to generate insightful and fluent data stories together with the computer. Specifically, Erato only requires a number of keyframes provided by the user to briefly describe the topic and structure of a data story. Meanwhile, our system leverages a novel interpolation algorithm to help users insert intermediate frames between the keyframes to smooth the transition. We evaluated the effectiveness and usefulness of the Erato system via a series of evaluations including a Turing test, a controlled user study, a performance validation, and interviews with three expert users. The evaluation results showed that the proposed interpolation technique was able to generate coherent story content and help users create data stories more efficiently.

Strategic argumentation dialogues for persuasion: Framework and experiments based on modelling the beliefs and concerns of the persuadee

Persuasion is an important and yet complex aspect of human intelligence. When undertaken through dialogue, the deployment of good arguments, and therefore counterarguments, clearly has a significant effect on the ability to be successful in persuasion. Two key dimensions for determining whether an argument is “good” in a particular dialogue are the degree to which the intended audience believes the argument and counterarguments, and the impact that the argument has on the concerns of the intended audience. In this paper, we present a framework for modelling persuadees in terms of their beliefs and concerns, and for harnessing these models in optimizing the choice of move in persuasion dialogues. Our approach is based on the Monte Carlo Tree Search which allows optimization in real-time. We provide empirical results of a study with human participants that compares an automated persuasion system based on this technology with a baseline system that does not take the beliefs and concerns into account in its strategy.

SofaGym: An Open Platform for Reinforcement Learning Based on Soft Robot Simulations

OpenAI Gym is one of the standard interfaces used to train Reinforcement Learning (RL) Algorithms. The Simulation Open Framework Architecture (SOFA) is a physics-based engine that is used for soft robotics simulation and control based on real-time models of deformation. The aim of this article is to present SofaGym, an open-source software to create OpenAI Gym interfaces, called environments, out of soft robot digital twins. The link between soft robotics and RL offers new challenges for both fields: representation of the soft robot in an RL context, complex interactions with the environment, use of specific mechanical tools to control soft robots, transfer of policies learned in simulation to the real world, etc. The article presents the large possible uses of SofaGym to tackle these challenges by using RL and planning algorithms. This publication contains neither new algorithms nor new models but proposes a new platform, open to the community, that offers non existing possibilities of coupling RL to physics-based simulation of soft robots. We present 11 environments, representing a wide variety of soft robots and applications; we highlight the challenges showcased by each environment. We propose methods of solving the task using traditional control, RL, and planning and point out research perspectives using the platform.

Machine learning overcomes human bias in the discovery of self-assembling peptides

Peptide materials have a wide array of functions, from tissue engineering and surface coatings to catalysis and sensing. Tuning the sequence of amino acids that comprise the peptide modulates peptide functionality, but a small increase in sequence length leads to a dramatic increase in the number of peptide candidates. Traditionally, peptide design is guided by human expertise and intuition and typically yields fewer than ten peptides per study, but these approaches are not easily scalable and are susceptible to human bias. Here we introduce a machine learning workflow-AI-expert-that combines Monte Carlo tree search and random forest with molecular dynamics simulations to develop a fully autonomous computational search engine to discover peptide sequences with high potential for self-assembly. We demonstrate the efficacy of the AI-expert to efficiently search large spaces of tripeptides and pentapeptides. The predictability of AI-expert performs on par or better than our human experts and suggests several non-intuitive sequences with high self-assembly propensity, outlining its potential to overcome human bias and accelerate peptide discovery.

Deep Reinforcement Learning for the Detection of Abnormal Data in Smart Meters

The rapidly growing power data in smart grids have created difficulties in security management. The processing of large-scale power data with the use of artificial intelligence methods has become a hotspot research topic. Considering the early warning detection problem of smart meters, this paper proposes an abnormal data detection network based on Deep Reinforcement Learning, which includes a main network and a target network composed of deep learning networks. This work uses the greedy policy algorithm to find the action of the maximum value of Q based on the Q-learning method to obtain the optimal calculation policy. It also uses the reward value and discount factor to optimize the target value. In particular, this study uses the fuzzy c-means method to predict the future state information value, which improves the computational accuracy of the Deep Reinforcement Learning model. The experimental results show that compared with the traditional smart meter data anomaly detection method, the proposed model improves the accuracy of meter data anomaly detection.

Chemistry-informed molecular graph as reaction descriptor for machine-learned retrosynthesis planning

Infusing "chemical wisdom" should improve the data-driven approaches that rely exclusively on historical synthetic data for automatic retrosynthesis planning. For this purpose, we designed a chemistry-informed molecular graph (CIMG) to describe chemical reactions. A collection of key information that is most relevant to chemical reactions is integrated in CIMG:NMR chemical shifts as vertex features, bond dissociation energies as edge features, and solvent/catalyst information as global features. For any given compound as a target, a product CIMG is generated and exploited by a graph neural network (GNN) model to choose reaction template(s) leading to this product. A reactant CIMG is then inferred and used in two GNN models to select appropriate catalyst and solvent, respectively. Finally, a fourth GNN model compares the two CIMG descriptors to check the plausibility of the proposed reaction. A reaction vector is obtained for every molecule in training these models. The chemical wisdom of reaction propensity contained in the pretrained reaction vectors is exploited to autocategorize molecules/reactions and to accelerate Monte Carlo tree search (MCTS) for multistep retrosynthesis planning. Full synthetic routes with recommended catalysts/solvents are predicted efficiently using this CIMG-based approach.

De Novo Design of Molecules with Low Hole Reorganization Energy Based on a Quarter-Million Molecule DFT Screen: Part 2

Organic semiconductors have many desirable properties including improved manufacturing and flexible mechanical properties. Due to the vastness of chemical space, it is essential to efficiently explore chemical space when designing new materials, including through the use of generative techniques. New generative machine learning methods for molecular design continue to be published in the literature at a significant rate but successfully adapting methods to new chemistry and problem domains remains difficult. These challenges necessitate continual method evaluation to probe method viability for use in alternative applications not covered in the original works. In continuation of our previous work, we evaluate four additional machine-learning-based de novo methods for generating molecules with high predicted hole mobility for use in semiconductor applications. The four generative methods evaluated here are (1) Molecule Deep Q-Networks (MolDQN), which utilizes Deep-Q learning to directly optimize molecular structure graphs for desired properties instead of generating SMILES, (2) Graph-based Genetic Algorithm (GraphGA), which uses a genetic algorithm for optimization where crossovers and mutations are defined in terms of RDKit's reaction SMILES, (3) Generative Tensorial Reinforcement Learning (GENTRL), which is a variational autoencoder (VAE) with a learned prior distribution and optimized using reinforcement learning, and (4) Monte Carlo tree search exploration of chemical space in conjunction with a recurrent neural network (RNN) decoder (ChemTS). The generated molecules were evaluated using density functional theory (DFT) and we discovered better performing molecules with the GraphGA method compared to the other approaches.

MolSearch

Leveraging computational methods to generate small molecules with desired properties has been an active research area in the drug discovery field. Towards real-world applications, however, efficient generation of molecules that satisfy multiple property requirements simultaneously remains a key challenge. In this paper, we tackle this challenge using a search-based approach and propose a simple yet effective framework called MolSearch for multi-objective molecular generation (optimization). We show that given proper design and sufficient information, search-based methods can achieve performance comparable or even better than deep learning methods while being computationally efficient. Such efficiency enables massive exploration of chemical space given constrained computational resources. In particular, MolSearch starts with existing molecules and uses a two-stage search strategy to gradually modify them into new ones, based on transformation rules derived systematically and exhaustively from large compound libraries. We evaluate MolSearch in multiple benchmark generation settings and demonstrate its effectiveness and efficiency.

Branching Time Active Inference with Bayesian Filtering

Branching time active inference is a framework proposing to look at planning as a form of Bayesian model expansion. Its root can be found in active inference, a neuroscientific framework widely used for brain modeling, as well as in Monte Carlo tree search, a method broadly applied in the reinforcement learning literature. Up to now, the inference of the latent variables was carried out by taking advantage of the flexibility offered by variational message passing, an iterative process that can be understood as sending messages along the edges of a factor graph. In this letter, we harness the efficiency of an alternative method for inference, Bayesian filtering, which does not require the iteration of the update equations until convergence of the variational free energy. Instead, this scheme alternates between two phases: integration of evidence and prediction of future states. Both phases can be performed efficiently, and this provides a forty times speedup over the state of the art.