Variance aware reward smoothing for deep reinforcement learning

Active Inference and Reinforcement Learning: A Unified Inference on Continuous State and Action Spaces Under Partial Observability

Reinforcement learning (RL) has garnered significant attention for developing decision-making agents that aim to maximize rewards, specified by an external supervisor, within fully observable environments. However, many real-world problems involve partial or noisy observations, where agents cannot access complete and accurate information about the environment. These problems are commonly formulated as partially observable Markov decision processes (POMDPs). Previous studies have tackled RL in POMDPs by either incorporating the memory of past actions and observations or by inferring the true state of the environment from observed data. Nevertheless, aggregating observations and actions over time becomes impractical in problems with large decision-making time horizons and high-dimensional spaces. Furthermore, inference-based RL approaches often require many environmental samples to perform well, as they focus solely on reward maximization and neglect uncertainty in the inferred state. Active inference (AIF) is a framework naturally formulated in POMDPs and directs agents to select actions by minimizing a function called expected free energy (EFE). This supplies reward-maximizing (or exploitative) behavior, as in RL, with information-seeking (or exploratory) behavior. Despite this exploratory behavior of AIF, its use is limited to problems with small time horizons and discrete spaces due to the computational challenges associated with EFE. In this article, we propose a unified principle that establishes a theoretical connection between AIF and RL, enabling seamless integration of these two approaches and overcoming their limitations in continuous space POMDP settings. We substantiate our findings with rigorous theoretical analysis, providing novel perspectives for using AIF in designing and implementing artificial agents. Experimental results demonstrate the superior learning capabilities of our method compared to other alternative RL approaches in solving partially observable tasks with continuous spaces. Notably, our approach harnesses information-seeking exploration, enabling it to effectively solve reward-free problems and rendering explicit task reward design by an external supervisor optional.

Reinforcement learning for systems pharmacology-oriented and personalized drug design

INTRODUCTION

Many multi-genic systemic diseases such as neurological disorders, inflammatory diseases, and the majority of cancers do not have effective treatments yet. Reinforcement learning powered systems pharmacology is a potentially effective approach to designing personalized therapies for untreatable complex diseases.

AREAS COVERED

In this survey, state-of-the-art reinforcement learning methods and their latest applications to drug design are reviewed. The challenges on harnessing reinforcement learning for systems pharmacology and personalized medicine are discussed. Potential solutions to overcome the challenges are proposed.

EXPERT OPINION

In spite of successful application of advanced reinforcement learning techniques to target-based drug discovery, new reinforcement learning strategies are needed to address systems pharmacology-oriented personalized de novo drug design.

Exploring Multiple Application Scenarios of Visual Communication Course Using Deep Learning Under the Digital Twins

The emergence of intelligent technology has brought a particular impact and allows for virtuality-reality interaction in the educational field. In particular, digital twins (DTs) feature virtuality-reality symbiosis, solid virtual simulation, and high real-time interaction. It has also seen extended applications to the field of education. This study optimizes the design of the visual communication (Viscom) course based on the deep learning (DL) algorithm. Firstly, the theory of DL is analyzed following the relevant literature, and the typical DL networks, network structures, and related algorithms are introduced. Secondly, Viscom technology is expounded, and DL technology is applied to the Viscom course. Then, the applicability and feasibility of DL in the Viscom course are analyzed through a questionnaire survey (QS) design by collecting students’ attitudes towards Viscom courses before and after the experiment. After introducing DL into the Viscom course, the results show that students’ learning interest and satisfaction with the practical knowledge mastery have increased. However, the satisfaction with theoretical knowledge mastery before practical courses has decreased; overall, the teaching effect of the Viscom course has been improved. Therefore, the introduction of DL into the DT-enabled Viscom can provide a reference for developing the Viscom course. The research content offers technical support (TS) for integrating DT technology and modern education.