Recurrent Neural Networks With External Addressable Long-Term and Working Memory for Learning Long-Term Dependences

A multi-memory-augmented network with a curvy metric method for video anomaly detection

Anomaly detection task in video mainly refers to identifying anomalous events that do not conform to the learned normal patterns in the inferring phase. However, the Euclidean metric used in the learning and inferring phase by the most of the existing methods, which cannot measure the difference between the different high-dimensional data reasonably, because the Euclidean distance between the different high-dimensional data will gradually become the same as the dimension increases. In this paper, we propose a Multi-Memory-Augmented dual-flow network with a new curvy metric method, to remove this shortcoming of Euclidean metric. To the best of our knowledge, this is the first work to detect abnormal events using this novel curvy metric. A large number of comparative experiments show that this novel curvy metric can be inserted in any neural network based on the Euclidean metric due to its independence and the migration experiment results. In addition, the powerful representation capacity of deep network allows to take abnormal frames as normal, we employ several memory units to the dual-flow network that considers the diversity of normal patterns explicitly, while lessening the representation capacity of dual-flow network. Our model is easy to be trained and robust to be applied. Extensive experiments on five publicly available datasets verify the validity of our method, which reflect in the robustness to the normal events diversity as well as the sensitivity to abnormal events.

Deep learning for volatility forecasting in asset management

Predicting volatility is a critical activity for taking risk- adjusted decisions in asset trading and allocation. In order to provide effective decision-making support, in this paper we investigate the profitability of a deep Long Short-Term Memory (LSTM) Neural Network for forecasting daily stock market volatility using a panel of 28 assets representative of the Dow Jones Industrial Average index combined with the market factor proxied by the SPY and, separately, a panel of 92 assets belonging to the NASDAQ 100 index. The Dow Jones plus SPY data are from January 2002 to August 2008, while the NASDAQ 100 is from December 2012 to November 2017. If, on the one hand, we expect that this evolutionary behavior can be effectively captured adaptively through the use of Artificial Intelligence (AI) flexible methods, on the other, in this setting, standard parametric approaches could fail to provide optimal predictions. We compared the volatility forecasts generated by the LSTM approach to those obtained through use of widely recognized benchmarks models in this field, in particular, univariate parametric models such as the Realized Generalized Autoregressive Conditionally Heteroskedastic (R-GARCH) and the Glosten–Jagannathan–Runkle Multiplicative Error Models (GJR-MEM). The results demonstrate the superiority of the LSTM over the widely popular R-GARCH and GJR-MEM univariate parametric methods, when forecasting in condition of high volatility, while still producing comparable predictions for more tranquil periods.

Demystifying Deep Learning in Predictive Spatiotemporal Analytics: An Information-Theoretic Framework

Deep learning has achieved incredible success over the past years, especially in various challenging predictive spatiotemporal analytics (PSTA) tasks, such as disease prediction, climate forecast, and traffic prediction, where intrinsic dependence relationships among data exist and generally manifest at multiple spatiotemporal scales. However, given a specific PSTA task and the corresponding data set, how to appropriately determine the desired configuration of a deep learning model, theoretically analyze the model's learning behavior, and quantitatively characterize the model's learning capacity remains a mystery. In order to demystify the power of deep learning for PSTA in a theoretically sound and explainable way, in this article, we provide a comprehensive framework for deep learning model design and information-theoretic analysis. First, we develop and demonstrate a novel interactively and integratively connected deep recurrent neural network (I²DRNN) model. I²DRNN consists of three modules: an input module that integrates data from heterogeneous sources; a hidden module that captures the information at different scales while allowing the information to flow interactively between layers; and an output module that models the integrative effects of information from various hidden layers to generate the output predictions. Second, to theoretically prove that our designed model can learn multiscale spatiotemporal dependence in PSTA tasks, we provide an information-theoretic analysis to examine the information-based learning capacity (i-CAP) of the proposed model. In so doing, we can tackle an important open question in deep learning, that is, how to determine the necessary and sufficient configurations of a designed deep learning model with respect to the given learning data sets. Third, to validate the I²DRNN model and confirm its i-CAP, we systematically conduct a series of experiments involving both synthetic data sets and real-world PSTA tasks. The experimental results show that the I²DRNN model outperforms both classical and state-of-the-art models on all data sets and PSTA tasks. More importantly, as readily validated, the proposed model captures the multiscale spatiotemporal dependence, which is meaningful in the real-world context. Furthermore, the model configuration that corresponds to the best performance on a given data set always falls into the range between the necessary and sufficient configurations, as derived from the information-theoretic analysis.

Machine learning for condensed matter physics

Condensed matter physics (CMP) seeks to understand the microscopic interactions of matter at the quantum and atomistic levels, and describes how these interactions result in both mesoscopic and macroscopic properties. CMP overlaps with many other important branches of science, such as chemistry, materials science, statistical physics, and high-performance computing. With the advancements in modern machine learning (ML) technology, a keen interest in applying these algorithms to further CMP research has created a compelling new area of research at the intersection of both fields. In this review, we aim to explore the main areas within CMP, which have successfully applied ML techniques to further research, such as the description and use of ML schemes for potential energy surfaces, the characterization of topological phases of matter in lattice systems, the prediction of phase transitions in off-lattice and atomistic simulations, the interpretation of ML theories with physics-inspired frameworks and the enhancement of simulation methods with ML algorithms. We also discuss in detail the main challenges and drawbacks of using ML methods on CMP problems, as well as some perspectives for future developments.