Short-term cognitive networks, flexible reasoning and nonsynaptic learning

Long short-term cognitive networks

In this paper, we present a recurrent neural system named long short-term cognitive networks (LSTCNs) as a generalization of the short-term cognitive network (STCN) model. Such a generalization is motivated by the difficulty of forecasting very long time series efficiently. The LSTCN model can be defined as a collection of STCN blocks, each processing a specific time patch of the (multivariate) time series being modeled. In this neural ensemble, each block passes information to the subsequent one in the form of weight matrices representing the prior knowledge. As a second contribution, we propose a deterministic learning algorithm to compute the learnable weights while preserving the prior knowledge resulting from previous learning processes. As a third contribution, we introduce a feature influence score as a proxy to explain the forecasting process in multivariate time series. The simulations using three case studies show that our neural system reports small forecasting errors while being significantly faster than state-of-the-art recurrent models.

Long-term Cognitive Network-based architecture for multi-label classification

This paper presents a neural system to deal with multi-label classification problems that might involve sparse features. The architecture of this model involves three sequential blocks with well-defined functions. The first block consists of a multilayered feed-forward structure that extracts hidden features, thus reducing the problem dimensionality. This block is useful when dealing with sparse problems. The second block consists of a Long-term Cognitive Network-based model that operates on features extracted by the first block. The activation rule of this recurrent neural network is modified to prevent the vanishing of the input signal during the recurrent inference process. The modified activation rule combines the neurons' state in the previous abstract layer (iteration) with the initial state. Moreover, we add a bias component to shift the transfer functions as needed to obtain good approximations. Finally, the third block consists of an output layer that adapts the second block's outputs to the label space. We propose a backpropagation learning algorithm that uses a squared hinge loss function to maximize the margins between labels to train this network. The results show that our model outperforms the state-of-the-art algorithms in most datasets.

Construction and Supervised Learning of Long-Term Grey Cognitive Networks

Modeling a real-world system by means of a neural model involves numerous challenges that range from formulating transparent knowledge representations to obtaining reliable simulation errors. However, that knowledge is often difficult to formalize in a precise way using crisp numbers. In this paper, we present the long-term grey cognitive networks which expands the recently proposed long-term cognitive networks (LTCNs) with grey numbers. One advantage of our neural system is that it allows embedding knowledge into the network using weights and constricted neurons. In addition, we propose two procedures to construct the network in situations where only historical data are available, and a regularization method that is coupled with a nonsynaptic backpropagation algorithm. The results have shown that our proposal outperforms the LTCN model and other state-of-the-art methods in terms of accuracy.

Deterministic learning of hybrid Fuzzy Cognitive Maps and network reduction approaches

Hybrid artificial intelligence deals with the construction of intelligent systems by relying on both human knowledge and historical data records. In this paper, we approach this problem from a neural perspective, particularly when modeling and simulating dynamic systems. Firstly, we propose a Fuzzy Cognitive Map architecture in which experts are requested to define the interaction among the input neurons. As a second contribution, we introduce a fast and deterministic learning rule to compute the weights among input and output neurons. This parameterless learning method is based on the Moore-Penrose inverse and it can be performed in a single step. In addition, we discuss a model to determine the relevance of weights, which allows us to better understand the system. Last but not least, we introduce two calibration methods to adjust the model after the removal of potentially superfluous weights.

Nonsynaptic Error Backpropagation in Long-Term Cognitive Networks

We introduce a neural cognitive mapping technique named long-term cognitive network (LTCN) that is able to memorize long-term dependencies between a sequence of input and output vectors, especially in those scenarios that require predicting the values of multiple dependent variables at the same time. The proposed technique is an extension of a recently proposed method named short-term cognitive network that aims at preserving the expert knowledge encoded in the weight matrix while optimizing the nonlinear mappings provided by the transfer function of each neuron. A nonsynaptic, backpropagation-based learning algorithm powered by stochastic gradient descent is put forward to iteratively optimize four parameters of the generalized sigmoid transfer function associated with each neuron. Numerical simulations over 35 multivariate regression and pattern completion data sets confirm that the proposed LTCN algorithm attains statistically significant performance differences with respect to other well-known state-of-the-art methods.