Neural networks for estimating intrinsic dimension

Understanding autoencoders with information theoretic concepts

Despite their great success in practical applications, there is still a lack of theoretical and systematic methods to analyze deep neural networks. In this paper, we illustrate an advanced information theoretic methodology to understand the dynamics of learning and the design of autoencoders, a special type of deep learning architectures that resembles a communication channel. By generalizing the information plane to any cost function, and inspecting the roles and dynamics of different layers using layer-wise information quantities, we emphasize the role that mutual information plays in quantifying learning from data. We further suggest and also experimentally validate, for mean square error training, three fundamental properties regarding the layer-wise flow of information and intrinsic dimensionality of the bottleneck layer, using respectively the data processing inequality and the identification of a bifurcation point in the information plane that is controlled by the given data. Our observations have direct impact on the optimal design of autoencoders, the design of alternative feedforward training methods, and even in the problem of generalization.

Optimization of random searches on defective lattice networks

We study the general problem of how to search efficiently for targets randomly located on defective lattice networks--i.e., regular lattices which have some fraction of its nodes randomly removed. We consider large but finite triangular lattices and assume for the search dynamics that the walker chooses steps lengths lj from the power-law distribution P(lj) approximately lj(-mu) , with the exponent mu regulating the strategy of the search process. At each step lj, the searcher moves in straight lines and constantly looks within a detection radius of vision rv for the targets along the way. If there is contact with a defect, the movement stops and a new step length is chosen. Hence, the presence of defects decreases the efficiency of the overall process. We study numerically how three different aspects of the lattice influence the optimization of the search efficiency: (i) the type of boundary conditions, (ii) the concentration of targets and defects, and (iii) the category or class of search--destructive, nondestructive, or regenerative. Motivated by the results, we develop a type of mean-field model for the problem and obtain an analytical approximation for the search efficiency function. Finally we discuss, in the context of searches, how defective lattices compare with perfect lattices and with continuous environments.

Estimating nonlinear interdependences in dynamical systems using cellular nonlinear networks

We propose a method for estimating nonlinear interdependences between time series using cellular nonlinear networks. Our approach is based on the nonlinear dynamics of interacting nonlinear elements. We apply it to time series of coupled nonlinear model systems and to electroencephalographic time series from an epilepsy patient, and we show that an accurate approximation of symmetric and asymmetric realizations of a nonlinear interdependence measure can be achieved, thus allowing one to detect the strength and direction of couplings.

Data reduction in headspace analysis of blood and urine samples for robust bacterial identification

This paper demonstrates the application of chemical headspace analysis to the problem of classifying the presence of bacteria in biomedical samples by using computational tools. Blood and urine samples of disparate forms were analysed using a Cyrano Sciences C320 electronic nose together with an Agilent 4440 Chemosensor. The high dimensional data sets resulting from these devices present computational problems for parameter estimation of discriminant models. A variety of data reduction and pattern recognition techniques were employed in an attempt to optimise the classification process. A 100% successful classification rate for the blood data from the Agilent 4440 was achieved by combining a Sammon mapping with a radial basis function neural network. In comparison a successful classification rate of 80% was achieved for the urine data from the C320 which were analysed using a novel nonlinear time series model.

Estimating phase synchronization in dynamical systems using cellular nonlinear networks

We propose a method for estimating phase synchronization between time series using the parallel computing architecture of cellular nonlinear networks (CNN's). Applying this method to time series of coupled nonlinear model systems and to electroencephalographic time series from epilepsy patients, we show that an accurate approximation of the mean phase coherence R--a bivariate measure for phase synchronization--can be achieved with CNN's using polynomial-type templates.