A model with an intrinsic property of learning higher order correlations

Multibranch Formal Neuron: An Internally Nonlinear Learning Unit

The transformation of synaptic input into action potential in nerve cells is strongly influenced by the morphology of the dendritic arbor as well as the synaptic efficacy map. The multiplicity of dendritic branches strikingly enables a single cell to act as a highly nonlinear processing element. Studies have also found functional synaptic clustering whereby synapses that encode a common sensory feature are spatially clustered together on the branches. Motivated by these findings, here we introduce a multibranch formal model of the neuron that can integrate synaptic inputs nonlinearly through collective action of its dendritic branches and yields synaptic clustering. An analysis in support of its use as a computational building block is offered. Also offered is an accompanying gradient descent-based learning algorithm. The model unit spans a wide spectrum of nonlinearities, including the parity problem, and can outperform the multilayer perceptron in generalizing to unseen data. The occurrence of synaptic clustering boosts the generalization efficiency of the unit, which may also be the answer for the puzzling ubiquity of synaptic clustering in the real neurons. Our theoretical analysis is backed up by simulations. The study could pave the way to new artificial neural networks.

Minimal Sign Representation of Boolean Functions: Algorithms and Exact Results for Low Dimensions

Boolean functions (BFs) are central in many fields of engineering and mathematics, such as cryptography, circuit design, and combinatorics. Moreover, they provide a simple framework for studying neural computation mechanisms of the brain. Many representation schemes for BFs exist to satisfy the needs of the domain they are used in. In neural computation, it is of interest to know how many input lines a neuron would need to represent a given BF. A common BF representation to study this is the so-called polynomial sign representation where [Formula: see text] and 1 are associated with true and false, respectively. The polynomial is treated as a real-valued function and evaluated at its parameters, and the sign of the polynomial is then taken as the function value. The number of input lines for the modeled neuron is exactly the number of terms in the polynomial. This letter investigates the minimum number of terms, that is, the minimum threshold density, that is sufficient to represent a given BF and more generally aims to find the maximum over this quantity for all BFs in a given dimension. With this work, for the first time exact results for four- and five-variable BFs are obtained, and strong bounds for six-variable BFs are derived. In addition, some connections between the sign representation framework and bent functions are derived, which are generally studied for their desirable cryptographic properties.

Sign-representation of Boolean functions using a small number of monomials

This paper presents a deterministic algorithm that can construct a higher-order neuron representation for an arbitrary n-variable Boolean function with a fan-in less than 0.75 x 2(n), and provides related theoretical results. When the logic constants True and False are identified by +1 and -1, an n-variable Boolean function is identified by a unique dichotomy of the n-dimensional hypercube. With this equivalence, all n-variable Boolean functions can be uniquely represented by linear combinations of monomials, the products of input variables. A polynomial function whose sign matches the truth table of a given Boolean function is said to sign-represent that Boolean function. The artificial neural units that implement this sign-representation scheme are often called higher-order neurons or polynomial threshold units. This paper investigates the freedom provided by the sign-representation framework in terms of the fan-in of these artificial neural units. In particular, we look for sign-representations with a small number of monomials. Although there are methods developed for finding a reduced set of monomials to represent Boolean functions, there are no deterministic algorithms for computing non-trivial solutions with guarantees on the number of monomials in the found sign-representations. This work fills this gap by providing deterministic algorithms which are guaranteed to find solutions with fewer than 0.75 x 2(n) monomials for n-variable Boolean functions. Although the algorithms presented here are computationally costly, it is expected that several research directions can be spawned from the current study, such as reducing the 0.75 x 2(n) bound and devising efficient algorithms for finding sign-representations with a small number of monomials.

Detailed numerical investigation of the dissipative stochastic mechanics based neuron model

Recently, a physical approach for the description of neuronal dynamics under the influence of ion channel noise was proposed in the realm of dissipative stochastic mechanics (Güler, Phys Rev E 76:041918, 2007). Led by the presence of a multiple number of gates in an ion channel, the approach establishes a viewpoint that ion channels are exposed to two kinds of noise: the intrinsic noise, associated with the stochasticity in the movement of gating particles between the inner and the outer faces of the membrane, and the topological noise, associated with the uncertainty in accessing the permissible topological states of open gates. Renormalizations of the membrane capacitance and of a membrane voltage dependent potential function were found to arise from the mutual interaction of the two noisy systems. The formalism therein was scrutinized using a special membrane with some tailored properties giving the Rose-Hindmarsh dynamics in the deterministic limit. In this paper, the resultant computational neuron model of the above approach is investigated in detail numerically for its dynamics using time-independent input currents. The following are the major findings obtained. The intrinsic noise gives rise to two significant coexisting effects: it initiates spiking activity even in some range of input currents for which the corresponding deterministic model is quiet and causes bursting in some other range of input currents for which the deterministic model fires tonically. The renormalization corrections are found to augment the above behavioral transitions from quiescence to spiking and from tonic firing to bursting, and, therefore, the bursting activity is found to take place in a wider range of input currents for larger values of the correction coefficients. Some findings concerning the diffusive behavior in the voltage space are also reported.

An upper bound on the minimum number of monomials required to separate dichotomies of {-1, 1}n

It is known that any dichotomy of {-1, 1}n can be learned (separated) with a higher-order neuron (polynomial function) with 2n inputs (monomials). In general, less than 2n monomials are sufficient to solve a given dichotomy. In spite of the efforts to develop algorithms for finding solutions with fewer monomials, there have been relatively fewer studies investigating maximum density (Pi(n)), the minimum number of monomials that would suffice to separate an arbitrary dichotomy of {-1, 1}n . This article derives a theoretical (upper) bound for this quantity, superseding previously known bounds. The main theorem here states that for any binary classification problem in {-1, 1}n (n > 1), one can always find a polynomial function solution with 2n -2n/4 or fewer monomials. In particular, any dichotomy of {-1, 1}n can be learned by a higher-order neuron with a fan-in of 2n -2n/4 or less. With this result, for the first time, a deterministic ratio bound independent of n is established as Pi(n)/2n < or = 0 75. The main theorem is constructive, so it provides a deterministic algorithm for achieving the theoretical result. The study presented provides the basic mathematical tools and forms the basis for further analyses that may have implications for neural computation mechanisms employed in the cerebral cortex.