Tuning and jamming reduced to their minima

The fluidic memristor as a collective phenomenon in elastohydrodynamic networks

Fluid flow networks are ubiquitous and can be found in a broad range of contexts, from human-made systems such as water supply networks to living systems like animal and plant vasculature. In many cases, the elements forming these networks exhibit a highly non-linear pressure-flow relationship. Although we understand how these elements work individually, their collective behavior remains poorly understood. In this work, we combine experiments, theory, and numerical simulations to understand the main mechanisms underlying the collective behavior of soft flow networks with elements that exhibit negative differential resistance. Strikingly, our theoretical analysis and experiments reveal that a minimal network of nonlinear resistors, which we have termed a 'fluidic memristor', displays history-dependent resistance. This new class of element can be understood as a collection of hysteresis loops that allows this fluidic system to store information, and it can be directly used as a tunable resistor in fluidic setups. Our results provide insights that can inform other applications of fluid flow networks in soft materials science, biomedical settings, and soft robotics, and may also motivate new understanding of the flow networks involved in animal and plant physiology.

Physical effects of learning

Interacting many-body physical systems ranging from neural networks in the brain to folding proteins to self-modifying electrical circuits can learn to perform diverse tasks. This learning, both in nature and in engineered systems, can occur through evolutionary selection or through dynamical rules that drive active learning from experience. Here, we show that learning in linear physical networks with weak input signals leaves architectural imprints on the Hessian of a physical system. Compared to a generic organization of the system components, (a) the effective physical dimension of the response to inputs decreases, (b) the response of physical degrees of freedom to random perturbations (or system "susceptibility") increases, and (c) the low-eigenvalue eigenvectors of the Hessian align with the task. Overall, these effects embody the typical scenario for learning processes in physical systems in the weak input regime, suggesting ways of discovering whether a physical network may have been trained.

Triadic influence as a proxy for compatibility in social relationships

Networks of social interactions are the substrate upon which civilizations are built. Often, we create new bonds with people that we like or feel that our relationships are damaged through the intervention of third parties. Despite their importance and the huge impact that these processes have in our lives, quantitative scientific understanding of them is still in its infancy, mainly due to the difficulty of collecting large datasets of social networks including individual attributes. In this work, we present a thorough study of real social networks of 13 schools, with more than 3,000 students and 60,000 declared positive and negative relationships, including tests for personal traits of all the students. We introduce a metric-the "triadic influence"-that measures the influence of nearest neighbors in the relationships of their contacts. We use neural networks to predict the sign of the relationships in these social networks, extracting the probability that two students are friends or enemies depending on their personal attributes or the triadic influence. We alternatively use a high-dimensional embedding of the network structure to also predict the relationships. Remarkably, using the triadic influence (a simple one-dimensional metric) achieves the best accuracy, and adding the personal traits of the students does not improve the results, suggesting that the triadic influence acts as a proxy for the social compatibility of students. We postulate that the probabilities extracted from the neural networks-functions of the triadic influence and the personalities of the students-control the evolution of real social networks, opening an avenue for the quantitative study of these systems.

An analytical theory of curriculum learning in teacher-student networks

In animals and humans, curriculum learning-presenting data in a curated order-is critical to rapid learning and effective pedagogy. A long history of experiments has demonstrated the impact of curricula in a variety of animals but, despite its ubiquitous presence, a theoretical understanding of the phenomenon is still lacking. Surprisingly, in contrast to animal learning, curricula strategies are not widely used in machine learning and recent simulation studies reach the conclusion that curricula are moderately effective or even ineffective in most cases. This stark difference in the importance of curriculum raises a fundamental theoretical question: when and why does curriculum learning help? In this work, we analyse a prototypical neural network model of curriculum learning in the high-dimensional limit, employing statistical physics methods. We study a task in which a sparse set of informative features are embedded amidst a large set of noisy features. We analytically derive average learning trajectories for simple neural networks on this task, which establish a clear speed benefit for curriculum learning in the online setting. However, when training experiences can be stored and replayed (for instance, during sleep), the advantage of curriculum in standard neural networks disappears, in line with observations from the deep learning literature. Inspired by synaptic consolidation techniques developed to combat catastrophic forgetting, we propose curriculum-aware algorithms that consolidate synapses at curriculum change points and investigate whether this can boost the benefits of curricula. We derive generalisation performance as a function of consolidation strength (implemented as an L 2 regularisation/elastic coupling connecting learning phases), and show that curriculum-aware algorithms can yield a large improvement in test performance. Our reduced analytical descriptions help reconcile apparently conflicting empirical results, trace regimes where curriculum learning yields the largest gains, and provide experimentally-accessible predictions for the impact of task parameters on curriculum benefits. More broadly, our results suggest that fully exploiting a curriculum may require explicit adjustments in the loss.

Model architecture can transform catastrophic forgetting into positive transfer

The work of McCloskey and Cohen popularized the concept of catastrophic interference. They used a neural network that tried to learn addition using two groups of examples as two different tasks. In their case, learning the second task rapidly deteriorated the acquired knowledge about the previous one. We hypothesize that this could be a symptom of a fundamental problem: addition is an algorithmic task that should not be learned through pattern recognition. Therefore, other model architectures better suited for this task would avoid catastrophic forgetting. We use a neural network with a different architecture that can be trained to recover the correct algorithm for the addition of binary numbers. This neural network includes conditional clauses that are naturally treated within the back-propagation algorithm. We test it in the setting proposed by McCloskey and Cohen and training on random additions one by one. The neural network not only does not suffer from catastrophic forgetting but it improves its predictive power on unseen pairs of numbers as training progresses. We also show that this is a robust effect, also present when averaging many simulations. This work emphasizes the importance that neural network architecture has for the emergence of catastrophic forgetting and introduces a neural network that is able to learn an algorithm.