A maximum entropy framework for nonexponential distributions

Correlations and Kappa Distributions: Numerical Experiment and Physical Understanding

Kappa distributions, their statistical framework, and their thermodynamic origin describe systems with correlations among their particle energies, residing in stationary states out of classical thermal equilibrium/space plasmas, from solar wind to the outer heliosphere, are such systems. We show how correlations from long-range interactions compete with collisions to define the specific shape of particle velocity distributions, using a simple numerical experiment with collisions and a variable amount of correlation among the particles. When the correlations are turned off, collisions drive any initial distribution to evolve toward equilibrium and a Maxwell-Boltzmann (MB) distribution. However, when some correlation is introduced, the distribution evolves toward a different stationary state defined by a kappa distribution with some finite value of the thermodynamic kappa κ (where κ→∞ corresponds to a MB distribution). Furthermore, the stronger the correlations, the lower the κ value. This simple numerical experiment illuminates the role of correlations in forming stationary state particle distributions, which are described by kappa distributions, as well as the physical interpretation of correlations from long-range interactions and how they are related to the thermodynamic kappa.

The theory of thermodynamic relativity

We introduce the theory of thermodynamic relativity, a unified theoretical framework for describing both entropies and velocities, and their respective physical disciplines of thermodynamics and kinematics, which share a surprisingly identical description with relativity. This is the first study to generalize relativity in a thermodynamic context, leading naturally to anisotropic and nonlinear adaptations of relativity; thermodynamic relativity constitutes a new path of generalization, as compared to the "traditional" passage from special to general theory based on curved spacetime. We show that entropy and velocity are characterized by three identical postulates, which provide the basis of a broader framework of relativity: (1) no privileged reference frame with zero value; (2) existence of an invariant and fixed value for all reference frames; and (3) existence of stationarity. The postulates lead to a unique way of addition for entropies and for velocities, called kappa-addition. We develop a systematic method of constructing a generalized framework of the theory of relativity, based on the kappa-addition formulation, which is fully consistent with both thermodynamics and kinematics. We call this novel and unified theoretical framework for simultaneously describing entropy and velocity "thermodynamic relativity". From the generality of the kappa-addition formulation, we focus on the cases corresponding to linear adaptations of special relativity. Then, we show how the developed thermodynamic relativity leads to the addition of entropies in nonextensive thermodynamics and the addition of velocities in Einstein's isotropic special relativity, as in two extreme cases, while intermediate cases correspond to a possible anisotropic adaptation of relativity. Using thermodynamic relativity for velocities, we start from the kappa-addition of velocities and construct the basic formulations of the linear anisotropic special relativity; e.g., the asymmetric Lorentz transformation, the nondiagonal metric, and the energy-momentum-velocity relationships. Then, we discuss the physical consequences of the possible anisotropy in known relativistic effects, such as, (i) matter-antimatter asymmetry, (ii) time dilation, and (iii) Doppler effect, and show how these might be used to detect and quantify a potential anisotropy.

Effect of contact inhibition locomotion on confined cellular organization

Experiments performed using micro-patterned one dimensional collision assays have allowed a precise quantitative analysis of the collective manifestation of contact inhibition locomotion (CIL) wherein, individual migrating cells reorient their direction of motion when they come in contact with other cells. Inspired by these experiments, we present a discrete, minimal 1D Active spin model that mimics the CIL interaction between cells in one dimensional channels. We analyze the emergent collective behaviour of migrating cells in such confined geometries, as well as the sensitivity of the emergent patterns to driving forces that couple to cell motion. In the absence of vacancies, akin to dense cell packing, the translation dynamics is arrested and the model reduces to an equilibrium spin model which can be solved exactly. In the presence of vacancies, the interplay of activity-driven translation, cell polarity switching, and CIL results in an exponential steady cluster size distribution. We define a dimensionless Péclet number Q-the ratio of the translation rate and directional switching rate of particles in the absence of CIL. While the average cluster size increases monotonically as a function of Q, it exhibits a non-monotonic dependence on CIL strength, when the Q is sufficiently high. In the high Q limit, an analytical form of average cluster size can be obtained approximately by effectively mapping the system to an equivalent equilibrium process involving clusters of different sizes wherein the cluster size distribution is obtained by minimizing an effective Helmholtz free energy for the system. The resultant prediction of exponential dependence on CIL strength of the average cluster size and [Formula: see text] dependence of the average cluster size is borne out to reasonable accuracy as long as the CIL strength is not very large. The consequent prediction of a single scaling function of Q, particle density and CIL interaction strength, characterizing the distribution function of the cluster sizes and resultant data collapse is observed for a range of parameters.

Entropy defect in thermodynamics

This paper describes the physical foundations of the newly discovered "entropy defect" as a basic concept of thermodynamics. The entropy defect quantifies the change in entropy caused by the order induced in a system through the additional correlations among its constituents when two or more subsystems are assembled. This defect is closely analogous to the mass defect that arises when nuclear particle systems are assembled. The entropy defect determines how the entropy of the system compares to its constituent's entropies and stands on three fundamental properties: each constituent's entropy must be (i) separable, (ii) symmetric, and (iii) bounded. We show that these properties provide a solid foundation for the entropy defect and for generalizing thermodynamics to describe systems residing out of the classical thermal equilibrium, both in stationary and nonstationary states. In stationary states, the consequent thermodynamics generalizes the classical framework, which was based on the Boltzmann-Gibbs entropy and Maxwell-Boltzmann canonical distribution of particle velocities, into the respective entropy and canonical distribution associated with kappa distributions. In nonstationary states, the entropy defect similarly acts as a negative feedback, or reduction of the increase of entropy, preventing its unbounded growth toward infinity.

Maximum entropy and constraints in composite systems

The principle of maximum entropy (PME), as expounded by Jaynes, is based on the maximization of the Boltzmann-Gibbs-Shannon (BGS) entropy subject to linear constraints. The resulting probability distributions are of canonical (exponential) form. However, the rationale for linear constraints is nebulous, and probability distributions are not always canonical. Here we show that the correct noncanonical distribution for a system in equilibrium with a finite heat bath is implied by the unconstrained maximization of the total BGS entropy of the system and bath together. This procedure is shown to be equivalent to maximizing the BGS entropy of the system alone subject to a contrived nonlinear constraint which reduces to (a) the usual linear constraint for an infinite heat bath, and (b) a previously enigmatic logarithmic constraint which implies a power-law distribution for a large but finite heat bath. This procedure eliminates the uncertainty as to the proper constraints, and easily generalizes to arbitrary composite systems, for which it provides a simpler alternative to the Jaynes PME.

The Maximum Caliber Variational Principle for Nonequilibria

Ever since Clausius in 1865 and Boltzmann in 1877, the concepts of entropy and of its maximization have been the foundations for predicting how material equilibria derive from microscopic properties. But, despite much work, there has been no equally satisfactory general variational principle for nonequilibrium situations. However, in 1980, a new avenue was opened by E.T. Jaynes and by Shore and Johnson. We review here maximum caliber, which is a maximum-entropy-like principle that can infer distributions of flows over pathways, given dynamical constraints. This approach is providing new insights, particularly into few-particle complex systems, such as gene circuits, protein conformational reaction coordinates, network traffic, bird flocking, cell motility, and neuronal firing.

From Boltzmann to Zipf through Shannon and Jaynes

The word-frequency distribution provides the fundamental building blocks that generate discourse in natural language. It is well known, from empirical evidence, that the word-frequency distribution of almost any text is described by Zipf’s law, at least approximately. Following Stephens and Bialek (2010), we interpret the frequency of any word as arising from the interaction potentials between its constituent letters. Indeed, Jaynes’ maximum-entropy principle, with the constrains given by every empirical two-letter marginal distribution, leads to a Boltzmann distribution for word probabilities, with an energy-like function given by the sum of the all-to-all pairwise (two-letter) potentials. The so-called improved iterative-scaling algorithm allows us finding the potentials from the empirical two-letter marginals. We considerably extend Stephens and Bialek’s results, applying this formalism to words with length of up to six letters from the English subset of the recently created Standardized Project Gutenberg Corpus. We find that the model is able to reproduce Zipf’s law, but with some limitations: the general Zipf’s power-law regime is obtained, but the probability of individual words shows considerable scattering. In this way, a pure statistical-physics framework is used to describe the probabilities of words. As a by-product, we find that both the empirical two-letter marginal distributions and the interaction-potential distributions follow well-defined statistical laws.

Relating Vertex and Global Graph Entropy in Randomly Generated Graphs

Combinatoric measures of entropy capture the complexity of a graph but rely upon the calculation of its independent sets, or collections of non-adjacent vertices. This decomposition of the vertex set is a known NP-Complete problem and for most real world graphs is an inaccessible calculation. Recent work by Dehmer et al. and Tee et al. identified a number of vertex level measures that do not suffer from this pathological computational complexity, but that can be shown to be effective at quantifying graph complexity. In this paper, we consider whether these local measures are fundamentally equivalent to global entropy measures. Specifically, we investigate the existence of a correlation between vertex level and global measures of entropy for a narrow subset of random graphs. We use the greedy algorithm approximation for calculating the chromatic information and therefore Körner entropy. We are able to demonstrate strong correlation for this subset of graphs and outline how this may arise theoretically.

Perspective: Maximum caliber is a general variational principle for dynamical systems

We review here Maximum Caliber (Max Cal), a general variational principle for inferring distributions of paths in dynamical processes and networks. Max Cal is to dynamical trajectories what the principle of maximum entropy is to equilibrium states or stationary populations. In Max Cal, you maximize a path entropy over all possible pathways, subject to dynamical constraints, in order to predict relative path weights. Many well-known relationships of non-equilibrium statistical physics-such as the Green-Kubo fluctuation-dissipation relations, Onsager's reciprocal relations, and Prigogine's minimum entropy production-are limited to near-equilibrium processes. Max Cal is more general. While it can readily derive these results under those limits, Max Cal is also applicable far from equilibrium. We give examples of Max Cal as a method of inference about trajectory distributions from limited data, finding reaction coordinates in bio-molecular simulations, and modeling the complex dynamics of non-thermal systems such as gene regulatory networks or the collective firing of neurons. We also survey its basis in principle and some limitations.

Interstellar Travel and Galactic Colonization: Insights from Percolation Theory and the Yule Process

In this paper, percolation theory is employed to place tentative bounds on the probability p of interstellar travel and the emergence of a civilization (or panspermia) that colonizes the entire Galaxy. The ensuing ramifications with regard to the Fermi paradox are also explored. In particular, it is suggested that the correlation function of inhabited exoplanets can be used to observationally constrain p in the near future. It is shown, by using a mathematical evolution model known as the Yule process, that the probability distribution for civilizations with a given number of colonized worlds is likely to exhibit a power-law tail. Some of the dynamical aspects of this issue, including the question of timescales and generalizing percolation theory, were also studied. The limitations of these models, and other avenues for future inquiry, are also outlined.

KEY WORDS

Complex life-Extraterrestrial life-Panspermia-Life detection-SETI. Astrobiology 16, 418-426.

Large-Scale Analysis of Zipf's Law in English Texts

Despite being a paradigm of quantitative linguistics, Zipf’s law for words suffers from three main problems: its formulation is ambiguous, its validity has not been tested rigorously from a statistical point of view, and it has not been confronted to a representatively large number of texts. So, we can summarize the current support of Zipf’s law in texts as anecdotic. We try to solve these issues by studying three different versions of Zipf’s law and fitting them to all available English texts in the Project Gutenberg database (consisting of more than 30 000 texts). To do so we use state-of-the art tools in fitting and goodness-of-fit tests, carefully tailored to the peculiarities of text statistics. Remarkably, one of the three versions of Zipf’s law, consisting of a pure power-law form in the complementary cumulative distribution function of word frequencies, is able to fit more than 40% of the texts in the database (at the 0.05 significance level), for the whole domain of frequencies (from 1 to the maximum value), and with only one free parameter (the exponent).

Stationary properties of maximum-entropy random walks

Maximum-entropy (ME) inference of state probabilities using state-dependent constraints is popular in the study of complex systems. In stochastic systems, how state space topology and path-dependent constraints affect ME-inferred state probabilities remains unknown. To that end, we derive the transition probabilities and the stationary distribution of a maximum path entropy Markov process subject to state- and path-dependent constraints. A main finding is that the stationary distribution over states differs significantly from the Boltzmann distribution and reflects a competition between path multiplicity and imposed constraints. We illustrate our results with particle diffusion on a two-dimensional landscape. Connections with the path integral approach to diffusion are discussed.

Equipartitions and a distribution for numbers: A statistical model for Benford's law

A statistical model for the fragmentation of a conserved quantity is analyzed, using the principle of maximum entropy and the theory of partitions. Upper and lower bounds for the restricted partitioning problem are derived and applied to the distribution of fragments. The resulting power law directly leads to Benford's law for the first digits of the parts.

Memoryless self-reinforcing directionality in endosomal active transport within living cells

In contrast to Brownian transport, the active motility of microbes, cells, animals and even humans often follows another random process known as truncated Lévy walk. These stochastic motions are characterized by clustered small steps and intermittent longer jumps that often extend towards the size of the entire system. As there are repeated suggestions, although disagreement, that Lévy walks have functional advantages over Brownian motion in random searching and transport kinetics, their intentional engineering into active materials could be useful. Here, we show experimentally in the classic active matter system of intracellular trafficking that Brownian-like steps self-organize into truncated Lévy walks through an apparent time-independent positive feedback such that directional persistence increases with the distance travelled persistently. A molecular model that allows the maximum output of the active propelling forces to fluctuate slowly fits the experiments quantitatively. Our findings offer design principles for programming efficient transport in active materials.

Third law of thermodynamics as a key test of generalized entropies

The laws of thermodynamics constrain the formulation of statistical mechanics at the microscopic level. The third law of thermodynamics states that the entropy must vanish at absolute zero temperature for systems with nondegenerate ground states in equilibrium. Conversely, the entropy can vanish only at absolute zero temperature. Here we ask whether or not generalized entropies satisfy this fundamental property. We propose a direct analytical procedure to test if a generalized entropy satisfies the third law, assuming only very general assumptions for the entropy S and energy U of an arbitrary N-level classical system. Mathematically, the method relies on exact calculation of β=dS/dU in terms of the microstate probabilities p(i). To illustrate this approach, we present exact results for the two best known generalizations of statistical mechanics. Specifically, we study the Kaniadakis entropy S(κ), which is additive, and the Tsallis entropy S(q), which is nonadditive. We show that the Kaniadakis entropy correctly satisfies the third law only for -1<κ<+1, thereby shedding light on why κ is conventionally restricted to this interval. Surprisingly, however, the Tsallis entropy violates the third law for q<1. Finally, we give a concrete example of the power of our proposed method by applying it to a paradigmatic system: the one-dimensional ferromagnetic Ising model with nearest-neighbor interactions.

Power laws in citation distributions: evidence from Scopus

Modeling distributions of citations to scientific papers is crucial for understanding how science develops. However, there is a considerable empirical controversy on which statistical model fits the citation distributions best. This paper is concerned with rigorous empirical detection of power-law behaviour in the distribution of citations received by the most highly cited scientific papers. We have used a large, novel data set on citations to scientific papers published between 1998 and 2002 drawn from Scopus. The power-law model is compared with a number of alternative models using a likelihood ratio test. We have found that the power-law hypothesis is rejected for around half of the Scopus fields of science. For these fields of science, the Yule, power-law with exponential cut-off and log-normal distributions seem to fit the data better than the pure power-law model. On the other hand, when the power-law hypothesis is not rejected, it is usually empirically indistinguishable from most of the alternative models. The pure power-law model seems to be the best model only for the most highly cited papers in “Physics and Astronomy”. Overall, our results seem to support theories implying that the most highly cited scientific papers follow the Yule, power-law with exponential cut-off or log-normal distribution. Our findings suggest also that power laws in citation distributions, when present, account only for a very small fraction of the published papers (less than 1 % for most of science fields) and that the power-law scaling parameter (exponent) is substantially higher (from around 3.2 to around 4.7) than found in the older literature.

Detecting temperature fluctuations at equilibrium

Equilibrium and dynamical properties of small systems are captured accurately when their temperature is allowed to vary.

Universal internucleotide statistics in full genomes: a footprint of the DNA structure and packaging?

Uncovering the fundamental laws that govern the complex DNA structural organization remains challenging and is largely based upon reconstructions from the primary nucleotide sequences. Here we investigate the distributions of the internucleotide intervals and their persistence properties in complete genomes of various organisms from Archaea and Bacteria to H. Sapiens aiming to reveal the manifestation of the universal DNA architecture. We find that in all considered organisms the internucleotide interval distributions exhibit the same -exponential form. While in prokaryotes a single -exponential function makes the best fit, in eukaryotes the PDF contains additionally a second -exponential, which in the human genome makes a perfect approximation over nearly 10 decades. We suggest that this functional form is a footprint of the heterogeneous DNA structure, where the first -exponential reflects the universal helical pitch that appears both in pro- and eukaryotic DNA, while the second -exponential is a specific marker of the large-scale eukaryotic DNA organization.

Predictability of extreme events in social media

It is part of our daily social-media experience that seemingly ordinary items (videos, news, publications, etc.) unexpectedly gain an enormous amount of attention. Here we investigate how unexpected these extreme events are. We propose a method that, given some information on the items, quantifies the predictability of events, i.e., the potential of identifying in advance the most successful items. Applying this method to different data, ranging from views in YouTube videos to posts in Usenet discussion groups, we invariantly find that the predictability increases for the most extreme events. This indicates that, despite the inherently stochastic collective dynamics of users, efficient prediction is possible for the most successful items.