Time scales in evolutionary dynamics

Emergence of cooperation through coevolving time scale in spatial prisoner's dilemma

We study evolutionary prisoner's dilemma game by considering adaptive strategy-selection time scale among individuals according to a "win-slower, lose-faster" rule: if an individual successfully resists the invasion of an opponent, she is prone to hold her strategy for longer time through decreasing her strategy-selection time scale; otherwise, she increases the time scale because of losing. We find that the greater the losers increase their strategy-selection time scales, the better for cooperation. Interestingly, optimal cooperation can be induced by proper adaptive rate in the strategy-selection time scale. Our results may have potential implications in the design of consensus protocol in multiagent systems.

Phase transitions to cooperation in the prisoner's dilemma

Game theory formalizes certain interactions between physical particles or between living beings in biology, sociology, and economics and quantifies the outcomes by payoffs. The prisoner's dilemma (PD) describes situations in which it is profitable if everybody cooperates rather than defects (free rides or cheats), but as cooperation is risky and defection is tempting, the expected outcome is defection. Nevertheless, some biological and social mechanisms can support cooperation by effectively transforming the payoffs. Here, we study the related phase transitions, which can be of first order (discontinuous) or of second order (continuous), implying a variety of different routes to cooperation. After classifying the transitions into cases of equilibrium displacement, equilibrium selection, and equilibrium creation, we show that a transition to cooperation may take place even if the stationary states and the eigenvalues of the replicator equation for the PD stay unchanged. Our example is based on adaptive group pressure, which makes the payoffs dependent on the endogenous dynamics in the population. The resulting bistability can invert the expected outcome in favor of cooperation.

Evolutionary dynamics of populations with conflicting interactions: classification and analytical treatment considering asymmetry and power

Evolutionary game theory has been successfully used to investigate the dynamics of systems, in which many entities have competitive interactions. From a physics point of view, it is interesting to study conditions under which a coordination or cooperation of interacting entities will occur, be it spins, particles, bacteria, animals, or humans. Here, we analyze the case, where the entities are heterogeneous, particularly the case of two populations with conflicting interactions and two possible states. For such systems, explicit mathematical formulas will be determined for the stationary solutions and the associated eigenvalues, which determine their stability. In this way, four different types of system dynamics can be classified and the various kinds of phase transitions between them will be discussed. While these results are interesting from a physics point of view, they are also relevant for social, economic, and biological systems, as they allow one to understand conditions for (1) the breakdown of cooperation, (2) the coexistence of different behaviors ("subcultures"), (3) the evolution of commonly shared behaviors ("norms"), and (4) the occurrence of polarization or conflict. We point out that norms have a similar function in social systems that forces have in physics.

Coevolutionary games--a mini review

Prevalence of cooperation within groups of selfish individuals is puzzling in that it contradicts with the basic premise of natural selection. Favoring players with higher fitness, the latter is key for understanding the challenges faced by cooperators when competing with defectors. Evolutionary game theory provides a competent theoretical framework for addressing the subtleties of cooperation in such situations, which are known as social dilemmas. Recent advances point towards the fact that the evolution of strategies alone may be insufficient to fully exploit the benefits offered by cooperative behavior. Indeed, while spatial structure and heterogeneity, for example, have been recognized as potent promoters of cooperation, coevolutionary rules can extend the potentials of such entities further, and even more importantly, lead to the understanding of their emergence. The introduction of coevolutionary rules to evolutionary games implies, that besides the evolution of strategies, another property may simultaneously be subject to evolution as well. Coevolutionary rules may affect the interaction network, the reproduction capability of players, their reputation, mobility or age. Here we review recent works on evolutionary games incorporating coevolutionary rules, as well as give a didactic description of potential pitfalls and misconceptions associated with the subject. In addition, we briefly outline directions for future research that we feel are promising, thereby particularly focusing on dynamical effects of coevolutionary rules on the evolution of cooperation, which are still widely open to research and thus hold promise of exciting new discoveries.

Effect of spatial structure on the evolution of cooperation

Spatial structure is known to have an impact on the evolution of cooperation, and so it has been intensively studied during recent years. Previous work has shown the relevance of some features, such as the synchronicity of the updating, the clustering of the network, or the influence of the update rule. This has been done, however, for concrete settings with particular games, networks, and update rules, with the consequence that some contradictions have arisen and a general understanding of these topics is missing in the broader context of the space of 2x2 games. To address this issue, we have performed a systematic and exhaustive simulation in the different degrees of freedom of the problem. In some cases, we generalize previous knowledge to the broader context of our study and explain the apparent contradictions. In other cases, however, our conclusions refute what seems to be established opinions in the field, as for example the robustness of the effect of spatial structure against changes in the update rule, or offer new insights into the subject, e.g., the relation between the intensity of selection and the asymmetry between the effects on games with mixed equilibria.

A mechanism of dynamical interactions for two-person social dilemmas

We propose a new mechanism of interactions between game-theoretical agents in which the weights of the connections between interacting individuals are dynamical, payoff-dependent variables. Their evolution depends on the difference between the payoff of the agents from a given type of encounter and their average payoff. The mechanism is studied in the frame of two models: agents distributed on a random graph, and a mean field model. Symmetric and asymmetric connections between the agents are introduced. Long time behavior of both systems is discussed for the Prisoner's Dilemma and the Snow Drift games.

Diversity of reproduction time scale promotes cooperation in spatial prisoner's dilemma games

We study an evolutionary spatial prisoner's dilemma game where the fitness of the players is determined by both the payoffs from the current interaction and their history. We consider the situation where the selection time scale is slower than the interaction time scale. This is done by implementing probabilistic reproduction on an individual level. We observe that both too fast and too slow reproduction rates hamper the emergence of cooperation. In other words, there exists an intermediate selection time scale that maximizes cooperation. Another factor we find to promote cooperation is a diversity of reproduction time scales.

Impact of aging on the evolution of cooperation in the spatial prisoner's dilemma game

Aging is always present, tailoring our interactions with others, and postulating a finite lifespan during which we are able to exercise them. We consider the prisoner's dilemma game on a square lattice and examine how quenched age distributions and different aging protocols influence the evolution of cooperation when taking the life experience and knowledge accumulation into account as time passes. In agreement with previous studies, we find that a quenched assignment of age to players, introducing heterogeneity to the game, substantially promotes cooperative behavior. Introduction of aging and subsequent death as a coevolutionary process may act detrimental on cooperation but enhances it efficiently if the offspring of individuals that have successfully passed their strategy is considered newborn. We study resulting age distributions of players and show that the heterogeneity is vital-yet insufficient-for explaining the observed differences in cooperator abundance on the spatial grid. The unexpected increment of cooperation levels can be explained by a dynamical effect that has a highly selective impact on the propagation of cooperator and defector states.

Evolutionary dynamics on graphs: Efficient method for weak selection

Investigating the evolutionary dynamics of game theoretical interactions in populations where individuals are arranged on a graph can be challenging in terms of computation time. Here, we propose an efficient method to study any type of game on arbitrary graph structures for weak selection. In this limit, evolutionary game dynamics represents a first-order correction to neutral evolution. Spatial correlations can be empirically determined under neutral evolution and provide the basis for formulating the game dynamics as a discrete Markov process by incorporating a detailed description of the microscopic dynamics based on the neutral correlations. This framework is then applied to one of the most intriguing questions in evolutionary biology: the evolution of cooperation. We demonstrate that the degree heterogeneity of a graph impedes cooperation and that the success of tit for tat depends not only on the number of rounds but also on the degree of the graph. Moreover, considering the mutation-selection equilibrium shows that the symmetry of the stationary distribution of states under weak selection is skewed in favor of defectors for larger selection strengths. In particular, degree heterogeneity--a prominent feature of scale-free networks--generally results in a more pronounced increase in the critical benefit-to-cost ratio required for evolution to favor cooperation as compared to regular graphs. This conclusion is corroborated by an analysis of the effects of population structures on the fixation probabilities of strategies in general 2 x 2 games for different types of graphs. Computer simulations confirm the predictive power of our method and illustrate the improved accuracy as compared to previous studies.

Partner switching stabilizes cooperation in coevolutionary prisoner's dilemma

Previous studies suggest that cooperation prevails when individuals can switch their interaction partners quickly. However, it is still unclear how quickly individuals should switch adverse partners to maximize cooperation. To address this issue, we propose a simple model of coevolutionary prisoner's dilemma in which individuals are allowed to either adjust their strategies or switch their defective partners. Interestingly, we find that, depending on the game parameter, there is an optimal tendency of switching adverse partnerships that maximizes the fraction of cooperators in the population. We confirm that the stabilization of cooperation by partner switching remains effective under some situations, where either normalized or accumulated payoff is used in strategy updating, and where either only cooperators or all individuals are privileged to sever disadvantageous partners. We also provide an extended pair approximation to study the coevolutionary dynamics. Our results may be helpful in understanding the role of partner switching in the stabilization of cooperation in the real world.

Stochastic sampling of interaction partners versus deterministic payoff assignment

Evolutionary game dynamics describes how successful strategies spread in a population. In well-mixed populations, the usual assumption, e.g. underlying the replicator dynamics, is that individuals obtain a payoff from interactions with a representative sample of the population. This determines their fitness. Here, we analyze a situation in which payoffs are obtained through a single interaction, so that individuals of the same type can have different payoffs. We show analytically that for weak selection, this scenario is identical to the usual approach in which an individual interacts with the whole population. For strong selection, however, differences arise that are reflected in the fixation probabilities and lead to deviating evolutionary dynamics.

Stochastic gain in finite populations

Flexible learning rates can lead to increased payoffs under the influence of noise. In a previous paper [Traulsen, Phys. Rev. Lett. 93, 028701 (2004)], we have demonstrated this effect based on a replicator dynamics model which is subject to external noise. Here, we utilize recent advances on finite population dynamics and their connection to the replicator equation to extend our findings and demonstrate the stochastic gain effect in finite population systems. Finite population dynamics is inherently stochastic, depending on the population size and the intensity of selection, which measures the balance between the deterministic and the stochastic parts of the dynamics. This internal noise can be exploited by a population using an appropriate microscopic update process, even if learning rates are constant.

Intelligent tit-for-tat in the iterated prisoner's dilemma game

We seek a route to the equilibrium where all the agents cooperate in the iterated prisoner's dilemma game on a two-dimensional plane, focusing on the role of tit-for-tat strategy. When a time horizon, within which a strategy can recall the past, is one time step, an equilibrium can be achieved as cooperating strategies dominate the whole population via proliferation of tit-for-tat. Extending the time horizon, we filter out poor strategies by simplified replicator dynamics and observe a similar evolutionary pattern to reach the cooperating equilibrium. In particular, the rise of a modified tit-for-tat strategy plays a central role, which implies how a robust strategy is adopted when provided with an enhanced memory capacity.

Mesoscopic structure conditions the emergence of cooperation on social networks

Background

We study the evolutionary Prisoner's Dilemma on two social networks substrates obtained from actual relational data.

Methodology/Principal Findings

We find very different cooperation levels on each of them that cannot be easily understood in terms of global statistical properties of both networks. We claim that the result can be understood at the mesoscopic scale, by studying the community structure of the networks. We explain the dependence of the cooperation level on the temptation parameter in terms of the internal structure of the communities and their interconnections. We then test our results on community-structured, specifically designed artificial networks, finding a good agreement with the observations in both real substrates.

Conclusion

Our results support the conclusion that studies of evolutionary games on model networks and their interpretation in terms of global properties may not be sufficient to study specific, real social systems. Further, the study allows us to define new quantitative parameters that summarize the mesoscopic structure of any network. In addition, the community perspective may be helpful to interpret the origin and behavior of existing networks as well as to design structures that show resilient cooperative behavior.

Breaking the symmetry between interaction and replacement in evolutionary dynamics on graphs

We study the evolution of cooperation modeled as symmetric 2x2 games in a population whose structure is split into an interaction graph defining who plays with whom and a replacement graph specifying evolutionary competition. We find it is always harder for cooperators to evolve whenever the two graphs do not coincide. In the thermodynamic limit, the dynamics on both graphs is given by a replicator equation with a rescaled payoff matrix in a rescaled time. Analytical results are obtained in the pair approximation and for weak selection. Their validity is confirmed by computer simulations.

Stochastic payoff evaluation increases the temperature of selection

We study stochastic evolutionary game dynamics in populations of finite size. Moreover, each individual has a randomly distributed number of interactions with other individuals. Therefore, the payoff of two individuals using the same strategy can be different. The resulting "payoff stochasticity" reduces the intensity of selection and therefore increases the temperature of selection. A simple mean-field approximation is derived that captures the average effect of the payoff stochasticity. Correction terms to the mean-field theory are computed and discussed.