Dynamically generated cyclic dominance in spatial prisoner's dilemma games

Memory and target payoff enhance cooperation in evolutionary social dilemmas

We proposed a neighbor selection mechanism based on memory and target payoff, where the target payoff is the maximum value of the group's average expected payoff. According to this mechanism, individuals prioritize selecting neighbors whose average payoffs in the last M rounds are close to the target payoff for strategy learning, aiming to maximize the group's expected payoff. Simulation results on the grid-based Prisoner's Dilemma and Snowdrift games demonstrate that this mechanism can significantly improve the group's payoff and cooperation level. Furthermore, the longer the memory length, the higher the group's payoff and cooperation level. Overall, the combination of memory and target payoff can lead to the emergence and persistence of cooperation in social dilemmas as individuals are motivated to cooperate based on both their past experiences and future goals. This interplay highlights the significance of taking into account numerous variables in comprehending and promoting cooperation within evolutionary frameworks.

Emergence of cooperation under punishment: A reinforcement learning perspective

Punishment is a common tactic to sustain cooperation and has been extensively studied for a long time. While most of previous game-theoretic work adopt the imitation learning framework where players imitate the strategies of those who are better off, the learning logic in the real world is often much more complex. In this work, we turn to the reinforcement learning paradigm, where individuals make their decisions based upon their experience and long-term returns. Specifically, we investigate the prisoners' dilemma game with a Q-learning algorithm, and cooperators probabilistically pose punishment on defectors in their neighborhood. Unexpectedly, we find that punishment could lead to either continuous or discontinuous cooperation phase transitions, and the nucleation process of cooperation clusters is reminiscent of the liquid-gas transition. The analysis of a Q-table reveals the evolution of the underlying "psychologic" changes, which explains the nucleation process and different levels of cooperation. The uncovered first-order phase transition indicates that great care needs to be taken when implementing the punishment compared to the continuous scenario.

Complex pathways to cooperation emergent from asymmetry in heterogeneous populations

Cooperation within asymmetric populations has garnered significant attention in evolutionary games. This paper explores cooperation evolution in populations with weak and strong players, using a game model where players choose between cooperation and defection. Asymmetry stems from different benefits for strong and weak cooperators, with their benefit ratio indicating the degree of asymmetry. Varied rankings of parameters including the asymmetry degree, cooperation costs, and benefits brought by weak players give rise to scenarios including the prisoner's dilemma (PDG) for both player types, the snowdrift game (SDG), and mixed PDG-SDG interactions. Our results indicate that in an infinite well-mixed population, defection remains the dominant strategy when strong players engage in the prisoner's dilemma game. However, if strong players play snowdrift games, global cooperation increases with the proportion of strong players. In this scenario, strong cooperators can prevail over strong defectors when the proportion of strong players is low, but the prevalence of cooperation among strong players decreases as their proportion increases. In contrast, within a square lattice, the optimum global cooperation emerges at intermediate proportions of strong players with moderate degrees of asymmetry. Additionally, weak players protect cooperative clusters from exploitation by strong defectors. This study highlights the complex dynamics of cooperation in asymmetric interactions, contributing to the theory of cooperation in asymmetric games.

Emerging solutions from the battle of defensive alliances

Competing strategies in an evolutionary game model, or species in a biosystem, can easily form a larger unit which protects them from the invasion of an external actor. Such a defensive alliance may have two, three, four or even more members. But how effective can be such formation against an alternative group composed by other competitors? To address this question we study a minimal model where a two-member and a four-member alliances fight in a symmetric and balanced way. By presenting representative phase diagrams, we systematically explore the whole parameter range which characterizes the inner dynamics of the alliances and the intensity of their interactions. The group formed by a pair, who can exchange their neighboring positions, prevail in the majority of the parameter region. The rival quartet can only win if their inner cyclic invasion rate is significant while the mixing rate of the pair is extremely low. At specific parameter values, when neither of the alliances is strong enough, new four-member solutions emerge where a rock-paper-scissors-like trio is extended by the other member of the pair. These new solutions coexist hence all six competitors can survive. The evolutionary process is accompanied by serious finite-size effects which can be mitigated by appropriately chosen prepared initial states.

Exit rights open complex pathways to cooperation

We study the evolutionary dynamics of the Prisoner's Dilemma game in which cooperators and defectors interact with another actor type called exiters. Rather than being exploited by defectors, exiters exit the game in favour of a small pay-off. We find that this simple extension of the game allows cooperation to flourish in well-mixed populations when iterations or reputation are added. In networked populations, however, the exit option is less conducive to cooperation. Instead, it enables the coexistence of cooperators, defectors, and exiters through cyclic dominance. Other outcomes are also possible as the exit pay-off increases or the network structure changes, including network-wide oscillations in actor abundances that may cause the extinction of exiters and the domination of defectors, although game parameters should favour exiting. The complex dynamics that emerges in the wake of a simple option to exit the game implies that nuances matter even if our analyses are restricted to incentives for rational behaviour.

A novel route to cyclic dominance in voluntary social dilemmas

Cooperation is the backbone of modern human societies, making it a priority to understand how successful cooperation-sustaining mechanisms operate. Cyclic dominance, a non-transitive set-up comprising at least three strategies wherein the first strategy overrules the second, which overrules the third, which, in turn, overrules the first strategy, is known to maintain biodiversity, drive competition between bacterial strains, and preserve cooperation in social dilemmas. Here, we present a novel route to cyclic dominance in voluntary social dilemmas by adding to the traditional mix of cooperators, defectors and loners, a fourth player type, risk-averse hedgers, who enact tit-for-tat upon paying a hedging cost to avoid being exploited. When this cost is sufficiently small, cooperators, defectors and hedgers enter a loop of cyclic dominance that preserves cooperation even under the most adverse conditions. By contrast, when the hedging cost is large, hedgers disappear, consequently reverting to the traditional interplay of cooperators, defectors, and loners. In the interim region of hedging costs, complex evolutionary dynamics ensues, prompting transitions between states with two, three or four competing strategies. Our results thus reveal that voluntary participation is but one pathway to sustained cooperation via cyclic dominance.

Invasion-controlled pattern formation in a generalized multispecies predator-prey system

Rock-scissors-paper game, as the simplest model of intransitive relation between competing agents, is a frequently quoted model to explain the stable diversity of competitors in the race of surviving. When increasing the number of competitors we may face a novel situation because beside the mentioned unidirectional predator-prey-like dominance a balanced or peer relation can emerge between some competitors. By utilizing this possibility in the present work we generalize a four-state predator-prey-type model where we establish two groups of species labeled by even and odd numbers. In particular, we introduce different invasion probabilities between and within these groups, which results in a tunable intensity of bidirectional invasion among peer species. Our study reveals an exceptional richness of pattern formations where five quantitatively different phases are observed by varying solely the strength of the mentioned inner invasion. The related transition points can be identified with the help of appropriate order parameters based on the spatial autocorrelation decay, on the fraction of empty sites, and on the variance of the species density. Furthermore, the application of diverse, alliance-specific inner invasion rates for different groups may result in the extinction of the pair of species where this inner invasion is moderate. These observations highlight that beyond the well-known and intensively studied cyclic dominance there is an additional source of complexity of pattern formation that has not been explored earlier.

Self-organized interdependence among populations promotes cooperation by means of coevolution

We show that self-organized interdependence promotes the evolution of cooperation in interdependent networks. The evolution of connections between networks occurs according to the following rule: if a player often wins against its opponent (regardless of its strategy), it is allowed to form an external link with the corresponding partner in another network to obtain additional benefit; otherwise, the opportunity to increase its benefit is lost. Through numerical simulation, it is unveiled that cooperation can be significantly promoted due to interdependent network reciprocity. Interestingly, the synchronization of evolutionary processes emerges on both networks, and individuals can take advantage of interdependent network reciprocity when both the strategies and the coevolving times in the two networks are synchronous.

Multistability in the cyclic competition system

Cyclically competition models have been successful to gain an insight of biodiversity mechanism in ecosystems. There are, however, still limitations to elucidate complex phenomena arising in real competition. In this paper, we report that a multistability occurs in a simple rock-paper-scissor cyclically competition model by assuming that intraspecific competition depends on the logistic growth of each species density. This complex stability is absent in any cyclically competition model, and we investigate how the proposed intraspecific competition affects biodiversity in the existing society of three species through macroscopic and microscopic approaches. When the system is multistable, we show basins of the asymptotically stable heteroclinic cycle and stable attractors to demonstrate how the survival state is determined by initial densities of three species. Also, we find that the multistability is associated with a subcritical Hopf bifurcation. This surprising finding will give an opportunity to interpret rich dynamical phenomena in ecosystems which may occur in cyclic competition systems with different types of interactions.

General two-species interacting Lotka-Volterra system: Population dynamics and wave propagation

The population dynamics of two interacting species modeled by the Lotka-Volterra (LV) model with general parameters that can promote or suppress the other species is studied. It is found that the properties of the two species' isoclines determine the interaction of species, leading to six regimes in the phase diagram of interspecies interaction; i.e., there are six different interspecific relationships described by the LV model. Four regimes allow for nontrivial species coexistence, among which it is found that three of them are stable, namely, weak competition, mutualism, and predator-prey scenarios can lead to win-win coexistence situations. The Lyapunov function for general nontrivial two-species coexistence is also constructed. Furthermore, in the presence of spatial diffusion of the species, the dynamics can lead to steady wavefront propagation and can alter the population map. Propagating wavefront solutions in one dimension are investigated analytically and by numerical solutions. The steady wavefront speeds are obtained analytically via nonlinear dynamics analysis and verified by numerical solutions. In addition to the inter- and intraspecific interaction parameters, the intrinsic speed parameters of each species play a decisive role in species populations and wave properties. In some regimes, both species can copropagate with the same wave speeds in a finite range of parameters. Our results are further discussed in the light of possible biological relevance and ecological implications.

Time-Delay Induced Dimensional Crossover in the Voter Model

We investigate the ordering dynamics of the voter model with time-delayed interactions. The dynamical process in the d-dimensional lattice is shown to be equivalent to the first passage problem of a random walker in the (d+1)-dimensional strip of a finite width determined by the delay time. The equivalence reveals that the time delay leads to the dimensional crossover from the (d+1)-dimensional scaling behavior at a short time to the d-dimensional scaling behavior at a long time. The scaling property in both regimes and the crossover time scale are obtained analytically, which are confirmed with the numerical simulation results.

Fixation of competing strategies when interacting agents differ in the time scale of strategy updating

A commonly used assumption in evolutionary game theory is that natural selection acts on individuals in the same time scale; e.g., players use the same frequency to update their strategies. Variation in learning rates within populations suggests that evolutionary game theory may not necessarily be restricted to uniform time scales associated with the game interaction and strategy adaption evolution. In this study, we remove this restricting assumption by dividing the population into fast and slow groups according to the players' strategy updating frequencies and investigate how different strategy compositions of one group influence the evolutionary outcome of the other's fixation probabilities of strategies within its own group. Analytical analysis and numerical calculations are performed to study the evolutionary dynamics of strategies in typical classes of two-player games (prisoner's dilemma game, snowdrift game, and stag-hunt game). The introduction of the heterogeneity in strategy-update time scales leads to substantial changes in the evolution dynamics of strategies. We provide an approximation formula for the fixation probability of mutant types in finite populations and study the outcome of strategy evolution under the weak selection. We find that although heterogeneity in time scales makes the collective evolutionary dynamics more complicated, the possible long-run evolutionary outcome can be effectively predicted under technical assumptions when knowing the population composition and payoff parameters.

Competition in the presence of aging: dominance, coexistence, and alternation between states

We study the stochastic dynamics of coupled states with transition probabilities depending on local persistence, this is, the time since a state has changed. When the system has a preference to adopt older states the system orders quickly due to the dominance of old states. When preference for new states prevails, the system can show coexistence of states or synchronized collective behavior resulting in long ordering times. In this case, the magnetization of the system oscillates around zero. Finally we discuss a potential application in social systems.

A double-edged sword: Benefits and pitfalls of heterogeneous punishment in evolutionary inspection games

As a simple model for criminal behavior, the traditional two-strategy inspection game yields counterintuitive results that fail to describe empirical data. The latter shows that crime is often recurrent, and that crime rates do not respond linearly to mitigation attempts. A more apt model entails ordinary people who neither commit nor sanction crime as the third strategy besides the criminals and punishers. Since ordinary people free-ride on the sanctioning efforts of punishers, they may introduce cyclic dominance that enables the coexistence of all three competing strategies. In this setup ordinary individuals become the biggest impediment to crime abatement. We therefore also consider heterogeneous punisher strategies, which seek to reduce their investment into fighting crime in order to attain a more competitive payoff. We show that this diversity of punishment leads to an explosion of complexity in the system, where the benefits and pitfalls of criminal behavior are revealed in the most unexpected ways. Due to the raise and fall of different alliances no less than six consecutive phase transitions occur in dependence on solely the temptation to succumb to criminal behavior, leading the population from ordinary people-dominated across punisher-dominated to crime-dominated phases, yet always failing to abolish crime completely.

Cyclic dominance in evolutionary games: a review

Rock is wrapped by paper, paper is cut by scissors and scissors are crushed by rock. This simple game is popular among children and adults to decide on trivial disputes that have no obvious winner, but cyclic dominance is also at the heart of predator-prey interactions, the mating strategy of side-blotched lizards, the overgrowth of marine sessile organisms and competition in microbial populations. Cyclical interactions also emerge spontaneously in evolutionary games entailing volunteering, reward, punishment, and in fact are common when the competing strategies are three or more, regardless of the particularities of the game. Here, we review recent advances on the rock-paper-scissors (RPS) and related evolutionary games, focusing, in particular, on pattern formation, the impact of mobility and the spontaneous emergence of cyclic dominance. We also review mean-field and zero-dimensional RPS models and the application of the complex Ginzburg-Landau equation, and we highlight the importance and usefulness of statistical physics for the successful study of large-scale ecological systems. Directions for future research, related, for example, to dynamical effects of coevolutionary rules and invasion reversals owing to multi-point interactions, are also outlined.

The role of opportunistic migration in cyclic games

We study cyclic evolutionary games in a spatial diluted grid environment in which agents strategically interact locally but can also opportunistically move to other positions within a given migration radius. We find that opportunistic migration can inverse the cyclic prevalence between the strategies when the frequency of random imitation is large enough compared to the payoff-driven imitation. At the transition the average size of the patterns diverges and this threatens diversity of strategies.

From pairwise to group interactions in games of cyclic dominance

We study the rock-paper-scissors game in structured populations, where the invasion rates determine individual payoffs that govern the process of strategy change. The traditional version of the game is recovered if the payoffs for each potential invasion stem from a single pairwise interaction. However, the transformation of invasion rates to payoffs also allows the usage of larger interaction ranges. In addition to the traditional pairwise interaction, we therefore consider simultaneous interactions with all nearest neighbors, as well as with all nearest and next-nearest neighbors, thus effectively going from single pair to group interactions in games of cyclic dominance. We show that differences in the interaction range affect not only the stationary fractions of strategies but also their relations of dominance. The transition from pairwise to group interactions can thus decelerate and even revert the direction of the invasion between the competing strategies. Like in evolutionary social dilemmas, in games of cyclic dominance, too, the indirect multipoint interactions that are due to group interactions hence play a pivotal role. Our results indicate that, in addition to the invasion rates, the interaction range is at least as important for the maintenance of biodiversity among cyclically competing strategies.

Costly advertising and the evolution of cooperation

In this paper, I investigate the co-evolution of fast and slow strategy spread and game strategies in populations of spatially distributed agents engaged in a one off evolutionary dilemma game. Agents are characterized by a pair of traits, a game strategy (cooperate or defect) and a binary ‘advertising’ strategy (advertise or don’t advertise). Advertising, which comes at a cost , allows investment into faster propagation of the agents’ traits to adjacent individuals. Importantly, game strategy and advertising strategy are subject to the same evolutionary mechanism. Via analytical reasoning and numerical simulations I demonstrate that a range of advertising costs exists, such that the prevalence of cooperation is significantly enhanced through co-evolution. Linking costly replication to the success of cooperators exposes a novel co-evolutionary mechanism that might contribute towards a better understanding of the origins of cooperation-supporting heterogeneity in agent populations.

Collective chasing behavior between cooperators and defectors in the spatial prisoner's dilemma

Cooperation is one of the essential factors for all biological organisms in major evolutionary transitions. Recent studies have investigated the effect of migration for the evolution of cooperation. However, little is known about whether and how an individuals' cooperativeness coevolves with mobility. One possibility is that mobility enhances cooperation by enabling cooperators to escape from defectors and form clusters; the other possibility is that mobility inhibits cooperation by helping the defectors to catch and exploit the groups of cooperators. In this study we investigate the coevolutionary dynamics by using the prisoner's dilemma game model on a lattice structure. The computer simulations demonstrate that natural selection maintains cooperation in the form of evolutionary chasing between the cooperators and defectors. First, cooperative groups grow and collectively move in the same direction. Then, mutant defectors emerge and invade the cooperative groups, after which the defectors exploit the cooperators. Then other cooperative groups emerge due to mutation and the cycle is repeated. Here, it is worth noting that, as a result of natural selection, the mobility evolves towards directional migration, but not to random or completely fixed migration. Furthermore, with directional migration, the rate of global population extinction is lower when compared with other cases without the evolution of mobility (i.e., when mobility is preset to random or fixed). These findings illustrate the coevolutionary dynamics of cooperation and mobility through the directional chasing between cooperators and defectors.

Wisdom of groups promotes cooperation in evolutionary social dilemmas

Whether or not to change strategy depends not only on the personal success of each individual, but also on the success of others. Using this as motivation, we study the evolution of cooperation in games that describe social dilemmas, where the propensity to adopt a different strategy depends both on individual fitness as well as on the strategies of neighbors. Regardless of whether the evolutionary process is governed by pairwise or group interactions, we show that plugging into the “wisdom of groups” strongly promotes cooperative behavior. The more the wider knowledge is taken into account the more the evolution of defectors is impaired. We explain this by revealing a dynamically decelerated invasion process, by means of which interfaces separating different domains remain smooth and defectors therefore become unable to efficiently invade cooperators. This in turn invigorates spatial reciprocity and establishes decentralized decision making as very beneficial for resolving social dilemmas.

Network reciprocity by coexisting learning and teaching strategies

We propose a network reciprocity model in which an agent probabilistically adopts learning or teaching strategies. In the learning adaptation mechanism, an agent may copy a neighbor's strategy through Fermi pairwise comparison. The teaching adaptation mechanism involves an agent imposing its strategy on a neighbor. Our simulations reveal that the reciprocity is significantly affected by the frequency with which learning and teaching agents coexist in a network and by the structure of the network itself.

Competition of individual and institutional punishments in spatial public goods games

We have studied the evolution of strategies in spatial public goods games where both individual (peer) and institutional (pool) punishments are present in addition to unconditional defector and cooperator strategies. The evolution of strategy distribution is governed by imitation based on the random sequential comparison of neighbors' payoff for a fixed level of noise. Using numerical simulations, we evaluate the strategy frequencies and phase diagrams when varying the synergy factor, punishment cost, and fine. Our attention is focused on two extreme cases describing all the relevant behaviors in such a complex system. According to our numerical data peer punishers prevail and control the system behavior in a large segments of parameters while pool punishers can only survive in the limit of weak peer punishment when a rich variety of solutions is observed. Paradoxically, the two types of punishment may extinguish each other's impact, resulting in the triumph of defectors. The technical difficulties and suggested methods are briefly discussed.

Effects of competition on pattern formation in the rock-paper-scissors game

We investigate the impact of cyclic competition on pattern formation in the rock-paper-scissors game. By separately considering random and prepared initial conditions, we observe a critical influence of the competition rate p on the stability of spiral waves and on the emergence of biodiversity. In particular, while increasing values of p promote biodiversity, they may act detrimentally on spatial pattern formation. For random initial conditions, we observe a phase transition from biodiversity to an absorbing phase, whereby the critical value of mobility grows linearly with increasing values of p on a log-log scale but then saturates as p becomes large. For prepared initial conditions, we observe the formation of single-armed spirals, but only for values of p that are below a critical value. Once above that value, the spirals break up and form disordered spatial structures, mainly because of the percolation of vacant sites. Thus there exists a critical value of the competition rates p(c) for stable single-armed spirals in finite populations. Importantly though, p(c) increases with increasing system size because noise reinforces the disintegration of ordered patterns. In addition, we also find that p(c) increases with the mobility. These phenomena are reproduced by a deterministic model that is based on nonlinear partial differential equations. Our findings indicate that competition is vital for the sustenance of biodiversity and the emergence of pattern formation in ecosystems governed by cyclical interactions.

Resolution of the stochastic strategy spatial prisoner's dilemma by means of particle swarm optimization

We study the evolution of cooperation among selfish individuals in the stochastic strategy spatial prisoner's dilemma game. We equip players with the particle swarm optimization technique, and find that it may lead to highly cooperative states even if the temptations to defect are strong. The concept of particle swarm optimization was originally introduced within a simple model of social dynamics that can describe the formation of a swarm, i.e., analogous to a swarm of bees searching for a food source. Essentially, particle swarm optimization foresees changes in the velocity profile of each player, such that the best locations are targeted and eventually occupied. In our case, each player keeps track of the highest payoff attained within a local topological neighborhood and its individual highest payoff. Thus, players make use of their own memory that keeps score of the most profitable strategy in previous actions, as well as use of the knowledge gained by the swarm as a whole, to find the best available strategy for themselves and the society. Following extensive simulations of this setup, we find a significant increase in the level of cooperation for a wide range of parameters, and also a full resolution of the prisoner's dilemma. We also demonstrate extreme efficiency of the optimization algorithm when dealing with environments that strongly favor the proliferation of defection, which in turn suggests that swarming could be an important phenomenon by means of which cooperation can be sustained even under highly unfavorable conditions. We thus present an alternative way of understanding the evolution of cooperative behavior and its ubiquitous presence in nature, and we hope that this study will be inspirational for future efforts aimed in this direction.

Phase diagrams for the spatial public goods game with pool punishment

The efficiency of institutionalized punishment is studied by evaluating the stationary states in the spatial public goods game comprising unconditional defectors, cooperators, and cooperating pool punishers as the three competing strategies. Fines and costs of pool punishment are considered as the two main parameters determining the stationary distributions of strategies on the square lattice. Each player collects a payoff from five five-person public goods games, and the evolution of strategies is subsequently governed by imitation based on pairwise comparisons at a low level of noise. The impact of pool punishment on the evolution of cooperation in structured populations is significantly different from that reported previously for peer punishment. Representative phase diagrams reveal remarkably rich behavior, depending also on the value of the synergy factor that characterizes the efficiency of investments payed into the common pool. Besides traditional single- and two-strategy stationary states, a rock-paper-scissors type of cyclic dominance can emerge in strikingly different ways.