Memory-based snowdrift game on networks

Cooperation in public goods games: Leveraging other-regarding reinforcement learning on hypergraphs

Cooperation is a self-organized collective behavior. It plays a significant role in the evolution of both ecosystems and human society. Reinforcement learning is different from imitation learning, offering a new perspective for exploring cooperation mechanisms. However, most existing studies with the public goods game (PGG) employ a self-regarding setup or are on pairwise interaction networks. Players in the real world, however, optimize their policies based not only on their histories but also on the histories of their coplayers, and the game is played in a group manner. In this work, we investigate the evolution of cooperation in the PGG under the other-regarding reinforcement learning evolutionary game on hypergraph by combining the Q-learning algorithm and evolutionary game framework, where other players' action history is incorporated and the game is played on hypergraphs. Our results show that as the synergy factor r[over ̂] increases, the parameter interval divides into three distinct regions-the absence of cooperation, medium cooperation, and high cooperation-accompanied by two abrupt transitions in the cooperation level near r[over ̂]_{1}^{*} and r[over ̂]_{2}^{*}, respectively. Interestingly, we identify regular and anticoordinated chessboard structures in the spatial pattern that positively contribute to the first cooperation transition but adversely affect the second. Furthermore, we provide a theoretical treatment for the first transition with an approximated r[over ̂]_{1}^{*} and reveal that players with a long-sighted perspective and low exploration rate are more likely to reciprocate kindness with each other, thus facilitating the emergence of cooperation. Our findings contribute to understanding the evolution of human cooperation, where other-regarding information and group interactions are commonplace.

Memory and target payoff enhance cooperation in evolutionary social dilemmas

We proposed a neighbor selection mechanism based on memory and target payoff, where the target payoff is the maximum value of the group's average expected payoff. According to this mechanism, individuals prioritize selecting neighbors whose average payoffs in the last M rounds are close to the target payoff for strategy learning, aiming to maximize the group's expected payoff. Simulation results on the grid-based Prisoner's Dilemma and Snowdrift games demonstrate that this mechanism can significantly improve the group's payoff and cooperation level. Furthermore, the longer the memory length, the higher the group's payoff and cooperation level. Overall, the combination of memory and target payoff can lead to the emergence and persistence of cooperation in social dilemmas as individuals are motivated to cooperate based on both their past experiences and future goals. This interplay highlights the significance of taking into account numerous variables in comprehending and promoting cooperation within evolutionary frameworks.

Emergence of cooperation under punishment: A reinforcement learning perspective

Punishment is a common tactic to sustain cooperation and has been extensively studied for a long time. While most of previous game-theoretic work adopt the imitation learning framework where players imitate the strategies of those who are better off, the learning logic in the real world is often much more complex. In this work, we turn to the reinforcement learning paradigm, where individuals make their decisions based upon their experience and long-term returns. Specifically, we investigate the prisoners' dilemma game with a Q-learning algorithm, and cooperators probabilistically pose punishment on defectors in their neighborhood. Unexpectedly, we find that punishment could lead to either continuous or discontinuous cooperation phase transitions, and the nucleation process of cooperation clusters is reminiscent of the liquid-gas transition. The analysis of a Q-table reveals the evolution of the underlying "psychologic" changes, which explains the nucleation process and different levels of cooperation. The uncovered first-order phase transition indicates that great care needs to be taken when implementing the punishment compared to the continuous scenario.

Interpersonal trust: Asymptotic analysis of a stochastic coordination game with multi-agent learning

We study the interpersonal trust of a population of agents, asking whether chance may decide if a population ends up with high trust or low trust. We model this by a discrete time, stochastic coordination game with pairwise interactions occurring at random in a finite population. Agents learn about the behavior of the population using a weighted average of what they have observed in past interactions. This learning rule, called an "exponential moving average," has one parameter that determines the weight of the most recent observation and may, thus, be interpreted as the agent's memory. We prove analytically that in the long run, the whole population always either trusts or doubts with the probability one. This remains true when the expectation of the dynamics would indicate otherwise. By simulation, we study the impact of the distribution of the payoff matrix and of the memory of the agents. We find that as the agent memory increases (i.e., the most recent observation weighs less), the actual dynamics increasingly resemble the expectation of the process. We conclude that it is possible that a population may converge upon high or low trust between its citizens simply by chance, though the game parameters (context of the society) may be quite telling.

Reputation and reciprocity

Reputation and reciprocity are key mechanisms for cooperation in human societies, often going hand in hand to favor prosocial behavior over selfish actions. Here we review recent researches at the interface of physics and evolutionary game theory that explored these two mechanisms. We focus on image scoring as the bearer of reputation, as well as on various types of reciprocity, including direct, indirect, and network reciprocity. We review different definitions of reputation and reciprocity dynamics, and we show how these affect the evolution of cooperation in social dilemmas. We consider first-order, second-order, as well as higher-order models in well-mixed and structured populations, and we review experimental works that support and inform the results of mathematical modeling and simulations. We also provide a synthesis of the reviewed researches along with an outlook in terms of six directions that seem particularly promising to explore in the future.

Prisoner's dilemma game model Based on historical strategy information

In many dilemmas, decisions are determined not by a single factor, but by multiple ones, including memory, reputation, reward and punishment. In recent years, how to design a mechanism to promote cooperation has become a research hot-spot. However, most of the previous studies mainly consider the historical benefits of the game, and pay less attention to the stability of the strategy (the frequency of strategy changes in the length of memory) and the proportion of memory in decision-making. The decision-making process of group evolution involves the influence of memory information on cooperative evolution in multi round games. It makes up for the lack of stability factors and weights in previous studies. Based on the above factors, a new strategy update rule is proposed to study the influence of the stability of historical strategy information on the evolution of cooperation in prisoner's dilemma game, and the influence of memory weight on cooperation is considered. The stability of the current strategy is measured by the strategy in historical memory (the number of times the strategy in memory is continuous and consistent with the current strategy), which can determine the probability of an individual learning the neighbor strategy next time. Numerical simulation shows that an appropriate increase in the length of historical memory is more conducive to the emergence of cooperation, and the greater the weight of historical strategy information is, the more conducive to promoting cooperation, which shows that historical strategy information is still the main factor in decision-making. This study will help us understand the cooperative evolution of many real systems, such as nature, biology, society and so on, and effectively design reasonable mechanisms to promote cooperation.

Incorporating social payoff into reinforcement learning promotes cooperation

Reinforcement learning has been demonstrated to be an effective approach to investigate the dynamic of strategy updating and the learning process of agents in game theory. Most studies have shown that Q-learning failed to resolve the problem of cooperation in well-mixed populations or homogeneous networks. To this aim, we investigate the self-regarding Q-learning's effect on cooperation in spatial prisoner's dilemma games by incorporating the social payoff. Here, we redefine the reward term of self-regarding Q-learning by involving the social payoff; that is, the reward is defined as a monotonic function of the individual payoff and the social payoff represented by its neighbors' payoff. Numerical simulations reveal that such a framework can facilitate cooperation remarkably because the social payoff ensures agents learn to cooperate toward socially optimal outcomes. Moreover, we find that self-regarding Q-learning is an innovative rule that ensures cooperators coexist with defectors even at high temptations to defection. The investigation of the emergence and stability of the sublattice-ordered structure shows that such a mechanism tends to generate a checkerboard pattern to increase agents' payoff. Finally, the effects of Q-learning parameters are also analyzed, and the robustness of this mechanism is verified on different networks.

Dynamical reciprocity in interacting games: Numerical results and mechanism analysis

We study the evolution of two mutually interacting pairwise games on different topologies. On two-dimensional square lattices, we reveal that the game-game interaction can promote the cooperation prevalence in all cases, and the cooperation-defection phase transitions even become absent and fairly high cooperation is expected when the interaction becomes very strong. A mean-field theory is developed that points out dynamical routes arising therein. Detailed analysis shows indeed that there are rich categories of interactions in either the individual or bulk scenario: invasion, neutral, and catalyzed types; their combination puts cooperators at a persistent advantage position, which boosts the cooperation. The robustness of the revealed reciprocity is strengthened by the studies of model variants, including the public goods game, asymmetrical or time-varying interactions, games of different types, games with timescale separation, different updating rules, etc. The structural complexities of the underlying population, such as Newman-Watts small world networks, Erdős-Rényi random networks, and Barabási-Albert networks, also do not alter the working of the dynamical reciprocity. In particular, as the number of games engaged increases, the cooperation level continuously improves in general. However, our analysis shows that the dynamical reciprocity works only in structured populations, otherwise the game-game interaction has no any impact on the cooperation at all. In brief, our work uncovers a cooperation mechanism in the structured populations, which indicates the great potential for human cooperation since concurrent issues are so often seen in the real world.

Evolution of Cooperation in Social Dilemmas with Assortative Interactions

Cooperation in social dilemmas plays a pivotal role in the formation of systems at all levels of complexity, from replicating molecules to multi-cellular organisms to human and animal societies. In spite of its ubiquity, the origin and stability of cooperation pose an evolutionary conundrum, since cooperation, though beneficial to others, is costly to the individual cooperator. Thus natural selection would be expected to favor selfish behavior in which individuals reap the benefits of cooperation without bearing the costs of cooperating themselves. Many proximate mechanisms have been proposed to account for the origin and maintenance of cooperation, including kin selection, direct reciprocity, indirect reciprocity, and evolution in structured populations. Despite the apparent diversity of these approaches they all share a unified underlying logic: namely, each mechanism results in assortative interactions in which individuals using the same strategy interact with a higher probability than they would at random. Here we study the evolution of cooperation in both discrete strategy and continuous strategy social dilemmas with assortative interactions. For the sake of tractability, assortativity is modeled by an individual interacting with another of the same type with probability r and interacting with a random individual in the population with probability 1−r, where r is a parameter that characterizes the degree of assortativity in the system. For discrete strategy social dilemmas we use both a generalization of replicator dynamics and individual-based simulations to elucidate the donation, snowdrift, and sculling games with assortative interactions, and determine the analogs of Hamilton’s rule, which govern the evolution of cooperation in these games. For continuous strategy social dilemmas we employ both a generalization of deterministic adaptive dynamics and individual-based simulations to study the donation, snowdrift, and tragedy of the commons games, and determine the effect of assortativity on the emergence and stability of cooperation.

Effect of memory, intolerance, and second-order reputation on cooperation

The understanding of cooperative behavior in social systems has been the subject of intense research over the past few decades. In this regard, the theoretical models used to explain cooperation in human societies have been complemented with a growing interest in experimental studies to validate the proposed mechanisms. In this work, we rely on previous experimental findings to build a theoretical model based on two cooperation driving mechanisms: second-order reputation and memory. Specifically, taking the donation game as a starting point, the agents are distributed among three strategies, namely, unconditional cooperators, unconditional defectors, and discriminators, where the latter follow a second-order assessment rule: shunning, stern judging, image scoring, or simple standing. A discriminator will cooperate if the evaluation of the recipient's last actions contained in his memory is above a threshold of (in)tolerance. In addition to the dynamics inherent to the game, another imitation dynamics, involving much longer times (generations), is introduced. The model is approached through a mean-field approximation that predicts the macroscopic behavior observed in Monte Carlo simulations. We found that, while in most second-order assessment rules, intolerance hinders cooperation, it has the opposite (positive) effect under the simple standing rule. Furthermore, we show that, when considering memory, the stern judging rule shows the lowest values of cooperation, while stricter rules show higher cooperation levels.

Understanding collective behaviors in reinforcement learning evolutionary games via a belief-based formalization

Collective behaviors by self-organization are ubiquitous in nature and human society and extensive efforts have been made to explore the mechanisms behind them. Artificial intelligence (AI) as a rapidly developing field is of great potential for these tasks. By combining reinforcement learning with evolutionary game (RLEG), we numerically discover a rich spectrum of collective behaviors-explosive events, oscillation, and stable states, etc., that are also often observed in the human society. In this work, we aim to provide a theoretical framework to investigate the RLEGs systematically. Specifically, we formalize AI-agents' learning processes in terms of belief switches and behavior modes defined as a series of actions following beliefs. Based on the preliminary results in the time-independent environment, we investigate the stability at the mixed equilibrium points in RLEGs generally, in which agents reside in one of the optimal behavior modes. Moreover, we adopt the maximum entropy principle to infer the composition of agents residing in each mode at a strictly stable point. When the theoretical analysis is applied to the 2×2 game setting, we can explain the uncovered collective behaviors and are able to construct equivalent systems intuitively. Also, the inferred composition of different modes is consistent with simulations. Our work may be helpful to understand the related collective emergence in human society as well as behavioral patterns at the individual level and potentially facilitate human-computer interactions in the future.

Cooperation in a generalized age-structured spatial game

The emergence and prevalence of cooperative behavior within a group of selfish individuals remains a puzzle for evolutionary game theory precisely because it conflicts directly with the central idea of natural selection. Accordingly, in recent years, the search for an understanding of how cooperation can be stimulated, even when it conflicts with individual interest, has intensified. We investigate the emergence of cooperation in an age-structured evolutionary spatial game. In it, players age with time and the payoff that they receive after each round depends on their age. We find that the outcome of the game is strongly influenced by the type of distribution used to modify the payoffs according to the age of each player. The results show that, under certain circumstances, cooperators may not only survive but dominate the population.

Environmental feedback promotes the evolution of cooperation in the structured populations

Environment plays a vital role in individual decision-making. In the game process, employing the strategy of the opponent who behaves better is nontrivial for the evolution and maintenance of cooperation, in that such a behavior may assist the player in achieving higher personal interests and more competitive superiorities. Enlightened by this thought, a coevolutionary model where the mechanisms of dynamic environment and preference selection are introduced in the networked prisoner's dilemma game is considered. Individual preference selection is introduced in the strategy update process to probe how the preferences of the latent strategy sources interfere with individual behaviors. The environment defined in the model is not only heterogeneous, but also evolves with the evolution of individual strategies. Through conducting large-scale Monte Carlo simulations, we draw a conclusion that the introduction of evolutionary environment-related preference selection is an effective promoter of cooperation even under a severe temptation. Our exploration indicates that the coevolutionary model may have a practical significance and provide a new insight into the analysis of the origin of cooperation in structured populations for further research.

Investment preference promotes cooperation in spatial public goods game

It is usually assumed that each cooperator contributes equally to different public pools in spatial public goods game. However, it is more reasonable to invest differently according to individual investment preference. In this paper, an extended public goods game, in which cooperators contribute to the groups according to the investment preference, is developed. The investment preference of a cooperator is characterized by the fraction of the cooperator from his/her own memory about a group and the intensity of investment preference is represented by a tunable parameter α. The well-mixed population and the structured population are analyzed under this mechanism. It is shown that the investment preference can give rise to coordination. Moreover, the extensive numerical simulation results show that with the increasing of investment preference density or memory length, the proportion of cooperation can increase monotonously. This is because the investment preference could help cooperators resist the invasion from defectors. Compared with the basic version, the new mechanism is able to promote cooperation effectively. Our research may provide a valuable insight for further exploring the nature of cooperation in the real world.

Changes in political party systems arising from conflict and transfer among political parties

Conflict that arises between two groups of different paradigms is an inevitable phenomenon, and a representative example of the conflict among different groups is a conflict phenomenon caused by competition among political parties. In this paper, we study the dynamical behavior of a political party system. Considering three major political parties, we investigate how political party systems can be changed by employing a mathematical model. By considering the transfer mechanism of recruitment as well as conflict of competition between political parties, we found that all parties are likely to coexist when both the competition and transfer between the parties are weak, or if either mechanism can occur at a relatively low level. Otherwise, a political party system is changed to a single-party system. In addition, we found that when a party system was changed into a single-party system, it appeared to be either bistable or multistable, and has been elucidate by linear stability analysis. Our results may provide insights to understand mechanisms how political party systems can be changed by conflict and transfer.

Coevolution of Vertex Weights Resolves Social Dilemma in Spatial Networks

In realistic social system, the role or influence of each individual varies and adaptively changes in time in the population. Inspired by this fact, we thus consider a new coevolution setup of game strategy and vertex weight on a square lattice. In detail, we model the structured population on a square lattice, on which the role or influence of each individual is depicted by vertex weight, and the prisoner’s dilemma game has been applied to describe the social dilemma of pairwise interactions of players. Through numerical simulation, we conclude that our coevolution setup can promote the evolution of cooperation effectively. Especially, there exists a moderate value of δ for each ε that can warrant an optimal resolution of social dilemma. For a further understanding of these results, we find that intermediate value of δ enables the strongest heterogeneous distribution of vertex weight. We hope our coevolution setup of vertex weight will provide new insight for the future research.

Win-stay-lose-learn promotes cooperation in the prisoner's dilemma game with voluntary participation

Voluntary participation, demonstrated to be a simple yet effective mechanism to promote persistent cooperative behavior, has been extensively studied. It has also been verified that the aspiration-based win-stay-lose-learn strategy updating rule promotes the evolution of cooperation. Inspired by this well-known fact, we combine the Win-Stay-Lose-Learn updating rule with voluntary participation: Players maintain their strategies when they are satisfied, or players attempt to imitate the strategy of one randomly chosen neighbor. We find that this mechanism maintains persistent cooperative behavior, even further promotes the evolution of cooperation under certain conditions.

Impact of Social Reward on the Evolution of the Cooperation Behavior in Complex Networks

Social reward, as a significant mechanism explaining the evolution of cooperation, has attracted great attention both theoretically and experimentally. In this paper, we study the evolution of cooperation by proposing a reward model in network population, where a third strategy, reward, as an independent yet particular type of cooperation is introduced in 2-person evolutionary games. Specifically, a new kind of role corresponding to reward strategy, reward agents, is defined, which is aimed at increasing the income of cooperators by applying to them a social reward. Results from numerical simulations show that consideration of social reward greatly promotes the evolution of cooperation, which is confirmed for different network topologies and two evolutionary games. Moreover, we explore the microscopic mechanisms for the promotion of cooperation in the three-strategy model. As expected, the reward agents play a vital role in the formation of cooperative clusters, thus resisting the aggression of defectors. Our research might provide valuable insights into further exploring the nature of cooperation in the real world.

Evolutionary mixed games in structured populations: Cooperation and the benefits of heterogeneity

Evolutionary games on networks traditionally involve the same game at each interaction. Here we depart from this assumption by considering mixed games, where the game played at each interaction is drawn uniformly at random from a set of two different games. While in well-mixed populations the random mixture of the two games is always equivalent to the average single game, in structured populations this is not always the case. We show that the outcome is, in fact, strongly dependent on the distance of separation of the two games in the parameter space. Effectively, this distance introduces payoff heterogeneity, and the average game is returned only if the heterogeneity is small. For higher levels of heterogeneity the distance to the average game grows, which often involves the promotion of cooperation. The presented results support preceding research that highlights the favorable role of heterogeneity regardless of its origin, and they also emphasize the importance of the population structure in amplifying facilitators of cooperation.

Evolution of Cooperation in Social Dilemmas on Complex Networks

Cooperation in social dilemmas is essential for the functioning of systems at multiple levels of complexity, from the simplest biological organisms to the most sophisticated human societies. Cooperation, although widespread, is fundamentally challenging to explain evolutionarily, since natural selection typically favors selfish behavior which is not socially optimal. Here we study the evolution of cooperation in three exemplars of key social dilemmas, representing the prisoner's dilemma, hawk-dove and coordination classes of games, in structured populations defined by complex networks. Using individual-based simulations of the games on model and empirical networks, we give a detailed comparative study of the effects of the structural properties of a network, such as its average degree, variance in degree distribution, clustering coefficient, and assortativity coefficient, on the promotion of cooperative behavior in all three classes of games.

Evolutionary dynamics of general group interactions in structured populations

The evolution of populations is influenced by many factors, and the simple classical models have been developed in a number of important ways. Both population structure and multiplayer interactions have been shown to significantly affect the evolution of important properties, such as the level of cooperation or of aggressive behavior. Here we combine these two key factors and develop the evolutionary dynamics of general group interactions in structured populations represented by regular graphs. The traditional linear and threshold public goods games are adopted as models to address the dynamics. We show that for linear group interactions, population structure can favor the evolution of cooperation compared to the well-mixed case, and we see that the more neighbors there are, the harder it is for cooperators to persist in structured populations. We further show that threshold group interactions could lead to the emergence of cooperation even in well-mixed populations. Here population structure sometimes inhibits cooperation for the threshold public goods game, where depending on the benefit to cost ratio, the outcomes are bistability or a monomorphic population of defectors or cooperators. Our results suggest, counterintuitively, that structured populations are not always beneficial for the evolution of cooperation for nonlinear group interactions.

Evolution of Cooperation on Spatial Network with Limited Resource

Considering the external resource offered by environment is limited, here, we will explore the cooperation on spatial networks with limited resource. The individual distributes the limited resource according to the payoffs acquired in games, and one with resource amounts is lower than critical survival resource level will be replaced by the offspring of its neighbors. We find that, for larger temptation to defect, the number of the dead decreases with the resource amount. However the cooperation behavior is interesting, the lower total resource and intermediate temptation to defect can greatly promote the cooperation on square lattice. Our result reveals that the limited resource contributes most to the cooperation.

Universal scaling for the dilemma strength in evolutionary games

Why would natural selection favor the prevalence of cooperation within the groups of selfish individuals? A fruitful framework to address this question is evolutionary game theory, the essence of which is captured in the so-called social dilemmas. Such dilemmas have sparked the development of a variety of mathematical approaches to assess the conditions under which cooperation evolves. Furthermore, borrowing from statistical physics and network science, the research of the evolutionary game dynamics has been enriched with phenomena such as pattern formation, equilibrium selection, and self-organization. Numerous advances in understanding the evolution of cooperative behavior over the last few decades have recently been distilled into five reciprocity mechanisms: direct reciprocity, indirect reciprocity, kin selection, group selection, and network reciprocity. However, when social viscosity is introduced into a population via any of the reciprocity mechanisms, the existing scaling parameters for the dilemma strength do not yield a unique answer as to how the evolutionary dynamics should unfold. Motivated by this problem, we review the developments that led to the present state of affairs, highlight the accompanying pitfalls, and propose new universal scaling parameters for the dilemma strength. We prove universality by showing that the conditions for an ESS and the expressions for the internal equilibriums in an infinite, well-mixed population subjected to any of the five reciprocity mechanisms depend only on the new scaling parameters. A similar result is shown to hold for the fixation probability of the different strategies in a finite, well-mixed population. Furthermore, by means of numerical simulations, the same scaling parameters are shown to be effective even if the evolution of cooperation is considered on the spatial networks (with the exception of highly heterogeneous setups). We close the discussion by suggesting promising directions for future research including (i) how to handle the dilemma strength in the context of co-evolution and (ii) where to seek opportunities for applying the game theoretical approach with meaningful impact.

Evolutionary dynamics of synergistic and discounted group interactions in structured populations

The emergence of cooperation between unrelated individuals enables researchers to study how the collective cooperative behavior survives in a world where egoists could get more short-term benefits. The spatial multi-player games, which invoke interactions between individuals who are not directly linked by the interactive networks, are drawing more and more attention in exploring the evolution of cooperation. Here we address the evolutionary dynamics in infinite structured populations with discounted, linear, and synergistic group interactions. The five classical scenarios are recovered from the dynamics: (i) dominating defection, (ii) dominating cooperation, (iii) co-existence, (iv) bi-stability, and (v) neutral variants. For linear interactions, the evolutionary dynamics is equivalent to that in finite as well as the well-mixed counterparts, which can be achieved by a payoff matrix transformation, and it illustrates that the more neighbors there are, the harder the cooperators survive. Yet both cooperation and defection emerge easier in finite populations than in infinite for discounted and synergistic interactions. Counterintuitively, we find that the synergistic group interactions always raise cooperators׳ barriers to occupy the population with the increase of the number of neighbors in infinite structured populations. Our results go against the common belief that synergistic interactions are necessarily beneficial for the cooperative behavior.

Dependency links can hinder the evolution of cooperation in the prisoner's dilemma game on lattices and networks

Networks with dependency links are more vulnerable when facing the attacks. Recent research also has demonstrated that the interdependent groups support the spreading of cooperation. We study the prisoner's dilemma games on spatial networks with dependency links, in which a fraction of individual pairs is selected to depend on each other. The dependency individuals can gain an extra payoff whose value is between the payoff of mutual cooperation and the value of temptation to defect. Thus, this mechanism reflects that the dependency relation is stronger than the relation of ordinary mutual cooperation, but it is not large enough to cause the defection of the dependency pair. We show that the dependence of individuals hinders, promotes and never affects the cooperation on regular ring networks, square lattice, random and scale-free networks, respectively. The results for the square lattice and regular ring networks are demonstrated by the pair approximation.

Experiments on individual strategy updating in iterated snowdrift game under random rematching

How do people actually play the iterated snowdrift games, particularly under random rematching protocol is far from well explored. Two sets of laboratory experiments on snowdrift game were conducted to investigate human strategy updating rules. Four groups of subjects were modeled by experience-weighted attraction learning theory at individual-level. Three out of the four groups (75%) passed model validation. Substantial heterogeneity is observed among the players who update their strategies in four typical types, whereas rare people behave like belief-based learners even under fixed pairing. Most subjects (63.9%) adopt the reinforcement learning (or alike) rules; but, interestingly, the performance of averaged reinforcement learners suffered. It is observed that two factors seem to benefit players in competition, i.e., the sensitivity to their recent experiences and the overall consideration of forgone payoffs. Moreover, subjects with changing opponents tend to learn faster based on their own recent experience, and display more diverse strategy updating rules than they do with fixed opponent. These findings suggest that most of subjects do apply reinforcement learning alike updating rules even under random rematching, although these rules may not improve their performance. The findings help evolutionary biology researchers to understand sophisticated human behavioral strategies in social dilemmas.

Phase transitions in a coevolving snowdrift game with costly rewiring

We propose and study a dissatisfied adaptive snowdrift game with a payoff parameter r that incorporates a cost for rewiring a connection. An agent, facing adverse local environment, may switch action without a cost or rewire an existing link with a cost a so as to attain a better competing environment. Detailed numerical simulations reveal nontrivial and nonmonotonic dependence of the frequency of cooperation and the densities of different types of links on a and r. A theory that treats the cooperative and noncooperative agents separately and accounts for spatial correlation up to neighboring agents is formulated. The theory gives results that are in good agreement with simulations. The frequency of cooperation f_{C} is enhanced (suppressed) at high rewiring cost relative to that at low rewiring cost when r is small (large). For a given value of r, there exists a critical value of the rewiring cost below which the system evolves into a phase of frozen dynamics with isolated noncooperative agents segregated from a cluster of cooperative agents, and above which the system evolves into a connected population of mixed actions with continual dynamics. The phase boundary on the a-r phase space that separates the two phases with distinct structural, population and dynamical properties is mapped out. The phase diagram reveals that, as a general feature, for small r (small a), the disconnected and segregated phase can survive over a wider range of a(r).

Aspiration-based partner switching boosts cooperation in social dilemmas

Most previous studies concerning linking dynamics often assumed that links pairing individuals should be identified and treated differently during topology adjusting procedure, in order to promote cooperation. A common assumption was that cooperators were expected to avoid being exploited by quickly breaking up relationships with defectors. Then the so-called prosocial links linking two cooperators (abbreviated as CC links hereafter) would be much favored by evolution, whereby cooperation was promoted. However, we suggest that this is not always necessary. Here, we developed a minimal model in which an aspiration-based partner switching mechanism was embedded to regulate the evolution of cooperation in social dilemmas. Individuals adjusted social ties in a self-questioning manner in line with the learning theory. Less game information was involved during dynamic linking and all links were tackled anonymously irrespective of their types (i.e., CD links, DD links, or CC links). The main results indicate that cooperation flourishes for a broad range of parameters. The denser the underlying network, the more difficult the evolution of cooperation. More importantly, moderate aspirations do much better in promoting the evolution of altruistic behavior and for most cases there exists the optimal aspiration level that most benefits cooperation. Too strong or too weak selection intensity turns out to be pretty conducive to the evolution of cooperation in such a dynamical system.

Aspiration dynamics of multi-player games in finite populations

On studying strategy update rules in the framework of evolutionary game theory, one can differentiate between imitation processes and aspiration-driven dynamics. In the former case, individuals imitate the strategy of a more successful peer. In the latter case, individuals adjust their strategies based on a comparison of their pay-offs from the evolutionary game to a value they aspire, called the level of aspiration. Unlike imitation processes of pairwise comparison, aspiration-driven updates do not require additional information about the strategic environment and can thus be interpreted as being more spontaneous. Recent work has mainly focused on understanding how aspiration dynamics alter the evolutionary outcome in structured populations. However, the baseline case for understanding strategy selection is the well-mixed population case, which is still lacking sufficient understanding. We explore how aspiration-driven strategy-update dynamics under imperfect rationality influence the average abundance of a strategy in multi-player evolutionary games with two strategies. We analytically derive a condition under which a strategy is more abundant than the other in the weak selection limiting case. This approach has a long-standing history in evolutionary games and is mostly applied for its mathematical approachability. Hence, we also explore strong selection numerically, which shows that our weak selection condition is a robust predictor of the average abundance of a strategy. The condition turns out to differ from that of a wide class of imitation dynamics, as long as the game is not dyadic. Therefore, a strategy favoured under imitation dynamics can be disfavoured under aspiration dynamics. This does not require any population structure, and thus highlights the intrinsic difference between imitation and aspiration dynamics.

Robustness of controllability for networks based on edge-attack

We study the controllability of networks in the process of cascading failures under two different attacking strategies, random and intentional attack, respectively. For the highest-load edge attack, it is found that the controllability of Erdős-Rényi network, that with moderate average degree, is less robust, whereas the Scale-free network with moderate power-law exponent shows strong robustness of controllability under the same attack strategy. The vulnerability of controllability under random and intentional attacks behave differently with the increasing of removal fraction, especially, we find that the robustness of control has important role in cascades for large removal fraction. The simulation results show that for Scale-free networks with various power-law exponents, the network has larger scale of cascades do not mean that there will be more increments of driver nodes. Meanwhile, the number of driver nodes in cascading failures is also related to the edges amount in strongly connected components.

Impact of social punishment on cooperative behavior in complex networks

Social punishment is a mechanism by which cooperative individuals spend part of their resources to penalize defectors. In this paper, we study the evolution of cooperation in 2-person evolutionary games on networks when a mechanism for social punishment is introduced. Specifically, we introduce a new kind of role, punisher, which is aimed at reducing the earnings of defectors by applying to them a social fee. Results from numerical simulations show that different equilibria allowing the three strategies to coexist are possible as well as that social punishment further enhance the robustness of cooperation. Our results are confirmed for different network topologies and two evolutionary games. In addition, we analyze the microscopic mechanisms that give rise to the observed macroscopic behaviors in both homogeneous and heterogeneous networks. Our conclusions might provide additional insights for understanding the roots of cooperation in social systems.

Towards a snowdrift game optimization to vertex cover of networks

To solve the vertex cover problem in an agent-based and distributed networking systems utilizing local information, we treat each vertex as an intelligent rational agent rather than an inanimate one and provide a spatial-snowdrift-game-based optimization framework to vertex cover of networks. We analyze the inherent relation between the snowdrift game and the vertex cover: Strict Nash equilibriums of the spatial snowdrift game are the intermediate states between vertex-covered and minimal-vertex-covered states. Such equilibriums are obtained by employing the memory-based best response update rule. We also find that a better approximate solution in terms of the minimal vertex cover will be achieved by increasing the individuals' memory length, because such a process optimizes the individuals' strategies and helps them convert from bad equilibriums into better ones. Our findings pave a new way to solve the vertex cover problem from the perspective of agent-based self-organized optimization.

Evacuation of pedestrians from a single room by using snowdrift game theories

Game theory is introduced to simulate the complicated interaction relations among the conflicting pedestrians in a pedestrian flow system, which is defined on a square lattice with the parallel update rule. Modified on the traditional lattice gas model, each pedestrian can move to not only an empty site, but also an occupied site. It is found that each individual chooses its neighbor randomly and occupies the site with the probability W(x→y)=1/1+exp[-(P(x)-U(x))/κ], where P(x) is the x's payoff representing his personal energy, and U(x) is the average payoff of its neighborhood indicating the potential well energy if he stays. Two types of pedestrians are considered, and they interact with their neighbors following the payoff matrix of snowdrift game theory. The cost-to-benefit ratio r=c/(2b-c) (where b is the perfect payoff and c is the labor cost) represents the fear index of the pedestrians in this model. It is found that there exists a moderate value of r leading to the shortest escape time, and the situation for large values of r is better than that for small ones in general. In addition, the pedestrian flow system always arrives at a consistent state in which the two types of walkers have the same number and evolve by the same law irrespectively of the parameters, which can be interpreted as the self-organization effect of pedestrian flow. It is also proven that the time point of the onset of the steady state is unrelated to the scale of the pedestrians and the square lattice. Meanwhile, the system exhibits different dynamics before reaching the consistent state: the number of the two types of walkers oscillates when P(C)>0.5 (i.e., probability to change the present strategy), while no oscillation happens for P(C)≤0.5. Finally, it is shown that a smaller density of pedestrians ρ induces a shorter average escape time.

Optimizing cooperation on complex networks in the presence of failure

Cooperation has been recognized as a fundamental driving force in many natural, social, and economic systems. We investigate whether, given a complex-networked system in which agents (nodes) interact with one another according to the rules of evolutionary games and are subject to failure or death, cooperation can prevail and be optimized. We articulate a control scheme to maximize cooperation by introducing a time tolerance, a time duration that sustains an agent even if its payoff falls below a threshold. Strikingly, we find that a significant cooperation cluster can emerge when the time tolerance is approximately uniformly distributed over the network. A heuristic theory is derived to understand the optimization mechanism, which emphasizes the role played by medium-degree nodes. Implications for policy making to prevent or mitigate large-scale cascading breakdown are pointed out.

Inferring reputation promotes the evolution of cooperation in spatial social dilemma games

In realistic world individuals with high reputation are more likely to influence the collective behaviors. Due to the cost and error of information dissemination, however, it is unreasonable to assign each individual with a complete cognitive power, which means that not everyone can accurately realize others’ reputation situation. Here we introduce the mechanism of inferring reputation into the selection of potential strategy sources to explore the evolution of cooperation. Before the game each player is assigned with a randomly distributed parameter p denoting his ability to infer the reputation of others. The parameter p of each individual is kept constant during the game. The value of p indicates that the neighbor possessing highest reputation is chosen with the probability p and randomly choosing an opponent is left with the probability 1−p. We find that this novel mechanism can be seen as an universally applicable promoter of cooperation, which works on various interaction networks and in different types of evolutionary game. Of particular interest is the fact that, in the early stages of evolutionary process, cooperators with high reputation who are easily regarded as the potential strategy donors can quickly lead to the formation of extremely robust clusters of cooperators that are impervious to defector attacks. These clusters eventually help cooperators reach their undisputed dominance, which transcends what can be warranted by the spatial reciprocity alone. Moreover, we provide complete phase diagrams to depict the impact of uncertainty in strategy adoptions and conclude that the effective interaction topology structure may be altered under such a mechanism. When the estimation of reputation is extended, we also show that the moderate value of evaluation factor enables cooperation to thrive best. We thus present a viable method of understanding the ubiquitous cooperative behaviors in nature and hope that it will inspire further studies to resolve social dilemmas.

Different reactions to adverse neighborhoods in games of cooperation

In social dilemmas, cooperation among randomly interacting individuals is often difficult to achieve. The situation changes if interactions take place in a network where the network structure jointly evolves with the behavioral strategies of the interacting individuals. In particular, cooperation can be stabilized if individuals tend to cut interaction links when facing adverse neighborhoods. Here we consider two different types of reaction to adverse neighborhoods, and all possible mixtures between these reactions. When faced with a gloomy outlook, players can either choose to cut and rewire some of their links to other individuals, or they can migrate to another location and establish new links in the new local neighborhood. We find that in general local rewiring is more favorable for the evolution of cooperation than emigration from adverse neighborhoods. Rewiring helps to maintain the diversity in the degree distribution of players and favors the spontaneous emergence of cooperative clusters. Both properties are known to favor the evolution of cooperation on networks. Interestingly, a mixture of migration and rewiring is even more favorable for the evolution of cooperation than rewiring on its own. While most models only consider a single type of reaction to adverse neighborhoods, the coexistence of several such reactions may actually be an optimal setting for the evolution of cooperation.

Cooperative behavior in evolutionary snowdrift games with the unconditional imitation rule on regular lattices

We study an evolutionary snowdrift game with the unconditional imitation updating rule on regular lattices. Detailed numerical simulations establish the structure of plateaus and discontinuous jumps of the equilibrium cooperation frequency f(c) as a function of the cost-to-benefit ratio r. By analyzing the stability of local configurations, it is found that the transitions occur at values of r at which there are changes in the ranking of the payoffs to the different local configurations of agents using different strategies. Nonmonotonic behavior of f(c)(r) at the intermediate range of r is analyzed in terms of the formation of blocks of agents using the cooperative strategy that are stabilized by agents inside the block due to the updating rule. For random initial condition with 50%-50% agents of different strategies randomly dispersed, cooperation persists in the whole range of r and the level of cooperation is higher than that in the well-mixed case in a wide range of r. These results are in sharp contrast to those based on the replicator updating rule. The sensitivity to initial states with different fractions of cooperative agents is also discussed. The results serve to illustrate that both the spatial structure and the updating rule are important in determining the level of cooperation in a competing population. When extreme initial states are used where there are very few agents of a strategy in a background of the opposite strategy, the result would depend on the stability of the clusters formed by the initially minority agents.

Effects of dynamical grouping on cooperation in N-person evolutionary snowdrift game

A population typically consists of agents that continually distribute themselves into different groups at different times. This dynamic grouping has recently been shown to be essential in explaining many features observed in human activities including social, economic, and military activities. We study the effects of dynamic grouping on the level of cooperation in a modified evolutionary N-person snowdrift game. Due to the formation of dynamical groups, the competition takes place in groups of different sizes at different times and players of different strategies are mixed by the grouping dynamics. It is found that the level of cooperation is greatly enhanced by the dynamic grouping of agents, when compared with a static population of the same size. As a parameter β, which characterizes the relative importance of the reward and cost, increases, the fraction of cooperative players f(C) increases and it is possible to achieve a fully cooperative state. Analytically, we present a dynamical equation that incorporates the effects of the competing game and group size distribution. The distribution of cooperators in different groups is assumed to be a binomial distribution, which is confirmed by simulations. Results from the analytic equation are in good agreement with numerical results from simulations. We also present detailed simulation results of f(C) over the parameter space spanned by the probabilities of group coalescence ν(m) and group fragmentation ν(p) in the grouping dynamics. A high ν(m) and low ν(p) promotes cooperation, and a favorable reward characterized by a high β would lead to a fully cooperative state.

Cascading failures and the emergence of cooperation in evolutionary-game based models of social and economical networks

We study catastrophic behaviors in large networked systems in the paradigm of evolutionary games by incorporating a realistic "death" or "bankruptcy" mechanism. We find that a cascading bankruptcy process can arise when defection strategies exist and individuals are vulnerable to deficit. Strikingly, we observe that, after the catastrophic cascading process terminates, cooperators are the sole survivors, regardless of the game types and of the connection patterns among individuals as determined by the topology of the underlying network. It is necessary that individuals cooperate with each other to survive the catastrophic failures. Cooperation thus becomes the optimal strategy and absolutely outperforms defection in the game evolution with respect to the "death" mechanism. Our results can be useful for understanding large-scale catastrophe in real-world systems and in particular, they may yield insights into significant social and economical phenomena such as large-scale failures of financial institutions and corporations during an economic recession.

Crossover between structured and well-mixed networks in an evolutionary prisoner's dilemma game

In a spatial evolutionary prisoner's dilemma game (PDG), individuals interact with their neighbors and update their strategies according to some rules. As is well known, cooperators are destined to become extinct in a well-mixed population, whereas they could emerge and be sustained on a structured network. In this work, we introduce a simple model to investigate the crossover between a structured network and a well-mixed one in an evolutionary PDG. In the model, each link j is designated a rewiring parameter τ(j), which defines the time interval between two successive rewiring events for link j. By adjusting the rewiring parameter τ (the mean time interval for any link in the network), we could change a structured network into a well-mixed one. For the link rewiring events, three situations are considered: one synchronous situation and two asynchronous situations. Simulation results show that there are three regimes of τ: large τ where the density of cooperators ρ(c) rises to ρ(c,∞) (the value of ρ(c) for the case without link rewiring), small τ where the mean-field description for a well-mixed network is applicable, and moderate τ where the crossover between a structured network and a well-mixed one happens.

Coveting thy neighbors fitness as a means to resolve social dilemmas

In spatial evolutionary games the fitness of each individual is traditionally determined by the payoffs it obtains upon playing the game with its neighbors. Since defection yields the highest individual benefits, the outlook for cooperators is gloomy. While network reciprocity promotes collaborative efforts, chances of averting the impending social decline are slim if the temptation to defect is strong. It is, therefore, of interest to identify viable mechanisms that provide additional support for the evolution of cooperation. Inspired by the fact that the environment may be just as important as inheritance for individual development, we introduce a simple switch that allows a player to either keep its original payoff or use the average payoff of all its neighbors. Depending on which payoff is higher, the influence of either option can be tuned by means of a single parameter. We show that, in general, taking into account the environment promotes cooperation. Yet coveting the fitness of one's neighbors too strongly is not optimal. In fact, cooperation thrives best only if the influence of payoffs obtained in the traditional way is equal to that of the average payoff of the neighborhood. We present results for the prisoner's dilemma and the snowdrift game, for different levels of uncertainty governing the strategy adoption process, and for different neighborhood sizes. Our approach outlines a viable route to increased levels of cooperative behavior in structured populations, but one that requires a thoughtful implementation.

Ordering in spatial evolutionary games for pairwise collective strategy updates

Evolutionary 2×2 games are studied with players located on a square lattice. During the evolution the randomly chosen neighboring players try to maximize their collective income by adopting a random strategy pair with a probability dependent on the difference of their summed payoffs between the final and initial states assuming quenched strategies in their neighborhood. In the case of the anticoordination game this system behaves like an antiferromagnetic kinetic Ising model. Within a wide region of social dilemmas this dynamical rule supports the formation of similar spatial arrangement of the cooperators and defectors ensuring the optimum total payoff if the temptation to choose defection exceeds a threshold value dependent on the sucker's payoff. The comparison of the results with those achieved for pairwise imitation and myopic strategy updates has indicated the relevant advantage of pairwise collective strategy update in the maintenance of cooperation.

Emergence of cooperation through coevolving time scale in spatial prisoner's dilemma

We study evolutionary prisoner's dilemma game by considering adaptive strategy-selection time scale among individuals according to a "win-slower, lose-faster" rule: if an individual successfully resists the invasion of an opponent, she is prone to hold her strategy for longer time through decreasing her strategy-selection time scale; otherwise, she increases the time scale because of losing. We find that the greater the losers increase their strategy-selection time scales, the better for cooperation. Interestingly, optimal cooperation can be induced by proper adaptive rate in the strategy-selection time scale. Our results may have potential implications in the design of consensus protocol in multiagent systems.

Effective usage of credit records promotes cooperation on weighted networks

The cooperative behaviors of players on weighted networks are investigated by incorporation of trust mechanisms into a well-accepted game model, i.e., the networked prisoner's dilemma game, afterwards some weight-updating schemes are designed according to the credit records. Despite the differences in network topologies and strategy updating protocols, a simple yet significant principle surfaces that, to promote the emergence of cooperation over abundant weighted networks, only the latest credit record of partners is required to be taken into consideration, whereas incorporating more previous records may even deteriorate the cooperation performance. To support such an appealing principle, we have investigated more deeply into the role of credit records so as to give a detailed explanation underlying it. The virtue of this work lies in providing insights into the effective usage of the currently available credit records.

Role of adaptive migration in promoting cooperation in spatial games

Recent work has revealed that success-driven migration can promote cooperation among selfish individuals in evolutionary games. This migration mechanism relies, however, on nonlocal information about the states of the individuals and their computational capabilities for prediction. We investigate the role of adaptive migration in cooperative behavior in the framework of spatial game by proposing an alternative migration strategy that requires only local information obtainable through game interactions. Our results demonstrate that adaptive migration can be effective in promoting cooperation in two ways. First, there exists an optimal degree of migration associated with the density of empty sites and migration speed, which leads to the highest level of cooperation. Second, adaptive migration can induce an outbreak of cooperation from an environment dominated by defectors. These findings hold for common types of evolutionary games that involve pairwise interactions.