Effect of memory on the prisoner's dilemma game in a square lattice

Emergence of cooperation under punishment: A reinforcement learning perspective

Punishment is a common tactic to sustain cooperation and has been extensively studied for a long time. While most of previous game-theoretic work adopt the imitation learning framework where players imitate the strategies of those who are better off, the learning logic in the real world is often much more complex. In this work, we turn to the reinforcement learning paradigm, where individuals make their decisions based upon their experience and long-term returns. Specifically, we investigate the prisoners' dilemma game with a Q-learning algorithm, and cooperators probabilistically pose punishment on defectors in their neighborhood. Unexpectedly, we find that punishment could lead to either continuous or discontinuous cooperation phase transitions, and the nucleation process of cooperation clusters is reminiscent of the liquid-gas transition. The analysis of a Q-table reveals the evolution of the underlying "psychologic" changes, which explains the nucleation process and different levels of cooperation. The uncovered first-order phase transition indicates that great care needs to be taken when implementing the punishment compared to the continuous scenario.

On Robust Stability and Stabilization of Networked Evolutionary Games with Time Delays

This paper investigates the robust stability and stabilization of networked evolutionary games (NEGs) with time delays. First, a mathematical model is presented to describe the dynamics of NEG with time-varying delays and disturbances. Second, an auxiliary system is constructed using the semi-tensor product of matrices and a dimension augmenting technique. Then, a verification condition of robust stability is derived. Third, in order to stabilize NEG to the Nash equilibrium, the robust stability problem is transformed into the robust stabilization problem. Moreover, an algorithm is proposed to design the stabilization controller. Finally, the validity of the results is verified by an example.

The effect of asymmetric reproductive ability on the evolution of cooperation on interdependent networks

In this paper, we consider an asymmetric reproductive ability on interdependent networks and investigate how this setting affects the evolution of cooperation. In detail, players decide to update their strategies at each step on main network (network B), while for sub network (network A), players update their strategies with a fixed probability p. Obviously, the system restores the traditional case when p = 1, where cooperation can survive by interdependent network reciprocity. And our asymmetric set-up comes into play when p < 1. Numerical simulation results show that our asymmetric coupling will hinder the overall cooperation level for small p. In detail, the introduction of asymmetric reproductive ability urges the formation of symmetry breaking and further weakens the positive impact by location synchronous effect. And the root cause is entirely distinct situation of utility differences on two networks. These observations further demonstrate a class of phenomena on interdependent networks that it would have catastrophic consequences on one network even if a unrelated change only occurs seemingly on another network.

Investment preference promotes cooperation in spatial public goods game

It is usually assumed that each cooperator contributes equally to different public pools in spatial public goods game. However, it is more reasonable to invest differently according to individual investment preference. In this paper, an extended public goods game, in which cooperators contribute to the groups according to the investment preference, is developed. The investment preference of a cooperator is characterized by the fraction of the cooperator from his/her own memory about a group and the intensity of investment preference is represented by a tunable parameter α. The well-mixed population and the structured population are analyzed under this mechanism. It is shown that the investment preference can give rise to coordination. Moreover, the extensive numerical simulation results show that with the increasing of investment preference density or memory length, the proportion of cooperation can increase monotonously. This is because the investment preference could help cooperators resist the invasion from defectors. Compared with the basic version, the new mechanism is able to promote cooperation effectively. Our research may provide a valuable insight for further exploring the nature of cooperation in the real world.

The evolution of cooperation in signed networks under the impact of structural balance

Structural balance plays an important role in the dynamics of signed networks. Based on structural balance, we generalize the evolution of cooperation in signed networks. Here we develop a new simulation model to study the impact of structural balance on the evolution of cooperation in signed networks. The simulation shows that cooperation prevails when an individual has a higher probability of adjusting the signs of its relations. We also find that structural balance forces the coexistence of cooperators and defectors, while the initial attributes of networks have little impact on the evolution of cooperation in the presence of structural balance, although they have a strong effect on the evolution of structural balance.

Changes in political party systems arising from conflict and transfer among political parties

Conflict that arises between two groups of different paradigms is an inevitable phenomenon, and a representative example of the conflict among different groups is a conflict phenomenon caused by competition among political parties. In this paper, we study the dynamical behavior of a political party system. Considering three major political parties, we investigate how political party systems can be changed by employing a mathematical model. By considering the transfer mechanism of recruitment as well as conflict of competition between political parties, we found that all parties are likely to coexist when both the competition and transfer between the parties are weak, or if either mechanism can occur at a relatively low level. Otherwise, a political party system is changed to a single-party system. In addition, we found that when a party system was changed into a single-party system, it appeared to be either bistable or multistable, and has been elucidate by linear stability analysis. Our results may provide insights to understand mechanisms how political party systems can be changed by conflict and transfer.

Impact of Social Reward on the Evolution of the Cooperation Behavior in Complex Networks

Social reward, as a significant mechanism explaining the evolution of cooperation, has attracted great attention both theoretically and experimentally. In this paper, we study the evolution of cooperation by proposing a reward model in network population, where a third strategy, reward, as an independent yet particular type of cooperation is introduced in 2-person evolutionary games. Specifically, a new kind of role corresponding to reward strategy, reward agents, is defined, which is aimed at increasing the income of cooperators by applying to them a social reward. Results from numerical simulations show that consideration of social reward greatly promotes the evolution of cooperation, which is confirmed for different network topologies and two evolutionary games. Moreover, we explore the microscopic mechanisms for the promotion of cooperation in the three-strategy model. As expected, the reward agents play a vital role in the formation of cooperative clusters, thus resisting the aggression of defectors. Our research might provide valuable insights into further exploring the nature of cooperation in the real world.

Mobility, fitness collection, and the breakdown of cooperation

The spatial arrangement of individuals is thought to overcome the dilemma of cooperation: When cooperators engage in clusters, they might share the benefit of cooperation while being more protected against noncooperating individuals, who benefit from cooperation but save the cost of cooperation. This is paradigmatically shown by the spatial prisoner's dilemma model. Here, we study this model in one and two spatial dimensions, but explicitly take into account that in biological setups, fitness collection and selection are separated processes occurring mostly on vastly different time scales. This separation is particularly important to understand the impact of mobility on the evolution of cooperation. We find that even small diffusive mobility strongly restricts cooperation since it enables noncooperative individuals to invade cooperative clusters. Thus, in most biological scenarios, where the mobility of competing individuals is an irrefutable fact, the spatial prisoner's dilemma alone cannot explain stable cooperation, but additional mechanisms are necessary for spatial structure to promote the evolution of cooperation. The breakdown of cooperation is analyzed in detail. We confirm the existence of a phase transition, here controlled by mobility and costs, which distinguishes between purely cooperative and noncooperative absorbing states. While in one dimension the model is in the class of the voter model, it belongs to the directed percolation universality class in two dimensions.

Optimizing cooperation on complex networks in the presence of failure

Cooperation has been recognized as a fundamental driving force in many natural, social, and economic systems. We investigate whether, given a complex-networked system in which agents (nodes) interact with one another according to the rules of evolutionary games and are subject to failure or death, cooperation can prevail and be optimized. We articulate a control scheme to maximize cooperation by introducing a time tolerance, a time duration that sustains an agent even if its payoff falls below a threshold. Strikingly, we find that a significant cooperation cluster can emerge when the time tolerance is approximately uniformly distributed over the network. A heuristic theory is derived to understand the optimization mechanism, which emphasizes the role played by medium-degree nodes. Implications for policy making to prevent or mitigate large-scale cascading breakdown are pointed out.

Cooperative behavior in evolutionary snowdrift games with the unconditional imitation rule on regular lattices

We study an evolutionary snowdrift game with the unconditional imitation updating rule on regular lattices. Detailed numerical simulations establish the structure of plateaus and discontinuous jumps of the equilibrium cooperation frequency f(c) as a function of the cost-to-benefit ratio r. By analyzing the stability of local configurations, it is found that the transitions occur at values of r at which there are changes in the ranking of the payoffs to the different local configurations of agents using different strategies. Nonmonotonic behavior of f(c)(r) at the intermediate range of r is analyzed in terms of the formation of blocks of agents using the cooperative strategy that are stabilized by agents inside the block due to the updating rule. For random initial condition with 50%-50% agents of different strategies randomly dispersed, cooperation persists in the whole range of r and the level of cooperation is higher than that in the well-mixed case in a wide range of r. These results are in sharp contrast to those based on the replicator updating rule. The sensitivity to initial states with different fractions of cooperative agents is also discussed. The results serve to illustrate that both the spatial structure and the updating rule are important in determining the level of cooperation in a competing population. When extreme initial states are used where there are very few agents of a strategy in a background of the opposite strategy, the result would depend on the stability of the clusters formed by the initially minority agents.

Limited memory can be beneficial for the evolution of cooperation

In this study we analyze the effect of working memory capacity on the evolution of cooperation and show a case in which societies with strongly limited memory achieve higher levels of cooperation than societies with larger memory. Agents in our evolutionary model are arranged on a network and interact in a prisoner's dilemma with their neighbors. They learn from their own experience and that of their neighbors in the network about the past behavior of others and use this information when making their choices. Each agent can only process information from her last h interactions. We show that if memory (h) is too short, cooperation does not emerge in the long run. A slight increase of memory length to around 5-10 periods, though, can lead to largely cooperative societies. Longer memory, on the other hand, is detrimental to cooperation in our model.

Cascading failures and the emergence of cooperation in evolutionary-game based models of social and economical networks

We study catastrophic behaviors in large networked systems in the paradigm of evolutionary games by incorporating a realistic "death" or "bankruptcy" mechanism. We find that a cascading bankruptcy process can arise when defection strategies exist and individuals are vulnerable to deficit. Strikingly, we observe that, after the catastrophic cascading process terminates, cooperators are the sole survivors, regardless of the game types and of the connection patterns among individuals as determined by the topology of the underlying network. It is necessary that individuals cooperate with each other to survive the catastrophic failures. Cooperation thus becomes the optimal strategy and absolutely outperforms defection in the game evolution with respect to the "death" mechanism. Our results can be useful for understanding large-scale catastrophe in real-world systems and in particular, they may yield insights into significant social and economical phenomena such as large-scale failures of financial institutions and corporations during an economic recession.

Diversity of reproduction time scale promotes cooperation in spatial prisoner's dilemma games

We study an evolutionary spatial prisoner's dilemma game where the fitness of the players is determined by both the payoffs from the current interaction and their history. We consider the situation where the selection time scale is slower than the interaction time scale. This is done by implementing probabilistic reproduction on an individual level. We observe that both too fast and too slow reproduction rates hamper the emergence of cooperation. In other words, there exists an intermediate selection time scale that maximizes cooperation. Another factor we find to promote cooperation is a diversity of reproduction time scales.

Effects of strategy-migration direction and noise in the evolutionary spatial prisoner's dilemma

Spatial games are crucial for understanding patterns of cooperation in nature (and to some extent society). They are known to be more sensitive to local symmetries than, e.g., spin models. This paper concerns the evolution of the prisoner's dilemma game on regular lattices with three different types of neighborhoods--the von Neumann, Moore, and kagomé types. We investigate two kinds of dynamics for the players to update their strategies (that can be unconditional cooperator or defector). Depending on the payoff difference, an individual can adopt the strategy of a random neighbor [a voter-model-like dynamics (VMLD)] or impose its strategy on a random neighbor, i.e., invasion-process-like dynamics (IPLD). In particular, we focus on the effects of noise, in combination with the strategy dynamics, on the evolution of cooperation. We find that VMLD, compared to IPLD, better supports the spreading and sustaining of cooperation. We see that noise has nontrivial effects on the evolution of cooperation: maximum cooperation density can be realized either at a medium noise level, in the limit of zero noise or in both these regions. The temptation to defect and the local interaction structure determine the outcome. Especially, in the low noise limit, the local interaction plays a crucial role in determining the fate of cooperators. We elucidate these both by numerical simulations and mean-field cluster approximation methods.