Evolution of extortion in structured populations

Pink-noise dynamics in an evolutionary game on a regular graph

We consider a multiplayer prisoner's dilemma game on a square lattice and regular graphs based on the pairwise-Fermi update rule, and we obtain heatmaps of the fraction of cooperators and the correlation of neighboring pairs. In the heatmap, we find a mixed region where cooperators and defectors coexist, and the correlation between neighbors is significantly enhanced. Moreover, we observe pink-noise behavior in the mixed region, where the power spectrum can be fitted by a power-law function of frequency. We also find that the pink-noise behavior can be reproduced in a simple random-walk model. In particular, we propose a modified random-walk model which can reproduce not only the pink-noise behavior but also the deviation from it observed in a low-frequency region.

Personal sustained cooperation based on networked evolutionary game theory

Evolutionary game theory on complex networks provides an effective theoretical tool to explain the emergence of sustained cooperative behavior. Human society has formed various organizational networks. The network structure and individual behavior take on a variety of forms. This diversity provides the basis for choice, so it is crucial for the emergence of cooperation. This article provides a dynamic algorithm for individual network evolution, and calculates the importance of different nodes in the network evolution process. In the dynamic evolution simulation, the probability of the cooperation strategy and betrayal strategy is described. In the individual interaction network, cooperative behavior will promote the continuous evolution of individual relationships and form a better aggregative interpersonal network. The interpersonal network of betrayal has been in a relatively loose state, and its continuity must rely on the participation of new nodes, but there will be certain "weak links" in the existing nodes of the network.

Adaptive dynamics of memory-one strategies in the repeated donation game

Human interactions can take the form of social dilemmas: collectively, people fare best if all cooperate but each individual is tempted to free ride. Social dilemmas can be resolved when individuals interact repeatedly. Repetition allows them to adopt reciprocal strategies which incentivize cooperation. The most basic model for direct reciprocity is the repeated donation game, a variant of the prisoner's dilemma. Two players interact over many rounds; in each round they decide whether to cooperate or to defect. Strategies take into account the history of the play. Memory-one strategies depend only on the previous round. Even though they are among the most elementary strategies of direct reciprocity, their evolutionary dynamics has been difficult to study analytically. As a result, much previous work has relied on simulations. Here, we derive and analyze their adaptive dynamics. We show that the four-dimensional space of memory-one strategies has an invariant three-dimensional subspace, generated by the memory-one counting strategies. Counting strategies record how many players cooperated in the previous round, without considering who cooperated. We give a partial characterization of adaptive dynamics for memory-one strategies and a full characterization for memory-one counting strategies.

Mutation enhances cooperation in direct reciprocity

Direct reciprocity is a powerful mechanism for the evolution of cooperation based on repeated interactions between the same individuals. But high levels of cooperation evolve only if the benefit-to-cost ratio exceeds a certain threshold that depends on memory length. For the best-explored case of one-round memory, that threshold is two. Here, we report that intermediate mutation rates lead to high levels of cooperation, even if the benefit-to-cost ratio is only marginally above one, and even if individuals only use a minimum of past information. This surprising observation is caused by two effects. First, mutation generates diversity which undermines the evolutionary stability of defectors. Second, mutation leads to diverse communities of cooperators that are more resilient than homogeneous ones. This finding is relevant because many real-world opportunities for cooperation have small benefit-to-cost ratios, which are between one and two, and we describe how direct reciprocity can attain cooperation in such settings. Our result can be interpreted as showing that diversity, rather than uniformity, promotes evolution of cooperation.

The evolutionary extortion game of multiple groups in hypernetworks

As a type of zero-determinant strategies, the extortion strategy was found to be an evolutionarily stable strategy in structural groups. However, instead of complex networks structure, this paper focus on a multi-group game in hypernetworks, using the framework of a gift giving game driven by replicator-like dynamics. We find that the extortion is evolutionarily stable in the hypernetwork structure. The extortion game in hypernetworks can promote the emergence of the cooperative behavior compared to the traditional dual-strategy game and the extortion game in complex networks. The results show that the cooperation behavior attracts most of the groups for the smaller benefit value. With the increase of benefit value, cooperators turn into defectors and extortioners, but cooperation behavior still survives in hypernetworks under extreme conditions. Moreover, small-scale groups are more conducive to cooperation.

Adapting paths against zero-determinant strategies in repeated prisoner's dilemma games

Long-term cooperation, competition, or exploitation among individuals can be modeled through repeated games. In repeated games, Press and Dyson discovered zero-determinant (ZD) strategies that enforce a special relationship between two players. This special relationship implies that a ZD player can unilaterally impose a linear payoff relationship to the opponent regardless of the opponent's strategies. A ZD player also has a property that can lead the opponent to an unconditional cooperation if the opponent tries to improve its payoff. This property has been mathematically confirmed by Chen and Zinger. Humans often underestimate a payoff obtained in the future. However, such discounting was not considered in their analysis. Here, we mathematically explored whether a ZD player can lead the opponent to an unconditional cooperation even if a discount factor is incorporated. Specifically, we represented the expected payoff with a discount factor as the form of determinants and calculated whether the values obtained by partially differentiating each factor in the strategy vector become positive. As a result, we proved that the strategy vector ends up as an unconditional cooperation even when starting from any initial strategy. This result was confirmed through numerical calculations. We extended the applicability of ZD strategies to real world problems.

Direct reciprocity between individuals that use different strategy spaces

In repeated interactions, players can use strategies that respond to the outcome of previous rounds. Much of the existing literature on direct reciprocity assumes that all competing individuals use the same strategy space. Here, we study both learning and evolutionary dynamics of players that differ in the strategy space they explore. We focus on the infinitely repeated donation game and compare three natural strategy spaces: memory-1 strategies, which consider the last moves of both players, reactive strategies, which respond to the last move of the co-player, and unconditional strategies. These three strategy spaces differ in the memory capacity that is needed. We compute the long term average payoff that is achieved in a pairwise learning process. We find that smaller strategy spaces can dominate larger ones. For weak selection, unconditional players dominate both reactive and memory-1 players. For intermediate selection, reactive players dominate memory-1 players. Only for strong selection and low cost-to-benefit ratio, memory-1 players dominate the others. We observe that the supergame between strategy spaces can be a social dilemma: maximum payoff is achieved if both players explore a larger strategy space, but smaller strategy spaces dominate.

Neighborhood size effects on the evolution of cooperation under myopic dynamics

We study the evolution of cooperation in 2×2 social dilemma games in which players are located on a two-dimensional square lattice. During the evolution, each player modifies her strategy by means of myopic update dynamic to maximize her payoff while composing neighborhoods of different sizes, which are characterized by the corresponding radius, r. An investigation of the sublattice-ordered spatial structure for different values of r reveals that some patterns formed by cooperators and defectors can help the former to survive, even under untoward conditions. In contrast to individuals who resist the invasion of defectors by forming clusters due to network reciprocity, innovators spontaneously organize a socially divisive structure that provides strong support for the evolution of cooperation and advances better social systems.

A unified framework of direct and indirect reciprocity

Direct and indirect reciprocity are key mechanisms for the evolution of cooperation. Direct reciprocity means that individuals use their own experience to decide whether to cooperate with another person. Indirect reciprocity means that they also consider the experiences of others. Although these two mechanisms are intertwined, they are typically studied in isolation. Here, we introduce a mathematical framework that allows us to explore both kinds of reciprocity simultaneously. We show that the well-known 'generous tit-for-tat' strategy of direct reciprocity has a natural analogue in indirect reciprocity, which we call 'generous scoring'. Using an equilibrium analysis, we characterize under which conditions either of the two strategies can maintain cooperation. With simulations, we additionally explore which kind of reciprocity evolves when members of a population engage in social learning to adapt to their environment. Our results draw unexpected connections between direct and indirect reciprocity while highlighting important differences regarding their evolvability.

Human players manage to extort more than the mutual cooperation payoff in repeated social dilemmas

Social dilemmas are mixed-motive games. Although the players have a common interest in maintaining cooperation, each may try to obtain a larger payoff by cooperating less than the other. This phenomenon received increased attention after Press and Dyson discovered a class of strategies for the repeated prisoner's dilemma (extortionate strategies) that secure for themselves a payoff that is never smaller, but can be larger, than the opponent's payoff. We conducted an experiment to test whether humans adopt extortionate strategies when playing a social dilemma. Our results reveal that human subjects do try to extort a larger payoff from their opponents. However, they are only successful when extortionate strategies are part of a Nash equilibrium. In settings where extortionate strategies do not appear in any Nash equilibrium, attempts at extortion only result in a breakdown of cooperation. Our subjects recognized the different incentives implied by the two settings, and they were ready to "extort" the opponent when allowed to do so. This suggests that deviations from mutually cooperative equilibria, which are usually attributed to players' impatience, coordination problems, or lack of information, can instead be driven by subjects trying to reach more favorable outcomes.

Conditions for the existence of zero-determinant strategies under observation errors in repeated games

Repeated games are useful models to analyze long term interactions of living species and complex social phenomena. Zero-determinant (ZD) strategies in repeated games discovered by Press and Dyson in 2012 enforce a linear payoff relationship between a focal player and the opponent. This linear relationship can be set arbitrarily by a ZD player. Hence, a subclass of ZD strategies can fix the opponent's expected payoff and another subclass of the strategies can exceed the opponent for the expected payoff. Since this discovery, theories for ZD strategies are extended to cope with various natural situations. It is especially important to consider the theory of ZD strategies for repeated games with a discount factor and observation errors because it allows the theory to be applicable in the real world. Recent studies revealed their existence of ZD strategies even in repeated games with both factors. However, the conditions for the existence has not been sufficiently analyzed. Here, we mathematically analyzed the conditions in repeated games with both factors. First, we derived the thresholds of a discount factor and observation errors which ensure the existence of Equalizer and positively correlated ZD (pcZD) strategies, which are well-known subclasses of ZD strategies. We found that ZD strategies exist only when a discount factor remains high as the error rates increase. Next, we derived the conditions for the expected payoff of the opponent enforced by Equalizer as well as the conditions for the slope and base line payoff of linear lines enforced by pcZD. As a result, we found that, as error rates increase or a discount factor decreases, the conditions for the linear line that Equalizer or pcZD can enforce become strict.

Memory-two zero-determinant strategies in repeated games

Repeated games have provided an explanation of how mutual cooperation can be achieved even if defection is more favourable in a one-shot game in the Prisoner's Dilemma situation. Recently found zero-determinant (ZD) strategies have substantially been investigated in evolutionary game theory. The original memory-one ZD strategies unilaterally enforce linear relationships between average pay-offs of players. Here, we extend the concept of ZD strategies to memory-two strategies in repeated games. Memory-two ZD strategies unilaterally enforce linear relationships between correlation functions of pay-offs and pay-offs of the previous round. Examples of memory-two ZD strategy in the repeated Prisoner's Dilemma game are provided, some of which generalize the tit-for-tat strategy to a memory-two case. Extension of ZD strategies to memory-n case with n ≥ ~2 is also straightforward.

Five rules for friendly rivalry in direct reciprocity

Direct reciprocity is one of the key mechanisms accounting for cooperation in our social life. According to recent understanding, most of classical strategies for direct reciprocity fall into one of two classes, ‘partners’ or ‘rivals’. A ‘partner’ is a generous strategy achieving mutual cooperation, and a ‘rival’ never lets the co-player become better off. They have different working conditions: For example, partners show good performance in a large population, whereas rivals do in head-to-head matches. By means of exhaustive enumeration, we demonstrate the existence of strategies that act as both partners and rivals. Among them, we focus on a human-interpretable strategy, named ‘CAPRI’ after its five characteristic ingredients, i.e., cooperate, accept, punish, recover, and defect otherwise. Our evolutionary simulation shows excellent performance of CAPRI in a broad range of environmental conditions.

Zero-determinant strategies under observation errors in repeated games

Zero-determinant (ZD) strategies are a novel class of strategies in the repeated prisoner's dilemma (RPD) game discovered by Press and Dyson. This strategy set enforces a linear payoff relationship between a focal player and the opponent regardless of the opponent's strategy. In the RPD game, games with discounting and observation errors represent an important generalization, because they are better able to capture real life interactions which are often noisy. However, they have not been considered in the original discovery of ZD strategies. In some preceding studies, each of them has been considered independently. Here, we analytically study the strategies that enforce linear payoff relationships in the RPD game considering both a discount factor and observation errors. As a result, we first reveal that the payoffs of two players can be represented by the form of determinants as shown by Press and Dyson even with the two factors. Then, we search for all possible strategies that enforce linear payoff relationships and find that both ZD strategies and unconditional strategies are the only strategy sets to satisfy the condition. We also show that neither Extortion nor Generous strategies, which are subsets of ZD strategies, exist when there are errors. Finally, we numerically derive the threshold values above which the subsets of ZD strategies exist. These results contribute to a deep understanding of ZD strategies in society.

Automata representation of successful strategies for social dilemmas

In a social dilemma, cooperation is collectively optimal, yet individually each group member prefers to defect. A class of successful strategies of direct reciprocity were recently found for the iterated prisoner’s dilemma and for the iterated three-person public-goods game: By a successful strategy, we mean that it constitutes a cooperative Nash equilibrium under implementation error, with assuring that the long-term payoff never becomes less than the co-players’ regardless of their strategies, when the error rate is small. Although we have a list of actions prescribed by each successful strategy, the rationale behind them has not been fully understood for the iterated public-goods game because the list has hundreds of entries to deal with every relevant history of previous interactions. In this paper, we propose a method to convert such history-based representation into an automaton with a minimal number of states. Our main finding is that a successful strategy for the iterated three-person public-goods game can be represented as a 10-state automaton by this method. In this automaton, each state can be interpreted as the player’s internal judgement of the situation, such as trustworthiness of the co-players and the need to redeem oneself after defection. This result thus suggests a comprehensible way to choose an appropriate action at each step towards cooperation based on a situational judgement, which is mapped from the history of interactions.

Linear algebraic structure of zero-determinant strategies in repeated games

Zero-determinant (ZD) strategies, a recently found novel class of strategies in repeated games, has attracted much attention in evolutionary game theory. A ZD strategy unilaterally enforces a linear relation between average payoffs of players. Although existence and evolutional stability of ZD strategies have been studied in simple games, their mathematical properties have not been well-known yet. For example, what happens when more than one players employ ZD strategies have not been clarified. In this paper, we provide a general framework for investigating situations where more than one players employ ZD strategies in terms of linear algebra. First, we theoretically prove that a set of linear relations of average payoffs enforced by ZD strategies always has solutions, which implies that incompatible linear relations are impossible. Second, we prove that linear payoff relations are independent of each other under some conditions. These results hold for general games with public monitoring including perfect-monitoring games. Furthermore, we provide a simple example of a two-player game in which one player can simultaneously enforce two linear relations, that is, simultaneously control her and her opponent's average payoffs. All of these results elucidate general mathematical properties of ZD strategies.

Synergistic third-party rewarding and punishment in the public goods game

We study the evolution of cooperation in the spatial public goods game in the presence of third-party rewarding and punishment. The third party executes public intervention, punishing groups where cooperation is weak and rewarding groups where cooperation is strong. We consider four different scenarios to determine what works best for cooperation, in particular, neither rewarding nor punishment, only rewarding, only punishment or both rewarding and punishment. We observe strong synergistic effects when rewarding and punishment are simultaneously applied, which are absent if neither of the two incentives or just each individual incentive is applied by the third party. We find that public cooperation can be sustained at comparatively low third-party costs under adverse conditions, which is impossible if just positive or negative incentives are applied. We also examine the impact of defection tolerance and application frequency, showing that the higher the tolerance and the frequency of rewarding and punishment, the more cooperation thrives. Phase diagrams and characteristic spatial distributions of strategies are presented to corroborate these results, which will hopefully prove useful for more efficient public policies in support of cooperation in social dilemmas.

Heterogeneous indirect reciprocity promotes the evolution of cooperation in structured populations

A fundamental aspect of all biological system is cooperation. Human society is based to a large extent on mechanisms that favor cooperation. Indirect reciprocity is a crucial factor for the emergence and maintenance of cooperation in evolutionary games. In this work, we introduce a mechanism of heterogeneous indirect reciprocity in the prisoner's dilemma game, where an altruistic attribute is considered. The so-called altruistic attribute refers to the trait that when an altruistic individual cooperates, its neighbors, regardless of their strategies, can gain additional benefits. Intuitively, altruistic cooperative agents seem to do only what is beneficial for others, but in fact, their neighbors tend to cooperate in order to maintain the cooperative strategies of altruistic individuals. In this way, the neighbors of cooperative altruistic individuals not only guarantee their own additional benefits, but also indirectly protect the income of altruistic individuals, which strengthens the links between cooperative individuals, thus promoting the evolution of cooperation. The robustness of the results is verified on homogeneous and heterogeneous networks. Moreover, the results of individual heterogeneity corroborate the existing evidence that heterogeneity, almost irrespective of its origin, promotes cooperative actions. Our conclusions might provide additional insights into understanding the roots of cooperation in social systems.

Social Closure and the Evolution of Cooperation via Indirect Reciprocity

Direct and indirect reciprocity are good candidates to explain the fundamental problem of evolution of cooperation. We explore the conditions under which different types of reciprocity gain dominance and their performances in sustaining cooperation in the PD played on simple networks. We confirm that direct reciprocity gains dominance over indirect reciprocity strategies also in larger populations, as long as it has no memory constraints. In the absence of direct reciprocity, or when its memory is flawed, different forms of indirect reciprocity strategies are able to dominate and to support cooperation. We show that indirect reciprocity relying on social capital inherent in closed triads is the best competitor among them, outperforming indirect reciprocity that uses information from any source. Results hold in a wide range of conditions with different evolutionary update rules, extent of evolutionary pressure, initial conditions, population size, and density.

Seven rules to avoid the tragedy of the commons

Cooperation among self-interested players in a social dilemma is fragile and easily interrupted by mistakes. In this work, we study the repeated n-person public-goods game and search for a strategy that forms a cooperative Nash equilibrium in the presence of implementation error with a guarantee that the resulting payoff will be no less than any of the co-players'. By enumerating strategic possibilities for n=3, we show that such a strategy indeed exists when its memory length m equals three. It means that a deterministic strategy can be publicly employed to stabilize cooperation against error with avoiding the risk of being exploited. We furthermore show that, for general n-person public-goods game, m ≥ n is necessary to satisfy the above criteria.

Partners and rivals in direct reciprocity

Reciprocity is a major factor in human social life and accounts for a large part of cooperation in our communities. Direct reciprocity arises when repeated interactions occur between the same individuals. The framework of iterated games formalizes this phenomenon. Despite being introduced more than five decades ago, the concept keeps offering beautiful surprises. Recent theoretical research driven by new mathematical tools has proposed a remarkable dichotomy among the crucial strategies: successful individuals either act as partners or as rivals. Rivals strive for unilateral advantages by applying selfish or extortionate strategies. Partners aim to share the payoff for mutual cooperation, but are ready to fight back when being exploited. Which of these behaviours evolves depends on the environment. Whereas small population sizes and a limited number of rounds favour rivalry, partner strategies are selected when populations are large and relationships stable. Only partners allow for evolution of cooperation, while the rivals' attempt to put themselves first leads to defection.

Cooperation enhanced by the coevolution of teaching activity in evolutionary prisoner's dilemma games with voluntary participation

Voluntary participation, as an additional strategy involved in repeated games, has been proved to be an efficient way to promote the evolution of cooperation theoretically and empirically. Besides, current studies show that the coevolution of teaching activity can promote cooperation. Thus, inspired by aforementioned above, we investigate the effect of coevolution of teaching activity on the evolution of cooperation for prisoner's dilemma game with voluntary participation: when the focal player successfully enforces its strategy on the opponent, his teaching ability will get an increase. Through numerical simulation, we have shown that voluntary participation could effectively promote the fraction of cooperation, which is also affected by the value of increment. Furthermore, we investigate the influence of the increment value on the density of different strategies and find that there exists an optimal increment value that plays an utmost role on the evolutionary dynamics. With regard to this observation, we unveil that an optimal value of increment can lead to strongest heterogeneity in agents' teaching ability, further promoting the evolution of cooperation.

Network effects on coordination in asymmetric games

Network structure can have an important effect on the behavior of players in an iterated 2 × 2 game. We study the effect of network structure on global and local behavior in asymmetric coordination games using best response dynamics. We find that global behavior is highly dependent on network topology. Random (Erdös-Rényi) networks mostly converge to homogeneous behavior, but the higher the clustering in the network the more heterogeneous the behavior becomes. Behavior within the communities of the network is almost exclusively homogeneous. The findings suggest that clustering of networks facilitates self-organization of uniform behavior within clusters, but heterogeneous behavior between clusters. At the local level we find that some nodes are more important in determining the equilibrium behavior than other nodes. Degree centrality is for most networks the main predictor for the behavior and nodes with an even degree have an advantage over nodes with an uneven degree in dictating the behavior. We conclude that the behavior is difficult to predict for (Erdös-Rényi) networks and that the network imposes the behavior as a function of clustering and degree heterogeneity in other networks.

Divide-and-conquer Tournament on Social Networks

In social gaming networks, previous studies extensively investigated the influence of a variety of strategies on reciprocal behaviors in the prisoner's dilemma game. The studied frameworks range from the case that an individual uniformly cooperates or defects with all social contacts, to the recently reported divide-and-conquer games, where an individual can choose a particular move to play with each neighbor. In this paper, we investigate a divide-and-conquer tournament among 14 well-known strategies on social gaming networks. In the tournament, an individual's fitness is measured by accumulated and average payoff aggregated for a certain number of rounds. On the base of their fitness, the evolution of the population follows a local learning mechanism. Our observation indicates that the distribution of individuals adopting a strategy in degree ranking fundamentally changes the frequency of the strategy. In the divide-and-conquer gaming networks, our result suggests that the connectivity in social networks and strategy are two key factors that govern the evolution of the population.

Extortion provides alternative routes to the evolution of cooperation in structured populations

In this paper, we study the evolution of cooperation in structured populations (individuals are located on either a regular lattice or a scale-free network) in the context of repeated games by involving three types of strategies, namely, unconditional cooperation, unconditional defection, and extortion. The strategy updating of the players is ruled by the replicator-like dynamics. We find that extortion strategies can act as catalysts to promote the emergence of cooperation in structured populations via different mechanisms. Specifically, on regular lattice, extortioners behave as both a shield, which can enwrap cooperators inside and keep them away from defectors, and a spear, which can defeat those surrounding defectors with the help of the neighboring cooperators. Particularly, the enhancement of cooperation displays a resonance-like behavior, suggesting the existence of optimal extortion strength mostly favoring the evolution of cooperation, which is in good agreement with the predictions from the generalized mean-field approximation theory. On scale-free network, the hubs, who are likely occupied by extortioners or defectors at the very beginning, are then prone to be conquered by cooperators on small-degree nodes as time elapses, thus establishing a bottom-up mechanism for the emergence and maintenance of cooperation.

Role-separating ordering in social dilemmas controlled by topological frustration

''Three is a crowd" is an old proverb that applies as much to social interactions as it does to frustrated configurations in statistical physics models. Accordingly, social relations within a triangle deserve special attention. With this motivation, we explore the impact of topological frustration on the evolutionary dynamics of the snowdrift game on a triangular lattice. This topology provides an irreconcilable frustration, which prevents anticoordination of competing strategies that would be needed for an optimal outcome of the game. By using different strategy updating protocols, we observe complex spatial patterns in dependence on payoff values that are reminiscent to a honeycomb-like organization, which helps to minimize the negative consequence of the topological frustration. We relate the emergence of these patterns to the microscopic dynamics of the evolutionary process, both by means of mean-field approximations and Monte Carlo simulations. For comparison, we also consider the same evolutionary dynamics on the square lattice, where of course the topological frustration is absent. However, with the deletion of diagonal links of the triangular lattice, we can gradually bridge the gap to the square lattice. Interestingly, in this case the level of cooperation in the system is a direct indicator of the level of topological frustration, thus providing a method to determine frustration levels in an arbitrary interaction network.

Impact of migration on the multi-strategy selection in finite group-structured populations

For large quantities of spatial models, the multi-strategy selection under weak selection is the sum of two competition terms: the pairwise competition and the competition of multiple strategies with equal frequency. Two parameters σ₁ and σ₂ quantify the dependence of the multi-strategy selection on these two terms, respectively. Unlike previous studies, we here do not require large populations for calculating σ₁ and σ₂, and perform the first quantitative analysis of the effect of migration on them in group-structured populations of any finite sizes. The Moran and the Wright-Fisher process have the following common findings. Compared with well-mixed populations, migration causes σ₁ to change with the mutation probability from a decreasing curve to an inverted U-shaped curve and maintains the increase of σ₂. Migration (probability and range) leads to a significant change of σ₁ but a negligible one of σ₂. The way that migration changes σ₁ is qualitatively similar to its influence on the single parameter characterizing the two-strategy selection. The Moran process is more effective in increasing σ₁ for most migration probabilities and the Wright-Fisher process is always more effective in increasing σ₂. Finally, our findings are used to study the evolution of cooperation under direct reciprocity.

Asymmetric Power Boosts Extortion in an Economic Experiment

Direct reciprocity is a major mechanism for the evolution of cooperation. Several classical studies have suggested that humans should quickly learn to adopt reciprocal strategies to establish mutual cooperation in repeated interactions. On the other hand, the recently discovered theory of ZD strategies has found that subjects who use extortionate strategies are able to exploit and subdue cooperators. Although such extortioners have been predicted to succeed in any population of adaptive opponents, theoretical follow-up studies questioned whether extortion can evolve in reality. However, most of these studies presumed that individuals have similar strategic possibilities and comparable outside options, whereas asymmetries are ubiquitous in real world applications. Here we show with a model and an economic experiment that extortionate strategies readily emerge once subjects differ in their strategic power. Our experiment combines a repeated social dilemma with asymmetric partner choice. In our main treatment there is one randomly chosen group member who is unilaterally allowed to exchange one of the other group members after every ten rounds of the social dilemma. We find that this asymmetric replacement opportunity generally promotes cooperation, but often the resulting payoff distribution reflects the underlying power structure. Almost half of the subjects in a better strategic position turn into extortioners, who quickly proceed to exploit their peers. By adapting their cooperation probabilities consistent with ZD theory, extortioners force their co-players to cooperate without being similarly cooperative themselves. Comparison to non-extortionate players under the same conditions indicates a substantial net gain to extortion. Our results thus highlight how power asymmetries can endanger mutually beneficial interactions, and transform them into exploitative relationships. In particular, our results indicate that the extortionate strategies predicted from ZD theory could play a more prominent role in our daily interactions than previously thought.

Stochastic win-stay-lose-shift strategy with dynamic aspirations in evolutionary social dilemmas

In times of plenty expectations rise, just as in times of crisis they fall. This can be mathematically described as a win-stay-lose-shift strategy with dynamic aspiration levels, where individuals aspire to be as wealthy as their average neighbor. Here we investigate this model in the realm of evolutionary social dilemmas on the square lattice and scale-free networks. By using the master equation and Monte Carlo simulations, we find that cooperators coexist with defectors in the whole phase diagram, even at high temptations to defect. We study the microscopic mechanism that is responsible for the striking persistence of cooperative behavior and find that cooperation spreads through second-order neighbors, rather than by means of network reciprocity that dominates in imitation-based models. For the square lattice the master equation can be solved analytically in the large temperature limit of the Fermi function, while for other cases the resulting differential equations must be solved numerically. Either way, we find good qualitative agreement with the Monte Carlo simulation results. Our analysis also reveals that the evolutionary outcomes are to a large degree independent of the network topology, including the number of neighbors that are considered for payoff determination on lattices, which further corroborates the local character of the microscopic dynamics. Unlike large-scale spatial patterns that typically emerge due to network reciprocity, here local checkerboard-like patterns remain virtually unaffected by differences in the macroscopic properties of the interaction network.

Combination with anti-tit-for-tat remedies problems of tit-for-tat

One of the most important questions in game theory concerns how mutual cooperation can be achieved and maintained in a social dilemma. In Axelrod's tournaments of the iterated prisoner's dilemma, Tit-for-Tat (TFT) demonstrated the role of reciprocity in the emergence of cooperation. However, the stability of TFT does not hold in the presence of implementation error, and a TFT population is prone to neutral drift to unconditional cooperation, which eventually invites defectors. We argue that a combination of TFT and anti-TFT (ATFT) overcomes these difficulties in a noisy environment, provided that ATFT is defined as choosing the opposite to the opponent's last move. According to this TFT-ATFT strategy, a player normally uses TFT; turns to ATFT upon recognizing his or her own error; returns to TFT either when mutual cooperation is recovered or when the opponent unilaterally defects twice in a row. The proposed strategy provides simple and deterministic behavioral rules for correcting implementation error in a way that cannot be exploited by the opponent, and suppresses the neutral drift to unconditional cooperation.

Comparing reactive and memory-one strategies of direct reciprocity

Direct reciprocity is a mechanism for the evolution of cooperation based on repeated interactions. When individuals meet repeatedly, they can use conditional strategies to enforce cooperative outcomes that would not be feasible in one-shot social dilemmas. Direct reciprocity requires that individuals keep track of their past interactions and find the right response. However, there are natural bounds on strategic complexity: Humans find it difficult to remember past interactions accurately, especially over long timespans. Given these limitations, it is natural to ask how complex strategies need to be for cooperation to evolve. Here, we study stochastic evolutionary game dynamics in finite populations to systematically compare the evolutionary performance of reactive strategies, which only respond to the co-player's previous move, and memory-one strategies, which take into account the own and the co-player's previous move. In both cases, we compare deterministic strategy and stochastic strategy spaces. For reactive strategies and small costs, we find that stochasticity benefits cooperation, because it allows for generous-tit-for-tat. For memory one strategies and small costs, we find that stochasticity does not increase the propensity for cooperation, because the deterministic rule of win-stay, lose-shift works best. For memory one strategies and large costs, however, stochasticity can augment cooperation.

Extortion can outperform generosity in the iterated prisoner's dilemma

Zero-determinant (ZD) strategies, as discovered by Press and Dyson, can enforce a linear relationship between a pair of players' scores in the iterated prisoner's dilemma. Particularly, the extortionate ZD strategies can enforce and exploit cooperation, providing a player with a score advantage, and consequently higher scores than those from either mutual cooperation or generous ZD strategies. In laboratory experiments in which human subjects were paired with computer co-players, we demonstrate that both the generous and the extortionate ZD strategies indeed enforce a unilateral control of the reward. When the experimental setting is sufficiently long and the computerized nature of the opponent is known to human subjects, the extortionate strategy outperforms the generous strategy. Human subjects' cooperation rates when playing against extortionate and generous ZD strategies are similar after learning has occurred. More than half of extortionate strategists finally obtain an average score higher than that from mutual cooperation.

The zero-determinant (ZD) strategies discovered by Press and Dyson overturned several decades of consensus about the iterated prisoner's dilemma. Here, the authors provide the first empirical evidence in support of Press and Dyson's theory, by showing that knowledge of the opponent and the length of the interaction can facilitate the Generous and Extortionate ZD strategies as predicted.

Evolutionary mixed games in structured populations: Cooperation and the benefits of heterogeneity

Evolutionary games on networks traditionally involve the same game at each interaction. Here we depart from this assumption by considering mixed games, where the game played at each interaction is drawn uniformly at random from a set of two different games. While in well-mixed populations the random mixture of the two games is always equivalent to the average single game, in structured populations this is not always the case. We show that the outcome is, in fact, strongly dependent on the distance of separation of the two games in the parameter space. Effectively, this distance introduces payoff heterogeneity, and the average game is returned only if the heterogeneity is small. For higher levels of heterogeneity the distance to the average game grows, which often involves the promotion of cooperation. The presented results support preceding research that highlights the favorable role of heterogeneity regardless of its origin, and they also emphasize the importance of the population structure in amplifying facilitators of cooperation.

Autocratic strategies for iterated games with arbitrary action spaces

The recent discovery of zero-determinant strategies for the iterated prisoner's dilemma sparked a surge of interest in the surprising fact that a player can exert unilateral control over iterated interactions. These remarkable strategies, however, are known to exist only in games in which players choose between two alternative actions such as "cooperate" and "defect." Here we introduce a broader class of autocratic strategies by extending zero-determinant strategies to iterated games with more general action spaces. We use the continuous donation game as an example, which represents an instance of the prisoner's dilemma that intuitively extends to a continuous range of cooperation levels. Surprisingly, despite the fact that the opponent has infinitely many donation levels from which to choose, a player can devise an autocratic strategy to enforce a linear relationship between his or her payoff and that of the opponent even when restricting his or her actions to merely two discrete levels of cooperation. In particular, a player can use such a strategy to extort an unfair share of the payoffs from the opponent. Therefore, although the action space of the continuous donation game dwarfs that of the classic prisoner's dilemma, players can still devise relatively simple autocratic and, in particular, extortionate strategies.

The Emergence of Relationship-based Cooperation

This paper investigates the emergence of relationship-based cooperation by coupling two simple mechanisms into the model: tie strength based investment preference and homophily assumption. We construct the model by categorizing game participants into four types: prosocialists (players who prefers to invest in their intimate friends), antisocialists (players who prefer to invest in strangers), egoists (players who never cooperate) and altruists (players who cooperate indifferently with anyone). We show that the relationship-based cooperation (prosocialists) is favored throughout the evolution if we assume players of the same type have stronger ties than different ones. Moreover, we discover that strengthening the internal bonds within the strategic clusters further promotes the competitiveness of prosocialists and therefore facilitates the emergence of relationship-based cooperation in our proposed scenarios. The robustness of the model is also tested under different strategy updating rules and network structures. The results show that this argument is robust against the variations of initial conditions and therefore can be considered as a fundamental theoretical framework to study relationship-based cooperation in reality.

Zero-Determinant Strategies in Iterated Public Goods Game

Recently, Press and Dyson have proposed a new class of probabilistic and conditional strategies for the two-player iterated Prisoner’s Dilemma, so-called zero-determinant strategies. A player adopting zero-determinant strategies is able to pin the expected payoff of the opponents or to enforce a linear relationship between his own payoff and the opponents’ payoff, in a unilateral way. This paper considers zero-determinant strategies in the iterated public goods game, a representative multi-player game where in each round each player will choose whether or not to put his tokens into a public pot, and the tokens in this pot are multiplied by a factor larger than one and then evenly divided among all players. The analytical and numerical results exhibit a similar yet different scenario to the case of two-player games: (i) with small number of players or a small multiplication factor, a player is able to unilaterally pin the expected total payoff of all other players; (ii) a player is able to set the ratio between his payoff and the total payoff of all other players, but this ratio is limited by an upper bound if the multiplication factor exceeds a threshold that depends on the number of players.

Partners or rivals? Strategies for the iterated prisoner's dilemma

Within the class of memory-one strategies for the iterated Prisoner's Dilemma, we characterize partner strategies, competitive strategies and zero-determinant strategies. If a player uses a partner strategy, both players can fairly share the social optimum; but a co-player preferring an unfair solution will be penalized by obtaining a reduced payoff. A player using a competitive strategy never obtains less than the co-player. A player using a zero-determinant strategy unilaterally enforces a linear relation between the two players' payoffs. These properties hold for every strategy used by the co-player, whether memory-one or not.

Evolutionary performance of zero-determinant strategies in multiplayer games

Repetition is one of the key mechanisms to maintain cooperation. In long-term relationships, in which individuals can react to their peers׳ past actions, evolution can promote cooperative strategies that would not be stable in one-shot encounters. The iterated prisoner׳s dilemma illustrates the power of repetition. Many of the key strategies for this game, such as ALLD, ALLC, Tit-for-Tat, or generous Tit-for-Tat, share a common property: players using these strategies enforce a linear relationship between their own payoff and their co-player׳s payoff. Such strategies have been termed zero-determinant (ZD). Recently, it was shown that ZD strategies also exist for multiplayer social dilemmas, and here we explore their evolutionary performance. For small group sizes, ZD strategies play a similar role as for the repeated prisoner׳s dilemma: extortionate ZD strategies are critical for the emergence of cooperation, whereas generous ZD strategies are important to maintain cooperation. In large groups, however, generous strategies tend to become unstable and selfish behaviors gain the upper hand. Our results suggest that repeated interactions alone are not sufficient to maintain large-scale cooperation. Instead, large groups require further mechanisms to sustain cooperation, such as the formation of alliances or institutions, or additional pairwise interactions between group members.

Extortion under uncertainty: Zero-determinant strategies in noisy games

Repeated game theory has been one of the most prevailing tools for understanding long-running relationships, which are the foundation in building human society. Recent works have revealed a new set of "zero-determinant" (ZD) strategies, which is an important advance in repeated games. A ZD strategy player can exert unilateral control on two players' payoffs. In particular, he can deterministically set the opponent's payoff or enforce an unfair linear relationship between the players' payoffs, thereby always seizing an advantageous share of payoffs. One of the limitations of the original ZD strategy, however, is that it does not capture the notion of robustness when the game is subjected to stochastic errors. In this paper, we propose a general model of ZD strategies for noisy repeated games and find that ZD strategies have high robustness against errors. We further derive the pinning strategy under noise, by which the ZD strategy player coercively sets the opponent's expected payoff to his desired level, although his payoff control ability declines with the increase of noise strength. Due to the uncertainty caused by noise, the ZD strategy player cannot ensure his payoff to be permanently higher than the opponent's, which implies dominant extortions do not exist even under low noise. While we show that the ZD strategy player can still establish a novel kind of extortions, named contingent extortions, where any increase of his own payoff always exceeds that of the opponent's by a fixed percentage, and the conditions under which the contingent extortions can be realized are more stringent as the noise becomes stronger.

Social exclusion in finite populations

Social exclusion, keeping free riders from benefit sharing, plays an important role in sustaining cooperation in our world. Here we propose two different exclusion regimes, namely, peer exclusion and pool exclusion, to investigate the evolution of social exclusion in finite populations. In the peer exclusion regime, each excluder expels all the defectors independently, and thus bears the total cost on his own, while in the pool exclusion regime, excluders spontaneously form an institution to carry out rejection of the free riders, and each excluder shares the cost equally. In a public goods game containing only excluders and defectors, it is found that peer excluders outperform pool excluders if the exclusion costs are small, and the situation is converse once the exclusion costs exceed some critical points, which holds true for all the selection intensities and different update rules. Moreover, excluders can dominate the whole population under a suitable parameters range in the presence of second-order free riders (cooperators), showing that exclusion has prominent advantages over common costly punishment. More importantly, our finding indicates that the group exclusion mechanism helps the cooperative union to survive under unfavorable conditions. Our results may give some insights into better understanding the prevalence of such a strategy in the real world and its significance in sustaining cooperation.

Conformity enhances network reciprocity in evolutionary social dilemmas

The pursuit of highest payoffs in evolutionary social dilemmas is risky and sometimes inferior to conformity. Choosing the most common strategy within the interaction range is safer because it ensures that the payoff of an individual will not be much lower than average. Herding instincts and crowd behaviour in humans and social animals also compel to conformity in their own right. Motivated by these facts, we here study the impact of conformity on the evolution of cooperation in social dilemmas. We show that an appropriate fraction of conformists within the population introduces an effective surface tension around cooperative clusters and ensures smooth interfaces between different strategy domains. Payoff-driven players brake the symmetry in favour of cooperation and enable an expansion of clusters past the boundaries imposed by traditional network reciprocity. This mechanism works even under the most testing conditions, and it is robust against variations of the interaction network as long as degree-normalized payoffs are applied. Conformity may thus be beneficial for the resolution of social dilemmas.

Relational diversity promotes cooperation in prisoner's dilemma games

Relational diversity can be characterized by heterogeneous distributions of tie strengths in social networks and this diversity is present not only among humans, but throughout the animal world. We account for this observation by analyzing two network datasets from Facebook. We measure the strength of a tie by calculating the extent of overlap of friends between the two individuals. Based on the previous findings in human experiments, we argue that it is very unlikely that players will allocate their investments equally to their neighbors. There is a tendency that players prefer to donate more to their intimate friends. We find that if players preferentially allocate their investments to their good friends, cooperation will be promoted in PDG. We proved that the facilitation of the cooperative strategy relies mostly on the cooperative allies between best friends, resulting in the formation of cooperative clusters which are able to prevail against the defectors even when there is a large cost to cooperate. Moreover, we discover that the effect of relational diversity cannot be analyzed by adopting classical complex networks models, because neither of the artificial networks is able to produce networks with diverse distributions of tie strengths. It is of vital importance to introduce real social networks to study the influence of diverse relations especially when it comes to humans. This research proposes a brand new perspective to understand the influence of social relations on the emergence of cooperation in evolutionary prisoner's dilemma games.

Boosting cooperation by involving extortion in spatial prisoner's dilemma games

We study the evolution of cooperation in spatial prisoner's dilemma games with and without extortion by adopting the aspiration-driven strategy updating rule. We focus explicitly on how the strategy updating manner (whether synchronous or asynchronous) and also the introduction of extortion strategy affect the collective outcome of the games. By means of Monte Carlo simulations as well as dynamical cluster techniques, we find that the involvement of extortioners facilitates the boom of cooperators in the population (and whom can always dominate the population if the temptation to defect is not too large) for both synchronous and asynchronous strategy updating, in stark contrast to the other case, where cooperation is promoted for an intermediate aspiration level with synchronous strategy updating, but is remarkably inhibited if the strategy updating is implemented asynchronously. We explain the results by configurational analysis and find that the presence of extortion leads to the checkerboard-like ordering of cooperators and extortioners, which enable cooperators to prevail in the population with both strategy updating manners. Moreover, extortion itself is evolutionary stable, and therefore acts as the incubator for the evolution of cooperation.