Stochastic game theory: for playing games, not just for doing theory

Co-evolution of heterogeneous cognition in spatial snowdrift game with asymmetric cost

The emergence of the evolutionary game on complex networks provides a fresh framework for studying cooperation behavior between complex populations. Numerous recent progress has been achieved in studying asymmetric games. However, there is still a substantial need to address how to flexibly express the individual asymmetric nature. In this paper, we employ mutual cognition among individuals to elucidate the asymmetry inherent in their interactions. Cognition arises from individuals' subjective assessments and significantly influences their decision-making processes. In social networks, mutual cognition among individuals is a persistent phenomenon and frequently displays heterogeneity as the influence of their interactions. This unequal cognitive dynamic will, in turn, influence the interactions, culminating in asymmetric outcomes. To better illustrate the inter-individual cognition in asymmetric snowdrift games, the concept of favor value is introduced here. On this basis, the evolution of cognition and its relationship with asymmetry degree are defined. In our simulation, we investigate how game cost and the intensity of individual cognitive changes impact the cooperation frequency. Furthermore, the temporal evolution of individual cognition and its variation under different parameters was also examined. The simulation results reveal that the emergence of heterogeneous cognition effectively addresses social dilemmas, with asymmetric interactions among individuals enhancing the propensity for cooperative choices. It is noteworthy that distinctions exist in the rules governing cooperation and cognitive evolution between regular networks and Watts-Strogatz small-world networks. In light of this, we deduce the relationship between cognition evolution and cooperative behavior in co-evolution and explore potential factors influencing cooperation within the system.

Diversity enables the jump towards cooperation for the Traveler's Dilemma

Social dilemmas are situations in which collective welfare is at odds with individual gain. One widely studied example, due to the conflict it poses between human behaviour and game theoretic reasoning, is the Traveler's Dilemma. The dilemma relies on the players' incentive to undercut their opponent at the expense of losing a collective high payoff. Such individual incentive leads players to a systematic mutual undercutting until the lowest possible payoff is reached, which is the game's unique Nash equilibrium. However, if players were satisfied with a high payoff -that is not necessarily higher than their opponent's- they would both be better off individually and collectively. Here, we explain how it is possible to converge to this cooperative high payoff equilibrium. Our analysis focuses on decomposing the dilemma into a local and a global game. We show that players need to escape the local maximisation and jump to the global game, in order to reach the cooperative equilibrium. Using a dynamic approach, based on evolutionary game theory and learning theory models, we find that diversity, understood as the presence of suboptimal strategies, is the general mechanism that enables the jump towards cooperation.

Two-stage stochastic formulation for relief operations with multiple agencies in simultaneous disasters

The increasing damage caused by disasters is a major challenge for disaster management authorities, especially in instances where simultaneous disasters affect different geographical areas. The uncertainty and chaotic conditions caused by these situations combined with the inherent complexity of collaboration between multiple stakeholders complicates delivering support for disaster victims. Decisions related to facility location, procurement, stock prepositioning and relief distribution are essential to ensure the provision of relief for these victims. There is a need to provide analytical models that can support integrated decision-making in settings with uncertainty caused by simultaneous disasters. However, there are no formulations tackling these decisions combining multiple suppliers, multiple agencies, and simultaneous disasters. This article introduces a novel bi-objective two-stage stochastic formulation for disaster preparedness and immediate response considering the interaction of multiple stakeholders in uncertain environments caused by the occurrence of simultaneous disasters. At the first stage, decisions related to the selection of suppliers, critical facilities, agencies involved, and pre-disaster procurement are defined. Resource allocation, relief distribution and procurement of extra resources after the events are decided at the second stage. The model was tested on data from the situation caused by simultaneous hurricanes and storms in Mexico during September of 2013. The case is contrasted with instances planning for disasters independently. The results show how planning for multiple disasters can help understand the real boundaries of the disaster response system, the benefits of integrated decision-making, the impact of deploying only the agencies required, and the criticality of considering human resources in disaster planning.

Persistent homology and the shape of evolutionary games

For nearly three decades, spatial games have produced a wealth of insights to the study of behavior and its relation to population structure. However, as different rules and factors are added or altered, the dynamics of spatial models often become increasingly complicated to interpret. To tackle this problem, we introduce persistent homology as a rigorous framework that can be used to both define and compute higher-order features of data in a manner which is invariant to parameter choices, robust to noise, and independent of human observation. Our work demonstrates its relevance for spatial games by showing how topological features of simulation data that persist over different spatial scales reflect the stability of strategies in 2D lattice games. To do so, we analyze the persistent homology of scenarios from two games: a Prisoner's Dilemma and a SIRS epidemic model. The experimental results show how the method accurately detects features that correspond to real aspects of the game dynamics. Unlike other tools that study dynamics of spatial systems, persistent homology can tell us something meaningful about population structure while remaining neutral about the underlying structure itself. Regardless of game complexity, since strategies either succeed or fail to conform to shapes of a certain topology there is much potential for the method to provide novel insights for a wide variety of spatially extended systems in biology, social science, and physics.

Bounded rational response equilibria in human sensorimotor interactions

The Nash equilibrium is one of the most central solution concepts to study strategic interactions between multiple players and has recently also been shown to capture sensorimotor interactions between players that are haptically coupled. While previous studies in behavioural economics have shown that systematic deviations from Nash equilibria in economic decision-making can be explained by the more general quantal response equilibria, such deviations have not been reported for the sensorimotor domain. Here we investigate haptically coupled dyads across three different sensorimotor games corresponding to the classic symmetric and asymmetric Prisoner's Dilemma, where the quantal response equilibrium predicts characteristic shifts across the three games, although the Nash equilibrium stays the same. We find that subjects exhibit the predicted deviations from the Nash solution. Furthermore, we show that taking into account subjects' priors for the games, we arrive at a more accurate description of bounded rational response equilibria that can be regarded as a quantal response equilibrium with non-uniform prior. Our results suggest that bounded rational response equilibria provide a general tool to explain sensorimotor interactions that include the Nash equilibrium as a special case in the absence of information processing limitations.

Smarter than others? Conjectures in lowest unique bid auctions

Research concerning various types of auctions, such as English auctions, Dutch auctions, highest-price sealed-bid auctions, and second-price sealed-bid auctions, is always a topic of considerable interest in interdisciplinary fields. The type of auction, known as a lowest unique bid auction (LUBA), has also attracted significant attention. Various models have been proposed, but they often fail to explain satisfactorily the real bid-distribution characteristics. This paper discusses LUBA bid-distribution characteristics, including the inverted-J shape and the exponential decrease in the upper region. The authors note that this type of distribution, which initially increases and later decreases, cannot be derived from the symmetric Nash equilibrium framework based on perfect information that has previously been used. A novel optimization model based on non-perfect information is presented. The kernel of this model is the premise that agents make decisions to achieve maximum profit based on imaginary information or assumptions regarding the behavior of others.

Evolution of individual versus social learning on social networks

A number of studies have investigated the roles played by individual and social learning in cultural phenomena and the relative advantages of the two learning strategies in variable environments. Because social learning involves the acquisition of behaviours from others, its utility depends on the availability of 'cultural models' exhibiting adaptive behaviours. This indicates that social networks play an essential role in the evolution of learning. However, possible effects of social structure on the evolution of learning have not been fully explored. Here, we develop a mathematical model to explore the evolutionary dynamics of learning strategies on social networks. We first derive the condition under which social learners (SLs) are selectively favoured over individual learners in a broad range of social network. We then obtain an analytical approximation of the long-term average frequency of SLs in homogeneous networks, from which we specify the condition, in terms of three relatedness measures, for social structure to facilitate the long-term evolution of social learning. Finally, we evaluate our approximation by Monte Carlo simulations in complete graphs, regular random graphs and scale-free networks. We formally show that whether social structure favours the evolution of social learning is determined by the relative magnitudes of two effects of social structure: localization in competition, by which competition between learning strategies is evaded, and localization in cultural transmission, which slows down the spread of adaptive traits. In addition, our estimates of the relatedness measures suggest that social structure disfavours the evolution of social learning when selection is weak.

An experimental investigation of evolutionary dynamics in the Rock-Paper-Scissors game

Game theory describes social behaviors in humans and other biological organisms. By far, the most powerful tool available to game theorists is the concept of a Nash Equilibrium (NE), which is motivated by perfect rationality. NE specifies a strategy for everyone, such that no one would benefit by deviating unilaterally from his/her strategy. Another powerful tool available to game theorists are evolutionary dynamics (ED). Motivated by evolutionary and learning processes, ED specify changes in strategies over time in a population, such that more successful strategies typically become more frequent. A simple game that illustrates interesting ED is the generalized Rock-Paper-Scissors (RPS) game. The RPS game extends the children's game to situations where winning or losing can matter more or less relative to tying. Here we investigate experimentally three RPS games, where the NE is always to randomize with equal probability, but the evolutionary stability of this strategy changes. Consistent with the prediction of ED we find that aggregate behavior is far away from NE when it is evolutionarily unstable. Our findings add to the growing literature that demonstrates the predictive validity of ED in large-scale incentivized laboratory experiments with human subjects.

Evolutionary dynamics of the traveler's dilemma and minimum-effort coordination games on complex networks

The traveler's dilemma game and the minimum-effort coordination game are social dilemmas that have received significant attention resulting from the fact that the predictions of classical game theory are inconsistent with the results found when the games are studied experimentally. Moreover, both the traveler's dilemma and the minimum-effort coordination games have potentially important applications in evolutionary biology. Interestingly, standard deterministic evolutionary game theory, as represented by the replicator dynamics in a well-mixed population, is also inadequate to account for the behavior observed in these games. Here we study the evolutionary dynamics of both these games in populations with interaction patterns described by a variety of complex network topologies. We investigate the evolutionary dynamics of these games through agent-based simulations on both model and empirical networks. In particular, we study the effects of network clustering and assortativity on the evolutionary dynamics of both games. In general, we show that the evolutionary behavior of the traveler's dilemma and minimum-effort coordination games on complex networks is in good agreement with that observed experimentally. Thus, formulating the traveler's dilemma and the minimum-effort coordination games on complex networks neatly resolves the paradoxical aspects of these games.

An application of evolutionary game theory to social dilemmas: the traveler's dilemma and the minimum effort coordination game

The Traveler's Dilemma game and the Minimum Effort Coordination game are two social dilemmas that have attracted considerable attention due to the fact that the predictions of classical game theory are at odds with the results found when the games are studied experimentally. Moreover, a direct application of deterministic evolutionary game theory, as embodied in the replicator dynamics, to these games does not explain the observed behavior. In this work, we formulate natural variants of these two games as smoothed continuous-strategy games. We study the evolutionary dynamics of these continuous-strategy games, both analytically and through agent-based simulations, and show that the behavior predicted theoretically is in accord with that observed experimentally. Thus, these variants of the Traveler's Dilemma and the Minimum Effort Coordination games provide a simple resolution of the paradoxical behavior associated with the original games.

Replicator dynamics with turnover of players

We study adaptive dynamics in games where players abandon the population at a given rate and are replaced by naive players characterized by a prior distribution over the admitted strategies. We demonstrate how such a process leads macroscopically to a variant of the replicator equation, with an additional term accounting for player turnover. We study how Nash equilibria and the dynamics of the system are modified by this additional term for prototypical examples such as the rock-paper-scissors game and different classes of two-action games played between two distinct populations. We conclude by showing how player turnover can account for nontrivial departures from Nash equilibria observed in data from lowest unique bid auctions.

Evolution of cooperation in spatial traveler's dilemma game

Traveler's dilemma (TD) is one of social dilemmas which has been well studied in the economics community, but it is attracted little attention in the physics community. The TD game is a two-person game. Each player can select an integer value between and () as a pure strategy. If both of them select the same value, the payoff to them will be that value. If the players select different values, say and (), then the payoff to the player who chooses the small value will be and the payoff to the other player will be . We term the player who selects a large value as the cooperator, and the one who chooses a small value as the defector. The reason is that if both of them select large values, it will result in a large total payoff. The Nash equilibrium of the TD game is to choose the smallest value . However, in previous behavioral studies, players in TD game typically select values that are much larger than , and the average selected value exhibits an inverse relationship with . To explain such anomalous behavior, in this paper, we study the evolution of cooperation in spatial traveler's dilemma game where the players are located on a square lattice and each player plays TD games with his neighbors. Players in our model can adopt their neighbors' strategies following two standard models of spatial game dynamics. Monte-Carlo simulation is applied to our model, and the results show that the cooperation level of the system, which is proportional to the average value of the strategies, decreases with increasing until is greater than the critical value where cooperation vanishes. Our findings indicate that spatial reciprocity promotes the evolution of cooperation in TD game and the spatial TD game model can interpret the anomalous behavior observed in previous behavioral experiments.

Stochastic evolutionary dynamics resolve the Traveler's Dilemma

Behavior in social dilemmas is often inconsistent with the predictions of classical game theory: people (and a wide variety of other organisms) are more cooperative than might be expected. Here we consider behavior in one such social dilemma, the Traveler's Dilemma, that has received considerable attention in the economics literature but is little known among theoretical biologists. The rules of the game are as follows. Two players each choose a value between R and M, where 0<R<M. If the players choose the same value, both receive that amount. If the players choose different values v(1) and v(2), where v(1)<v(2), then the player choosing v(1) receives v(1)+R and the player choosing v(2) receives v(1)-R. While the players would maximize their payoffs by both choosing the largest allowed value, M, the Nash equilibrium is to choose the smallest allowed value, R. In behavioral experiments, however, people generally choose values much larger than the minimum and the deviation from the expected equilibrium decreases with R. In this paper, we show that the cooperative behavior observed in the Traveler's Dilemma can be explained in an evolutionary framework. We study stochastic evolutionary dynamics in finite populations with varying intensity of selection and varying mutation rate. We derive analytic results showing that strategies choosing high values can be favored when selection is weak. More generally, selection favors strategies that choose high values if R is small (relative to M) and strategies that choose low values if R is large. Finally, we show that a two-parameter model involving the intensity of selection and the mutation rate can quantitatively reproduce data that from a Traveler's Dilemma experiment. These results demonstrate the power of evolutionary game theory for explaining human behavior in contexts that are challenging for standard economic game theory.

Equilibrium strategy and population-size effects in lowest unique bid auctions

In lowest unique bid auctions, N players bid for an item. The winner is whoever places the lowest bid, provided that it is also unique. We use a grand canonical approach to derive an analytical expression for the equilibrium distribution of strategies. We then study the properties of the solution as a function of the mean number of players, and compare them with a large data set of internet auctions. The theory agrees with the data with striking accuracy for small population-size N, while for larger N a qualitatively different distribution is observed. We interpret this result as the emergence of two different regimes, one in which adaptation is feasible and one in which it is not. Our results question the actual possibility of a large population to adapt and find the optimal strategy when participating in a collective game.

Evolutionary dynamics in finite populations can explain the full range of cooperative behaviors observed in the centipede game

Classical economic models make behavioral predictions based on the assumption that people are fully rational and care only about maximizing their own payoffs. Although this approach successfully explains human behavior in many situations, there is a wealth of experimental evidence demonstrating conditions where people deviate from the predictions of these models. One setting that has received particular attention is fixed length repeated games. Iterating a social dilemma can promote cooperation through direct reciprocity, even if it is common knowledge that all players are rational and self-interested. However, this is not the case if the length of the game is known to the players. In the final round, a rational player will defect, because there is no future to be concerned with. But if you know the other player will defect in the last round, then you should defect in the second to last round, and so on. This logic of backwards induction leads to immediate defection as the only rational (sub-game perfect Nash equilibrium) strategy. When people actually play such games, however, immediate defection is rare. Here we use evolutionary dynamics in finite populations to study the centipede game, which is designed to explore this issue of backwards induction. We make the following observation: since full cooperation can risk-dominate immediate defection in the centipede game, stochastic evolutionary dynamics can favor both delayed defection and even full cooperation. Furthermore, our evolutionary model can quantitatively reproduce human behavior from two experiments by fitting a single free parameter, which is the product of population size and selection intensity. Thus we provide evidence that people's cooperative behavior in fixed length games, which is often called 'irrational', may in fact be the favored outcome of natural selection.

Behavioral and neural predictors of upcoming decisions

Although it is widely known that brain regions such as the prefrontal cortex, the amygdala, and the ventral striatum play large roles in decision making, their precise contributions remain unclear. Here, we used functional magnetic resonance imaging and principles of reinforcement learning theory to investigate the relationship between current reinforcements and future decisions. In the experiment, subjects chose between high-risk (i.e., low probability of a large monetary reward) and low-risk (high probability of a small reward) decisions. For each subject, we estimated value functions that represented the degree to which reinforcements affected the value of decision options on the subsequent trial. Individual differences in value functions predicted not only trial-to-trial behavioral strategies, such as choosing high-risk decisions following high-risk rewards, but also the relationship between activity in prefrontal and subcortical regions during one trial and the decision made in the subsequent trial. These findings provide a novel link between behavior and neural activity by demonstrating that value functions are manifested both in adjustments in behavioral strategies and in the neural activity that accompanies those adjustments.

The Nash equilibrium: a perspective

In 1950, John Nash contributed a remarkable one-page PNAS article that defined and characterized a notion of equilibrium for n- person games. This notion, now called the "Nash equilibrium," has been widely applied and adapted in economics and other behavioral sciences. Indeed, game theory, with the Nash equilibrium as its centerpiece, is becoming the most prominent unifying theory of social science. In this perspective, we summarize the historical context and subsequent impact of Nash's contribution.