Calculating evolutionary dynamics in structured populations

Universal scaling of extinction time in stochastic evolutionary dynamics

Evolutionary dynamics is well captured by the replicator equations when the population is infinite and well-mixed. However, the extinction dynamics is modified with finite and structured populations. Experiments on the non-transitive ecosystem containing three populations of bacteria found that the ecological stability sensitively depends on the spatial structure of the populations. Based on the Reference-Gamble-Birth algorithm, we use agent-based Monte Carlo simulations to investigate the extinction dynamics in the rock-paper-scissors ecosystem with finite and structured populations. On the fully-connected network, the extinction time in stable and unstable regimes falls into two universal functions when plotted with the rescaled variables. On the two dimensional grid, the spatial structure changes the transition boundary between stable and unstable regimes but doesn't change its extinction trend. The finding of universal scaling in extinction dynamics is unexpected, and may provide a powerful method to classify different evolutionary dynamics into universal classes.

The role of evolutionary game theory in spatial and non-spatial models of the survival of cooperation in cancer: a review

Evolutionary game theory (EGT) is a branch of mathematics which considers populations of individuals interacting with each other to receive pay-offs. An individual’s pay-off is dependent on the strategy of its opponent(s) as well as on its own, and the higher its pay-off, the higher its reproductive fitness. Its offspring generally inherit its interaction strategy, subject to random mutation. Over time, the composition of the population shifts as different strategies spread or are driven extinct. In the last 25 years there has been a flood of interest in applying EGT to cancer modelling, with the aim of explaining how cancerous mutations spread through healthy tissue and how intercellular cooperation persists in tumour-cell populations. This review traces this body of work from theoretical analyses of well-mixed infinite populations through to more realistic spatial models of the development of cooperation between epithelial cells. We also consider work in which EGT has been used to make experimental predictions about the evolution of cancer, and discuss work that remains to be done before EGT can make large-scale contributions to clinical treatment and patient outcomes.

Cooperative success in epithelial public goods games

Cancer cells obtain mutations which rely on the production of diffusible growth factors to confer a fitness benefit. These mutations can be considered cooperative, and studied as public goods games within the framework of evolutionary game theory. The population structure, benefit function and update rule all influence the evolutionary success of cooperators. We model the evolution of cooperation in epithelial cells using the Voronoi tessellation model. Unlike traditional evolutionary graph theory, this allows us to implement global updating, for which birth and death events are spatially decoupled. We compare, for a sigmoid benefit function, the conditions for cooperation to be favoured and/or beneficial for well-mixed and structured populations. We find that when population structure is combined with global updating, cooperation is more successful than if there were local updating or the population were well-mixed. Interestingly, the qualitative behaviour for the well-mixed population and the Voronoi tessellation model is remarkably similar, but the latter case requires significantly lower incentives to ensure cooperation.

Fixation probabilities in evolutionary dynamics under weak selection

In evolutionary dynamics, a key measure of a mutant trait's success is the probability that it takes over the population given some initial mutant-appearance distribution. This "fixation probability" is difficult to compute in general, as it depends on the mutation's effect on the organism as well as the population's spatial structure, mating patterns, and other factors. In this study, we consider weak selection, which means that the mutation's effect on the organism is small. We obtain a weak-selection perturbation expansion of a mutant's fixation probability, from an arbitrary initial configuration of mutant and resident types. Our results apply to a broad class of stochastic evolutionary models, in which the size and spatial structure are arbitrary (but fixed). The problem of whether selection favors a given trait is thereby reduced from exponential to polynomial complexity in the population size, when selection is weak. We conclude by applying these methods to obtain new results for evolutionary dynamics on graphs.

Evolutionary multiplayer games on graphs with edge diversity

Evolutionary game dynamics in structured populations has been extensively explored in past decades. However, most previous studies assume that payoffs of individuals are fully determined by the strategic behaviors of interacting parties, and social ties between them only serve as the indicator of the existence of interactions. This assumption neglects important information carried by inter-personal social ties such as genetic similarity, geographic proximity, and social closeness, which may crucially affect the outcome of interactions. To model these situations, we present a framework of evolutionary multiplayer games on graphs with edge diversity, where different types of edges describe diverse social ties. Strategic behaviors together with social ties determine the resulting payoffs of interactants. Under weak selection, we provide a general formula to predict the success of one behavior over the other. We apply this formula to various examples which cannot be dealt with using previous models, including the division of labor and relationship- or edge-dependent games. We find that labor division can promote collective cooperation markedly. The evolutionary process based on relationship-dependent games can be approximated by interactions under a transformed and unified game. Our work stresses the importance of social ties and provides effective methods to reduce the calculating complexity in analyzing the evolution of realistic systems.

The outcome of an interaction often relies on not only interactants’ strategic behaviors but also genetic and physical relationships between interactants, such as genetic similarity and geographic proximity. Thus when encountering different opponents who use the same strategy, an individual may derive different payoffs. Social ties, acting as carriers of such information, are crucial to biological interactions. However, most prior studies simplify social ties as binary states (i.e., either present or absent) and ignore the information carried. Here we study evolutionary multiplayer games on graphs and introduce different types of edges to describe diverse social ties. We derive a simple rule to predict when a strategic behavior is more successful than the other. Based on this rule, we find that the labor division in eusocial insects could promote prosocial behavior. In addition, when payoff structures in different interactions are relationship-dependent, the condition for the success of one behavior can be obtained by studying interactions described by a unified payoff structure. Our work not only extends established results on the evolution of cooperation on graphs, but also shows the possibility to simplify complex and diverse interactions in real-world systems as simple and unified interactions in theoretical calculations.

Individualised aspiration dynamics: Calculation by proofs

Cooperation is key for the evolution of biological systems ranging from bacteria communities to human societies. Evolutionary processes can dramatically alter the cooperation level. Evolutionary processes are typically of two classes: comparison based and self-evaluation based. The fate of cooperation is extremely sensitive to the details of comparison based processes. For self-evaluation processes, however, it is still unclear whether the sensitivity remains. We concentrate on a class of self-evaluation processes based on aspiration, where all the individuals adjust behaviors based on their own aspirations. We prove that the evolutionary outcome with heterogeneous aspirations is the same as that of the homogeneous one for regular networks under weak selection limit. Simulation results further suggest that it is also valid for general networks across various distributions of personalised aspirations. Our result clearly indicates that self-evaluation processes are robust in contrast with comparison based rules. In addition, our result greatly simplifies the calculation of the aspiration dynamics, which is computationally expensive.

Cooperation is the cornerstone to understand how biological systems evolve. Previous studies have shown that cooperation is sensitive to the details of evolutionary processes, even if all the individuals update strategies in the same way. Here we propose a class of updating rules driven by self-evaluation, where each individual has its personal aspiration. The evolutionary outcome is the same as if all the individuals adopt the same aspiration for regular networks, provided the selection intensity is weak enough. In addition, we provide a simple numerical method to identify the favored strategy. Our result shows a very robust class of strategy updating rules. And it implies that complexity in updating rules does not necessarily lead to the sensitivity of evolutionary outcomes.

Defector clustering is linked to cooperation in a pathogenic bacterium

Spatial clustering is thought to favour the evolution of cooperation because it puts cooperators in a position to help each other. However, clustering also increases competition. The fate of cooperation may depend on how much cooperators cluster relative to defectors, but these clustering differences have not been the focus of previous models and experiments. By competing siderophore-producing cooperator and defector strains of the opportunistic pathogen Pseudomonas aeruginosa in experimental microhabitats, we found that at the spatial scale of individual interactions, cooperator clustering lowers cooperation, but defector clustering favours cooperation. A theoretical model and individual-based simulations show these counterintuitive effects can arise when competition and cooperation occur at a single resource-determined scale, with population dynamics crucially allowing cooperators and defectors to cluster differently. The results suggest that cooperation relies on the regulation of sufficient defector clustering relative to cooperator clustering, which may be important in bacteria, social amoeba and cancer inhibition.

Towards a mechanistic foundation of evolutionary theory

Most evolutionary thinking is based on the notion of fitness and related ideas such as fitness landscapes and evolutionary optima. Nevertheless, it is often unclear what fitness actually is, and its meaning often depends on the context. Here we argue that fitness should not be a basal ingredient in verbal or mathematical descriptions of evolution. Instead, we propose that evolutionary birth-death processes, in which individuals give birth and die at ever-changing rates, should be the basis of evolutionary theory, because such processes capture the fundamental events that generate evolutionary dynamics. In evolutionary birth-death processes, fitness is at best a derived quantity, and owing to the potential complexity of such processes, there is no guarantee that there is a simple scalar, such as fitness, that would describe long-term evolutionary outcomes. We discuss how evolutionary birth-death processes can provide useful perspectives on a number of central issues in evolution.

DOI:http://dx.doi.org/10.7554/eLife.23804.001

Impact of migration on the multi-strategy selection in finite group-structured populations

For large quantities of spatial models, the multi-strategy selection under weak selection is the sum of two competition terms: the pairwise competition and the competition of multiple strategies with equal frequency. Two parameters σ₁ and σ₂ quantify the dependence of the multi-strategy selection on these two terms, respectively. Unlike previous studies, we here do not require large populations for calculating σ₁ and σ₂, and perform the first quantitative analysis of the effect of migration on them in group-structured populations of any finite sizes. The Moran and the Wright-Fisher process have the following common findings. Compared with well-mixed populations, migration causes σ₁ to change with the mutation probability from a decreasing curve to an inverted U-shaped curve and maintains the increase of σ₂. Migration (probability and range) leads to a significant change of σ₁ but a negligible one of σ₂. The way that migration changes σ₁ is qualitatively similar to its influence on the single parameter characterizing the two-strategy selection. The Moran process is more effective in increasing σ₁ for most migration probabilities and the Wright-Fisher process is always more effective in increasing σ₂. Finally, our findings are used to study the evolution of cooperation under direct reciprocity.

Evolutionary Games of Multiplayer Cooperation on Graphs

There has been much interest in studying evolutionary games in structured populations, often modeled as graphs. However, most analytical results so far have only been obtained for two-player or linear games, while the study of more complex multiplayer games has been usually tackled by computer simulations. Here we investigate evolutionary multiplayer games on graphs updated with a Moran death-Birth process. For cycles, we obtain an exact analytical condition for cooperation to be favored by natural selection, given in terms of the payoffs of the game and a set of structure coefficients. For regular graphs of degree three and larger, we estimate this condition using a combination of pair approximation and diffusion approximation. For a large class of cooperation games, our approximations suggest that graph-structured populations are stronger promoters of cooperation than populations lacking spatial structure. Computer simulations validate our analytical approximations for random regular graphs and cycles, but show systematic differences for graphs with many loops such as lattices. In particular, our simulation results show that these kinds of graphs can even lead to more stringent conditions for the evolution of cooperation than well-mixed populations. Overall, we provide evidence suggesting that the complexity arising from many-player interactions and spatial structure can be captured by pair approximation in the case of random graphs, but that it need to be handled with care for graphs with high clustering.

Cooperation can be defined as the act of providing fitness benefits to other individuals, often at a personal cost. When interactions occur mainly with neighbors, assortment of strategies can favor cooperation but local competition can undermine it. Previous research has shown that a single coefficient can capture this trade-off when cooperative interactions take place between two players. More complicated, but also more realistic, models of cooperative interactions involving multiple players instead require several such coefficients, making it difficult to assess the effects of population structure. Here, we obtain analytical approximations for the coefficients of multiplayer games in graph-structured populations. Computer simulations show that, for particular instances of multiplayer games, these approximate coefficients predict the condition for cooperation to be promoted in random graphs well, but fail to do so in graphs with more structure, such as lattices. Our work extends and generalizes established results on the evolution of cooperation on graphs, but also highlights the importance of explicitly taking into account higher-order statistical associations in order to assess the evolutionary dynamics of cooperation in spatially structured populations.

Intermediate-Range Migration Furnishes a Narrow Margin of Efficiency in the Two-Strategy Competition

It is well-known that the effects of spatial selection on the two-strategy competition can be quantified by the structural coefficient σ under weak selection. We here calculate the accurate value of σ in group-structured populations of any finite size. In previous similar models, the large population size has been explicitly required for obtaining σ, and here we analyze quantitatively how large the population should be. Unlike previous models which have only involved the influences of the longest and the shortest migration rang on σ, we consider all migration ranges together. The new phenomena are that an intermediate range maximizes σ for medium migration probabilities which are of the tiny minority and the maximum value is slightly larger than those for other ranges. Furthermore, we find the ways that migration or mutation changes σ can vary significantly through determining analytically how the high-frequency steady states (distributions of either strategy over all groups) impact the expression of σ obtained before. Our findings can be directly used to resolve the dilemma of cooperation and provide a more intuitive understanding of spatial selection.

Games of multicellularity

Evolutionary game dynamics are often studied in the context of different population structures. Here we propose a new population structure that is inspired by simple multicellular life forms. In our model, cells reproduce but can stay together after reproduction. They reach complexes of a certain size, n, before producing single cells again. The cells within a complex derive payoff from an evolutionary game by interacting with each other. The reproductive rate of cells is proportional to their payoff. We consider all two-strategy games. We study deterministic evolutionary dynamics with mutations, and derive exact conditions for selection to favor one strategy over another. Our main result has the same symmetry as the well-known sigma condition, which has been proven for stochastic game dynamics and weak selection. For a maximum complex size of n=2 our result holds for any intensity of selection. For n≥3 it holds for weak selection. As specific examples we study the prisoner's dilemma and hawk-dove games. Our model advances theoretical work on multicellularity by allowing for frequency-dependent interactions within groups.

Cooperation in group-structured populations with two layers of interactions

Recently there has been a growing interest in studying multiplex networks where individuals are structured in multiple network layers. Previous agent-based simulations of games on multiplex networks reveal rich dynamics arising from interdependency of interactions along each network layer, yet there is little known about analytical conditions for cooperation to evolve thereof. Here we aim to tackle this issue by calculating the evolutionary dynamics of cooperation in group-structured populations with two layers of interactions. In our model, an individual is engaged in two layers of group interactions simultaneously and uses unrelated strategies across layers. Evolutionary competition of individuals is determined by the total payoffs accrued from two layers of interactions. We also consider migration which allows individuals to move to a new group within each layer. An approach combining the coalescence theory with the theory of random walks is established to overcome the analytical difficulty upon local migration. We obtain the exact results for all “isotropic” migration patterns, particularly for migration tuned with varying ranges. When the two layers use one game, the optimal migration ranges are proved identical across layers and become smaller as the migration probability grows.

Cellular cooperation with shift updating and repulsion

Population structure can facilitate evolution of cooperation. In a structured population, cooperators can form clusters which resist exploitation by defectors. Recently, it was observed that a shift update rule is an extremely strong amplifier of cooperation in a one dimensional spatial model. For the shift update rule, an individual is chosen for reproduction proportional to fecundity; the offspring is placed next to the parent; a random individual dies. Subsequently, the population is rearranged (shifted) until all individual cells are again evenly spaced out. For large population size and a one dimensional population structure, the shift update rule favors cooperation for any benefit-to-cost ratio greater than one. But every attempt to generalize shift updating to higher dimensions while maintaining its strong effect has failed. The reason is that in two dimensions the clusters are fragmented by the movements caused by rearranging the cells. Here we introduce the natural phenomenon of a repulsive force between cells of different types. After a birth and death event, the cells are being rearranged minimizing the overall energy expenditure. If the repulsive force is sufficiently high, shift becomes a strong promoter of cooperation in two dimensions.

Local densities connect spatial ecology to game, multilevel selection and inclusive fitness theories of cooperation

Cooperation plays a crucial role in many aspects of biology. We use the spatial ecological metrics of local densities to measure and model cooperative interactions. While local densities can be found as technical details in current theories, we aim to establish them as central to an approach that describes spatial effects in the evolution of cooperation. A resulting local interaction model neatly partitions various spatial and non-spatial selection mechanisms. Furthermore, local densities are shown to be fundamental for important metrics of game theory, multilevel selection theory and inclusive fitness theory. The corresponding metrics include structure coefficients, spatial variance, contextual covariance, relatedness, and inbreeding coefficient or F-statistics. Local densities serve as the basis of an emergent spatial theory that draws from and brings unity to multiple theories of cooperation.

Games among relatives revisited

We present a simple model for the evolution of social behavior in family-structured, finite sized populations. Interactions are represented as evolutionary games describing frequency-dependent selection. Individuals interact more frequently with siblings than with members of the general population, as quantified by an assortment parameter r, which can be interpreted as "relatedness". Other models, mostly of spatially structured populations, have shown that assortment can promote the evolution of cooperation by facilitating interaction between cooperators, but this effect depends on the details of the evolutionary process. For our model, we find that sibling assortment promotes cooperation in stringent social dilemmas such as the Prisoner's Dilemma, but not necessarily in other situations. These results are obtained through straightforward calculations of changes in gene frequency. We also analyze our model using inclusive fitness. We find that the quantity of inclusive fitness does not exist for general games. For special games, where inclusive fitness exists, it provides less information than the straightforward analysis.

Adaptive dynamics with interaction structure

Evolutionary dynamics depend critically on a population's interaction structure-the pattern of which individuals interact with which others, depending on the state of the population and the environment. Previous research has shown, for example, that cooperative behaviors disfavored in well-mixed populations can be favored when interactions occur only between spatial neighbors or group members. Combining the adaptive dynamics approach with recent advances in evolutionary game theory, we here introduce a general mathematical framework for analyzing the long-term evolution of continuous game strategies for a broad class of evolutionary models, encompassing many varieties of interaction structure. Our main result, the canonical equation of adaptive dynamics with interaction structure, characterizes expected evolutionary trajectories resulting from any such model, thereby generalizing a central tool of adaptive dynamics theory. Interestingly, the effects of different interaction structures and update rules on evolutionary trajectories are fully captured by just two real numbers associated with each model, which are independent of the considered game. The first, a structure coefficient, quantifies the effects on selection pressures and thus on the shapes of expected evolutionary trajectories. The second, an effective population size, quantifies the effects on selection responses and thus on the expected rates of adaptation. Applying our results to two social dilemmas, we show how the range of evolutionarily stable cooperative behaviors systematically varies with a model's structure coefficient.

Global migration can lead to stronger spatial selection than local migration

The outcome of evolutionary processes depends on population structure. It is well known that mobility plays an important role in affecting evolutionary dynamics in group structured populations. But it is largely unknown whether global or local migration leads to stronger spatial selection and would therefore favor to a larger extent the evolution of cooperation. To address this issue, we quantify the impacts of these two migration patterns on the evolutionary competition of two strategies in a finite island model. Global migration means that individuals can migrate from any one island to any other island. Local migration means that individuals can only migrate between islands that are nearest neighbors; we study a simple geometry where islands are arranged on a one-dimensional, regular cycle. We derive general results for weak selection and large population size. Our key parameters are: the number of islands, the migration rate and the mutation rate. Surprisingly, our comparative analysis reveals that global migration can lead to stronger spatial selection than local migration for a wide range of parameter conditions. Our work provides useful insights into understanding how different mobility patterns affect evolutionary processes.

Measures of success in a class of evolutionary models with fixed population size and structure

We investigate a class of evolutionary models, encompassing many established models of well-mixed and spatially structured populations. Models in this class have fixed population size and structure. Evolution proceeds as a Markov chain, with birth and death probabilities dependent on the current population state. Starting from basic assumptions, we show how the asymptotic (long-term) behavior of the evolutionary process can be characterized by probability distributions over the set of possible states. We then define and compare three quantities characterizing evolutionary success: fixation probability, expected frequency, and expected change due to selection. We show that these quantities yield the same conditions for success in the limit of low mutation rate, but may disagree when mutation is present. As part of our analysis, we derive versions of the Price equation and the replicator equation that describe the asymptotic behavior of the entire evolutionary process, rather than the change from a single state. We illustrate our results using the frequency-dependent Moran process and the birth-death process on graphs as examples. Our broader aim is to spearhead a new approach to evolutionary theory, in which general principles of evolution are proven as mathematical theorems from axioms.

Evolutionary shift dynamics on a cycle

We present a new model of evolutionary dynamics in one-dimensional space. Individuals are arranged on a cycle. When a new offspring is born, another individual dies and the rest shift around the cycle to make room. This rule, which is inspired by spatial evolution in somatic tissue and microbial colonies, has the remarkable property that, in the limit of large population size, evolution acts to maximize the payoff of the whole population. Therefore, social dilemmas, in which some individuals benefit at the expense of others, are resolved. We demonstrate this principle for both discrete and continuous games. We also discuss extensions of our model to other one-dimensional spatial configurations. We conclude that shift dynamics in one dimension is an unusually strong promoter of cooperative behavior.

Prosperity is associated with instability in dynamical networks

Social, biological and economic networks grow and decline with occasional fragmentation and re-formation, often explained in terms of external perturbations. We show that these phenomena can be a direct consequence of simple imitation and internal conflicts between 'cooperators' and 'defectors'. We employ a game-theoretic model of dynamic network formation where successful individuals are more likely to be imitated by newcomers who adopt their strategies and copy their social network. We find that, despite using the same mechanism, cooperators promote well-connected highly prosperous networks and defectors cause the network to fragment and lose its prosperity; defectors are unable to maintain the highly connected networks they invade. Once the network is fragmented it can be reconstructed by a new invasion of cooperators, leading to the cycle of formation and fragmentation seen, for example, in bacterial communities and socio-economic networks. In this endless struggle between cooperators and defectors we observe that cooperation leads to prosperity, but prosperity is associated with instability. Cooperation is prosperous when the network has frequent formation and fragmentation.

Evolutionary establishment of moral and double moral standards through spatial interactions

Situations where individuals have to contribute to joint efforts or share scarce resources are ubiquitous. Yet, without proper mechanisms to ensure cooperation, the evolutionary pressure to maximize individual success tends to create a tragedy of the commons (such as over-fishing or the destruction of our environment). This contribution addresses a number of related puzzles of human behavior with an evolutionary game theoretical approach as it has been successfully used to explain the behavior of other biological species many times, from bacteria to vertebrates. Our agent-based model distinguishes individuals applying four different behavioral strategies: non-cooperative individuals ("defectors"), cooperative individuals abstaining from punishment efforts (called "cooperators" or "second-order free-riders"), cooperators who punish non-cooperative behavior ("moralists"), and defectors, who punish other defectors despite being non-cooperative themselves ("immoralists"). By considering spatial interactions with neighboring individuals, our model reveals several interesting effects: First, moralists can fully eliminate cooperators. This spreading of punishing behavior requires a segregation of behavioral strategies and solves the "second-order free-rider problem". Second, the system behavior changes its character significantly even after very long times ("who laughs last laughs best effect"). Third, the presence of a number of defectors can largely accelerate the victory of moralists over non-punishing cooperators. Fourth, in order to succeed, moralists may profit from immoralists in a way that appears like an "unholy collaboration". Our findings suggest that the consideration of punishment strategies allows one to understand the establishment and spreading of "moral behavior" by means of game-theoretical concepts. This demonstrates that quantitative biological modeling approaches are powerful even in domains that have been addressed with non-mathematical concepts so far. The complex dynamics of certain social behaviors become understandable as the result of an evolutionary competition between different behavioral strategies.