###### From the Cover

# Evolution of cooperation by phenotypic similarity

^{a}Hisashi Ohtsuki,

^{b,}

^{c}John Wakeley,

^{d}Peter D. Taylor,

^{e}and Martin A. Nowak

^{a,}

^{d,}

^{1}

^{a}Program for Evolutionary Dynamics and Department of Mathematics, Harvard University, Cambridge, MA 02138;

^{b}Department of Value and Decision Science, Tokyo Institute of Technology, Tokyo 152-8552, Japan;

^{c}Precursory Research for Embryonic Science and Technology, Japan Science and Technology Agency, Saitama 332-0012, Japan;

^{d}Department of Organismic and Evolutionary Biology, Harvard University, Cambridge, MA 02138; and

^{e}Department of Mathematics and Statistics, Queen's University, Kingston, ON, Canada K7L 3N6

^{1}To whom correspondence should be addressed. E-mail: ude.dravrah@kawon_nitram

Author contributions: T.A., H.O., J.W., P.D.T., and M.A.N. performed research; and T.A., H.O., J.W., P.D.T., and M.A.N. wrote the paper.

## Abstract

The emergence of cooperation in populations of selfish individuals is a fascinating topic that has inspired much work in theoretical biology. Here, we study the evolution of cooperation in a model where individuals are characterized by phenotypic properties that are visible to others. The population is well mixed in the sense that everyone is equally likely to interact with everyone else, but the behavioral strategies can depend on distance in phenotype space. We study the interaction of cooperators and defectors. In our model, cooperators cooperate with those who are similar and defect otherwise. Defectors always defect. Individuals mutate to nearby phenotypes, which generates a random walk of the population in phenotype space. Our analysis brings together ideas from coalescence theory and evolutionary game dynamics. We obtain a precise condition for natural selection to favor cooperators over defectors. Cooperation is favored when the phenotypic mutation rate is large and the strategy mutation rate is small. In the optimal case for cooperators, in a one-dimensional phenotype space and for large population size, the critical benefit-to-cost ratio is given by $b/c=1+2/\sqrt{3}$. We also derive the fundamental condition for any two-strategy symmetric game and consider high-dimensional phenotype spaces.

**Keywords:**coalescent theory, evolutionary dynamics, evolutionary game theory, mathematical biology, stochastic process

Evolutionary game theory is the study of frequency-dependent selection (1–8). Fitness values depend on the relative abundance, or frequency, of various strategies in the population, for example, the frequency of cooperators and defectors. Evolutionary game theory has been applied to understand the evolution of cooperative interactions in viruses, bacteria, plants, animals, and humans (9–13). The classical approach to evolutionary game dynamics assumes well-mixed populations, where every individual is equally likely to interact with every other individual (4). Recent advances include the extension to populations that are structured by geography or other factors (14–25).

The term “greenbeard effect” was coined in sociobiology to describe the result of the following thought experiment (26, 27). What evolutionary dynamics will occur if a single gene is responsible for both a phenotypic signal (“a green beard”) and a behavioral response (for example, altruistic behavior toward individuals with like phenotypes)? Later, the term “armpit effect” was introduced (28) to refer to a self-referent phenotype that is used in identifying kin (29–31).

Both of these concepts are now seen as cases of “tag-based cooperation,” in which a generic system of phenotypic tags is used to indicate similarity or difference, and the evolutionary dynamics of cooperation are studied in the context of these tags. A first approach, based on computer simulations, assumed a well-mixed population, a continuum of tags, and an evolving threshold distance for cooperation (32). More recent models use numerical and analytic methods and often combine tags with viscous population structure (33–37). A general finding of these articles is that it is difficult to obtain cooperation in tag-based models for well-mixed populations, indicating that some spatial structure is needed (14).

Inspired by work on tag-based cooperation (32–34, 38) and building on a previous approach (39), we study evolutionary game dynamics in a model where the behavior depends on phenotypic distance (40, 41). As a particular example we explore the evolution of cooperation (42, 43). Studies of different organisms, including humans, support the idea that cooperation is more likely among similar individuals (31, 44–49). Our model applies to situations where individuals tend to like those who have similar attitudes and beliefs. We introduce a natural model in which individuals mutate to adjacent phenotypes in a possibly multidimensional phenotype space. We study one and infinitely many dimensions in detail. We develop a theory for general evolutionary games, not just the evolution of cooperation. Spatial structure is not needed for cooperation to be favored in our model. Moreover, in contrast to previous work (39), we develop an analytic machinery for describing heterogeneous populations in phenotype space.

Consider a population of asexual haploid individuals, with a population size *N* that is constant over time. Each individual is characterized by a phenotype, given by an integer *i* that can take any value from minus to plus infinity. Thus, this phenotype space is a one-dimensional and unbounded lattice. Individuals inherit the phenotype of their parent subject to some small variation. If the parent's phenotype is *i*, then the offspring has phenotype *i* − 1, *i*, or *i* + 1 with probabilities *v*, 1 − 2*v*, and *v*, respectively. The parameter *v* can vary between 0 and 1/2.

Let us consider a Wright–Fisher process. In each generation, all individuals produce the same large number of offspring. The next generation of *N* individuals is sampled from this pool of offspring. To introduce some fundamental concepts and quantities, we first study the model without any selection. No evolutionary game is yet being played, and there is only neutral drift in phenotype space. The entire population performs a random walk with a diffusion coefficient *v*, and by this process will tend to disperse over the lattice. In opposition to this, all of the individuals in the population will be, to some degree, related due to reproduction in a finite population. Thus, while occasionally the population may break up into two or more clusters, typically there is only a single cluster (50, 51). The standard deviation of the distribution in phenotype space, which is a measure for the width of the cluster, is $\sqrt{2Nv}$.

Next, we superimpose the neutral drift of two types: the strategies *A* and *B* (Fig. 1). Still for the moment, assuming no fitness differences, we have reproduction subject to mutation between *A* and *B*. Specifically, with probability *u* the offspring adopts a random strategy. The mutation–reproduction process defines a stationary distribution (52). If *u* is very small relative to *N*, the population tends to be either all-*A* or all-*B*. If *u* is large, the population tends toward one-half *A* and one-half *B*. Fig. 2 illustrates the random walk in phenotype space of the population composed of the two types *A* and *B*.

**...**

*v*= 0.25. The colors, red and blue, refer to arbitrary traits, because

**...**

By using coalescence theory (53, 54) many interesting and relevant properties of the distributions of both the strategies and phenotypic tags can be calculated. For example, the probability that two randomly chosen individuals have the same phenotype is $z=1/\sqrt{8Nv}$. The probability that two randomly chosen individuals have the same strategy and the same phenotype is *g* = *z*(1 − *Nu*/2). The probability that two individuals have the same strategy and a third individual has the same phenotype as the second is $h=z[1-Nu(2+\sqrt{3})/4]$. These results hold for large population size *N* and small mutation rate *u*; more precisely, we assume large *Nv* and small *Nu*. The relevance of *z*, *g*, and *h* will become clear below. The expressions for *z*, *g*, and *h* are derived for general *Nv* and *Nu* in supporting information (SI) *Appendix*, where they appear as Eqs. **10**, **19**, and **24**, respectively.

We can now use these insights to study game dynamics. We investigate the competition of cooperators, *C*, and defectors, *D*. Cooperators play a conditional strategy: they cooperate with all individuals who are close enough in phenotype space and defect otherwise. The notion of being close enough is modeled by a lattice structure. In particular, a cooperator with phenotype *i* cooperates only with other individuals of phenotype *i*. Defectors, in contrast, play an unconditional strategy: they always defect. Cooperation means paying a cost, *c*, for the other individual to receive a benefit *b*. The larger the total payoff of an individual, interacting equally with every member of the population, the larger the number of offspring it will produce on average. We want to calculate the critical benefit-to-cost ratio, *b*/*c*, that allows the game in phenotype space to favor the evolution of cooperation.

A configuration of the population is specified by *m*_{i} and *n*_{i}, which are the number of cooperators with phenotype *i* and the total number of individuals with phenotype *i*, respectively. The total payoff of all cooperators is ${F}_{C}={\sum}_{i}{m}_{i}(b{m}_{i}-c{n}_{i})$. The total payoff of all defectors is ${F}_{D}={\sum}_{i}({n}_{i}-{m}_{i})b{m}_{i}$. There are ${\sum}_{i}{m}_{i}$ cooperators and $N-{\sum}_{i}{m}_{i}$ defectors. The average payoff for a cooperator is ${f}_{C}={F}_{C}/{\sum}_{i}{m}_{i}$. The average payoff for a defector is ${f}_{D}={F}_{D}/(N-{\sum}_{i}{m}_{i})$. Cooperators have a higher fitness than defectors if *f*_{C} > *f*_{D}, which leads to ${\sum}_{i}{m}_{i}(b{m}_{i}-c{n}_{i})>{\sum}_{i}{m}_{i}{\sum}_{j}{m}_{j}{n}_{j}(b-c)/N$. Averaging these quantities over every possible configuration of the population, weighted by their stationary probability under neutrality, we obtain the fundamental condition

Under this condition cooperators are more abundant than defectors in the mutation-selection process. The above argument and our results are valid in the weak selection limit. A precise derivation of this inequality is presented in *SI Appendix*. Correlation terms similar to the ones above sometimes arise in studies of social behavior and population dynamics (26, 55). The first two terms in inequality (Eq. **1**) are pairwise correlations, while the third is notably a triplet correlation. Note that the argument leading to inequality (Eq. **1**) includes self-interaction, but that the effect of this becomes negligible when *N* is large.

When the population size is large, the averages in inequality (Eq. **1**) are proportional to the probabilities *g*, *z*, and *h* respectively, which we introduced earlier. Consequently, inequality (Eq. **1**) can be written as *bg* − *cz* > (*b* − *c*)*h*. Using the values of *z*,*g*, and *h* given above we obtain

which is approximately 2.16. If the benefit-to-cost ratio exceeds this number, then cooperators are more abundant than defectors in the mutation-selection process. The success of cooperators results from the balance of movement and clustering in phenotype space. Inequality (Eq. **2**) represents the condition for cooperators to be more abundant than defectors in a large population when the strategy mutation rate *u* is small (*Nu* 1) and the phenotypic mutation rate *v* is large (*Nv* 1). In *SI Appendix*, we derive conditions for any population size and mutation rates. Fig. 3 shows the excellent agreement between numerical simulations and analytical calculations. In general, we find that both lowering strategy mutations and increasing phenotypic mutations favor cooperators.

**...**

We can expand our analysis to study any 2 × 2 game, not only the interaction between cooperators and defectors. Consider two strategies *A* and *B* and the general payoff matrix

The payoffs for *A* versus *A*, *A* versus *B*, *B* versus *A*, and *B* versus *B* are given by *R*, *S*, *T*, *P*, respectively. *A* players use strategy *A* against other individuals with the same phenotype, otherwise they use *B*. *B* players always use strategy *B*. For the game in a one-dimensional phenotype space and large population size we find that *A* is more abundant than *B* if

For the derivation see Section 5.2 in the *SI Appendix*. This formula can be used for evaluating any two-strategy symmetric game in a one-dimensional phenotype space. In the *SI Appendix*, we discuss the snow-drift game and the stag-hunt game as particular examples.

We can also study higher-dimensional phenotype spaces. In general, for higher dimensions, it is easier for cooperators to overcome defectors. The intuitive reason is that in higher dimensions phenotypic identity also implies strategic identity. In Section 5.3 of the *SI Appendix*, we show that, in the limit of infinitely many dimensions, and under the same assumptions that produced conditions **2** and **4**, the crucial benefit-to-cost ratio in the Prisoner's Dilemma converges to *b*/*c* > 1. For general games, the equivalent result of condition **4** becomes *R* > *P*, which means the evolutionary process always chooses the strategy with the higher payoff against itself. Our basic approach can also be adapted to continuous, rather than discrete, phenotype spaces. In this case, no two individuals have exactly the same phenotype, but the conditional behavioral strategy is triggered by sufficient phenotypic similarity.

In summary, we have developed a model for the evolution of cooperation based on phenotypic similarity. Our approach builds on previous ideas of tag-based cooperation, but in contrast to earlier work (33–37), we do not need spatial population dynamics to obtain an advantage for cooperators. We derive a completely analytic theory that provides general insights. We find that the abundance of cooperators in the mutation-selection equilibrium is an increasing function of the phenotypic mutation rate and a decreasing function of the strategic mutation rate. These observations agree with the basic intuition that higher phenotypic mutation rates reduce the interactions between cooperators and defectors, whereas higher strategic mutation rates destabilize clusters of cooperators by allowing frequent invasion of newly mutated defectors. Therefore, cooperation is more likely to evolve if the strategy mutation rate is small and if the phenotypic mutation rate is large. In a genetic model this assumption may be fulfilled if the strategy is encoded by one or a few genes, whereas the phenotype is encoded by many genes. Also in a cultural model, it can be the case that the phenotypic mutation rates are higher than the strategic mutation rates; for example, people might find it easier to modify their superficial appearance than their fundamental behaviors. Furthermore, we show how the correlations between strategies and phenotypes can be obtained from neutral coalescence theory under the assumption that selection is weak (54, 56). Our theory can be applied to study any evolutionary game in the context of conditional behavior that is based on phenotypic similarity or difference.

## Acknowledgments.

This work was supported by the John Templeton Foundation, the National Science Foundation/National Institutes of Health (R01GM078986) joint program in mathematical biology, the Bill and Melinda Gates Foundation (Grand Challenges Grant 37874), the Japan Society for the Promotion of Science, and J. Epstein.

## Footnotes

The authors declare no conflict of interest.

See Commentary on page 8405.

This article contains supporting information online at www.pnas.org/cgi/content/full/0902528106/DCSupplemental.

## References

**National Academy of Sciences**

## Formats:

- Article |
- PubReader |
- ePub (beta) |
- PDF (1019K) |
- Citation

- Green-beard effect predicts the evolution of traitorousness in the two-tag Prisoner's dilemma.[J Theor Biol. 2011]
*Laird RA.**J Theor Biol. 2011 Nov 7; 288:84-91. Epub 2011 Aug 2.* - Spatial invasion of cooperation.[J Theor Biol. 2008]
*Langer P, Nowak MA, Hauert C.**J Theor Biol. 2008 Feb 21; 250(4):634-41. Epub 2007 Nov 6.* - Collective chasing behavior between cooperators and defectors in the spatial prisoner's dilemma.[PLoS One. 2013]
*Ichinose G, Saito M, Suzuki S.**PLoS One. 2013; 8(7):e67702. Epub 2013 Jul 5.* - Evolutionary dynamics in structured populations.[Philos Trans R Soc Lond B Biol Sci. 2010]
*Nowak MA, Tarnita CE, Antal T.**Philos Trans R Soc Lond B Biol Sci. 2010 Jan 12; 365(1537):19-30.* - From genes to games: cooperation and cyclic dominance in meiotic drive.[J Theor Biol. 2012]
*Traulsen A, Reed FA.**J Theor Biol. 2012 Apr 21; 299:120-5. Epub 2011 May 13.*

- Cooperation with both synergistic and local interactions can be worse than each alone[Scientific Reports. ]
*Li A, Wu B, Wang L.**Scientific Reports. 45536* - Extrapolating Weak Selection in Evolutionary Games[PLoS Computational Biology. 2013]
*Wu B, García J, Hauert C, Traulsen A.**PLoS Computational Biology. 2013 Dec; 9(12)e1003381* - Density games[Journal of Theoretical Biology. 2013]
*Novak S, Chatterjee K, Nowak MA.**Journal of Theoretical Biology. 2013 Oct 7; 334(100)26-34* - Processing power limits social group size: computational evidence for the cognitive costs of sociality[Proceedings of the Royal Society B: Biologi...]
*Dávid-Barrett T, Dunbar RI.**Proceedings of the Royal Society B: Biological Sciences. 2013 Aug 22; 280(1765)20131151* - Global migration can lead to stronger spatial selection than local migration[Journal of statistical physics. 2013]
*Fu F, Nowaks MA.**Journal of statistical physics. 2013 May 1; 151(3-4)637-653*

- PubMedPubMedPubMed citations for these articles

- From the Cover: Evolution of cooperation by phenotypic similarityFrom the Cover: Evolution of cooperation by phenotypic similarityProceedings of the National Academy of Sciences of the United States of America. May 26, 2009; 106(21)8597

Your browsing activity is empty.

Activity recording is turned off.

See more...