- We are sorry, but NCBI web applications do not support your browser and may not function properly. More information

# Tag-based indirect reciprocity by incomplete social information

^{1}Amari Research Unit, RIKEN Brain Science Institute, 2-1, Hirosawa, Wako, Saitama 351-0198, Japan

^{2}Department of Biology, Faculty of Sciences, Kyushu University, Hakozaki 6-10-1, Fukuoka 812-8581, Japan

^{*}Author and address for correspondence: Graduate School of Information Science and Technology, The University of Tokyo, 7-3-1 Hongo, Bunkyo, Tokyo 113-8656, Japan. (Email: pj.ca.oykot-u.i.tsim@adusam)

## Abstract

Evolution of altruistic behaviour in interacting individuals is accounted for by, for example, kin selection, direct reciprocity, spatially limited interaction and indirect reciprocity. Real social agents, particularly humans, often take actions based on similarity between themselves and others. Although tag-based indirect reciprocity in which altruism occurs exclusively among similar flocks is a natural expectation, its mechanism has not really been established. We propose a model of tag-based indirect reciprocity by assuming that each player may note strategies of others. We show that tag-based altruism can evolve to eradicate other strategies, including unconditional defectors for various initial strategy configurations and parameter sets. A prerequisite for altruism is that the strategy is sometimes, but not always, visible to others. Without visibility of strategies, policing does not take place and defection is optimal. With perfect visibility, what a player does is always witnessed by others and cooperation is optimal. In the intermediate regime, discriminators based on tag proximity, rather than mixture of generous players and defectors, are most likely to evolve. In this situation, altruism is realized based on homophily in which players are exclusively good to similar others.

**Keywords:**altruism, evolutionary game, indirect reciprocity, homophily

## 1. Introduction

According to classical game theory, defection rather than cooperation is the optimal behaviour from individualistic points of view. However, altruistic behaviour evolves in real social systems, particularly in human societies (Fehr & Fischbacher 2003). Altruism is explained by mechanisms such as kin selection (Hamilton 1964), assortative mating (Eshel & Cavalli-Sforza 1982; Bergstrom 2003), direct reciprocity (Trivers 1971; Axelrod & Hamilton 1981; Axelrod 1984), spatial reciprocity based on short-range interaction (Nowak & May 1992) and indirect reciprocity (Nowak & Sigmund 1998*a*,*b*, 2005; Brandt & Sigmund 2004, 2005; Ohtsuki & Iwasa 2004). In particular, indirect reciprocity occurs if each player carries an image score that indicates how often the player has cooperated before. Then, unconditional cooperators with high image scores are trusted by others, but they are too good to be resistant to invasion by unconditional defectors. Defectors have low scores and cannot count on altruism from others. Discriminators with moderate scores can evolve because good players trust them and defectors cannot exploit them.

In many situations, individuals are equipped with biological and social tags according to which they are categorized. Similar birds tend to flock together based on tags, which is assortative interaction (McPherson & Smith-Lovin 1987; Marsden 1988; McPherson *et al*. 2001; Newman 2003). Intuitively, players may exclusively help similar others to lead to indirect reciprocity. Imagine that we meet a stranger *P* from the same town or school by chance. Then, we may be inclined to help *P* even if we do not expect to see *P* later on. However, defection is theoretically better in such a one-shot encounter.

Tag-based reciprocity was proposed as a mechanism of indirect reciprocity (Riolo *et al*. 2001). Each player has a tolerance value as strategy. Altruistic behaviour happens when the difference between the opponent's tag and the tag of the focal player does not exceed the tolerance of the focal player. Altruism can evolve among players with close tags even if they have not met before. A key factor for tag-based altruism in this model is the assumption that players with the same tag value always help each other. If this condition is removed, which may be more realistic, cooperation does not evolve (Roberts & Sherratt 2002).

Nevertheless, these debates do not exclude the possible importance of tag-based altruism. In combination with population viscosity, which is known to elicit altruism in the Prisoner's Dilemma (Nowak & May 1992), use of tags reinforces altruism (Hammond & Axelrod 2006). Another scenario that revives tag-based altruism is that the tag and the tolerance genes sit on different alleles and inheritance of these genotypes are only loosely coupled. Thus, if many tag values are allowed and a population is spatially organized, tag-based altruism is viable (Jansen & van Baalen 2006).

In this work, we propose a mechanism of indirect reciprocity based on tags, without resorting to a prefabricated cooperative tendency (Riolo *et al*. 2001) and spatial structure of populations (Nowak & May 1992; Hammond & Axelrod 2006; Jansen & van Baalen 2006). The main assumption is that players may observe the tolerance (strategy) of opponents. In reality, third parties may communicate the information about others' strategies to a focal player. We show that tag-based indirect reciprocity emerges for a variety of initial configurations when tolerances are observable with moderate probabilities.

## 2. Model: strategy and pay-off

We assume *n* individuals involved in the evolutionary Prisoner's Dilemma game. In each round, a donor and a recipient are taken from the population randomly. If the donor decides to donate, the donor loses cost *c* and the recipient gains benefit *b*, which is larger than *c*. If the donor refuses to donate, both the donor and the recipient get nothing. A generation consists of a sufficiently small number of rounds that prohibits two specific players from interacting more than once on average. Therefore, there is no room for direct reciprocity. After a generation, the strategy of players with larger total pay-offs is more likely to be disseminated, whereas poor strategies are eliminated. Since *b*>*c*, the population pay-off is maximized when everybody is always altruistic. However, unconditional defection, but not cooperation, is the evolutionary stable strategy (ESS) (Axelrod & Hamilton 1981; Axelrod 1984); here is a dilemma.

We assume that the *i*th individual (1≤*i*≤*n*) has tag *w*_{i} and tolerance *μ*_{i}. Everybody can see others' tags. Tags may take nominal or graduated values. In either case, the distance between two tags 0≤*l*(*w*_{i},*w*_{j})≤*l*_{max} is defined, where *l*_{max} is the maximal distance. The tag *w*_{i} can be assumed to vary in time (Riolo *et al*. 2001; Jansen & van Baalen 2006), which is typically the case in, for example, the preference of political parties in democratic regimes and fashions in clothing. Tags evolve much more slowly than do tolerances in other situations. For example, inborn biological or social traits and strong beliefs can be considered invariant for a player, whereas tolerances can adapt via social learning on a relatively short time-scale. We focus on the latter case; we consider the evolution of tolerances with tags fixed.

Suppose that player *i* and player *j* are, respectively, chosen as donor and recipient in a single round. Then, the player *i* may donate to the player *j* if *i* and *j* are close enough according to *i*'s criterion. In the previous model (Riolo *et al*. 2001), *i* pays *c* to benefit *j* by *b* if *l*(*w*_{i},*w*_{j})≤*μ*_{i}, where *μ*_{i} is the tolerance level of *i*. Players with *μ*_{i}≥*l*_{max} always donate, whereas players with *μ*_{i}<0 never donate. These players do not resort to tags. Riolo *et al*. (2001) excluded unconditional defectors by setting *μ*_{i}≥0 and observed indirect reciprocity. However, we allow negative *μ*_{i} to investigate whether tag users can evolve in the presence of unconditional defectors (*μ*_{i}<0). Without additional mechanisms, unconditional defectors outperform unconditional cooperators and tag users; hence, there is no altruism (Roberts & Sherratt 2002).

To explore the possibility of tag-based indirect reciprocity, we introduce the probability *q* that the donor *i* detects *μ*_{j}, namely, the tolerance of the recipient *j*. We can tune the reputations of others' tolerances to be public with a large *q* or private with a small *q* (Nowak & Sigmund 1998*a*,*b*; Brandt & Sigmund 2004, 2005), whereas perfect social information (*q*=1) is assumed in other models (e.g. Ohtsuki & Iwasa 2004). If *i* knows that *l*(*w*_{i},*w*_{j})>*μ*_{j}, *i* is not motivated to be nice to *j*. Note that this is not because *i* does not expect cooperation back from *j*; *i* and *j* hardly interact more than once per generation. Rather, this is because *i* expects that *j* will not be nice to other players whose tags are close to *w*_{i}. If other players do the same, natural selection will eliminate *μ*_{j}, which may lead to an altruistic population.

If *q*=0, the donor never sees the tolerance of the recipient. In this case, *μ*<0 will be stable (Roberts & Sherratt 2002). If *q*=1, the tolerance of everybody is transparent to the entire population. Thus, small *μ*_{i} will disappear rapidly, and everybody will be eventually altruistic (*μ*=*l*_{max}). We examine what occurs for intermediate values of *q*.

## 3. Two-tag model

We begin with a minimal model in which each player has tag *w*^{a} or *w*^{b}. Such a minimal model was used to analyse the replicator dynamics of the original model by Riolo *et al*. (Traulsen & Schuster 2003). We define *l*(*w*^{a},*w*^{a})=*l*(*w*^{b},*w*^{b})=0 and *l*(*w*^{a},*w*^{b})=1. It suffices to consider three tolerance values *μ*=−1, 0 and 1. Players with *μ*=−1 are unconditional defectors. Players with *μ*=0 are tag users. Players with *μ*=1 are unconditional cooperators if *q*=0. When tolerances of opponents are visible (*q*>0), these players do not necessarily cooperate. We call them generous players. We could introduce unconditional cooperators and players who neglect others' tolerances even when available. However, these strategies are outperformed by players using tags and tolerances. Therefore, we consider only the three strategies (*μ*=1, 0 and −1).

Suppose that a population consists of *n* players and that *nt* (*n*(1−*t*)) players are endowed with tag *w*^{a} (*w*^{b}). We denote by ${p}_{\mu}^{a}$ the probability that a player with tag *w*^{a}, who we call *w*^{a}-player, has tolerance *μ*. Similarly, a *w*^{b}-player has tolerance *μ* with probability ${p}_{\mu}^{b}$. If the population is well mixed, any player meets a *w*^{a}-player (*w*^{b}-player) and tolerance *μ* with probability $t{p}_{\mu}^{a}((1-t){p}_{\mu}^{b})$. Since the tag is visible to others, players may prefer to interact with conspecifics. Such assortative interaction may stem from spatial structure of the population. To take assortativity into account, we assume that the two players with the same tag can meet more frequently. Precisely, a player who meets an opponent with the opposite tag looks for an alternative opponent with the same tag with probability *h* (0≤*h*≤1). Thus, a *w*^{a}-player interacts with a *w*^{a}-player (*w*^{b}-player) with probability *t*+*h*(1−*t*) ((1−*h*)(1−*t*)), whereas a *w*^{b}-player interacts with a *w*^{a}-player (*w*^{b}-player) with probability (1−*h*)*t* (1−*t*+*ht*) (see table 1; Eshel & Cavalli-Sforza 1982; Bergstrom 2003). Non-assortative interaction is reproduced by *h*=0, and *h*=1 defines completely assortative interaction in which players with opposite tags never interact.

Disregarding normalization, the pay-off ${\Pi}_{\mu}^{a}({\Pi}_{\mu}^{b})$ for a *w*^{a}-player (*w*^{b}-player) with tolerance *μ* is given by

and

Then, we examine the replicator dynamics (Taylor & Jonker 1978; Hofbauer & Sigmund 1998) of the population density described by

The pay-off of *w*^{a}-players depends on the tolerance distribution of *w*^{b}-players, who do not compete with *w*^{a}-players, and vice versa. There are six variables (${p}_{-1}^{a}$, ${p}_{0}^{a}$, ${p}_{1}^{a}$, ${p}_{-1}^{b}$, ${p}_{0}^{b}$ and ${p}_{1}^{b}$). Since we are interested in competition among tolerances with tags fixed, the number of *w*^{a}-players (=*nt*) and that of *w*^{b}-players (=*n*(1−*t*)) is preserved. Then, we obtain ${p}_{-1}^{a}+{p}_{0}^{a}+{p}_{1}^{a}=1$, ${p}_{-1}^{b}+{p}_{0}^{b}+{p}_{1}^{b}=1$, and the replicator system represented by equations (3.7) and (3.8) is four-dimensional.

### (a) Symmetric strategies

Let us start with a simple case in which *w*^{a}-players equal *w*^{b}-players in number (*t*=0.5) and the initial condition is symmetric: ${p}_{\mu}^{a}={p}_{\mu}^{b}$ (*μ*=−1, 0 and 1). Then, the dynamics represented by equations (3.7) and (3.8) preserve the relations ${p}_{\mu}\equiv {p}_{\mu}^{a}={p}_{\mu}^{b}$ and ${\Pi}_{\mu}\equiv {\Pi}_{\mu}^{a}={\Pi}_{\mu}^{b}$ (*μ*=−1, 0 and 1), and are equivalent to the two-dimensional replicator dynamics with *p*_{−1}, *p*_{0} and *p*_{1} (*p*_{−1}+*p*_{0}+*p*_{1}=1). Equations (3.1)–(3.6) are reduced to

and

Then, we have

and

As schematically shown in figure 1, the tolerance distribution of a population is mapped to a point in the simplex defined by *p*_{0}+*p*_{1}+*p*_{2}=1 and *p*_{0}, *p*_{1}, *p*_{2}≥0. On the line *p*_{−1}=0 (or *p*_{0}+*p*_{1}=1), we have ${\Pi}_{1}>{\Pi}_{0}\leftrightarrow {p}_{1}>A$, where *A*=*c*(1−*q*)/[(*b*−*c*)*q*]≥0. Similarly, we derive ${\Pi}_{1}>{\Pi}_{-1}\leftrightarrow {p}_{1}>A$ on *p*_{0}=0 and ${\Pi}_{0}>{\Pi}_{-1}\leftrightarrow {p}_{0}>A$ on *p*_{1}=0. Therefore, if *A*<1, then (*p*_{−1},*p*_{0},*p*_{1})=(1,0,0),(0,1,0),(0,0,1) are ESSs. The condition *A*<1 is equivalent to *bq*>*c*, which coincides with the requirement for altruism in the image scoring model (Nowak & Sigmund 1998*a*,*b*). If *bq*≤*c*, *Π*_{−1} is dominant, and unconditional defectors eventually occupy the population.

*a*–

*c*) Schematics of population dynamics for

*h*=0 and (

*a*) small, (

*b*) intermediate, and (

*c*) large

*q*. (

*d*,

*e*) Comparison of different degrees of assortativity with

*b*=1.0,

*c*=0.3 and

*q*=0.5. (

*d*)

*h*=0 and (

*e*)

*h*=0.8.

**...**

Let us concentrate on the case *bq*>*c* (*A*<1). In addition to the three corners of the simplex, (*p*_{−1},*p*_{0},*p*_{1})=(1−*A*,0,*A*) is a degenerated source (see Appendix A). Dynamics with non-assortative interaction (*h*=0) are compared in figure 1*a–c* for different values of *q*. For a small *q* (*bq**c*), most initial configuration leads to domination by unconditional defectors (*p*_{−1}=1 and *p*_{0}=*p*_{1}=0), as shown in figure 1*a*. Since players can keep their tolerances secret, it is better to behave selfishly (Roberts & Sherratt 2002). As *q* increases (equivalently, *A* decreases), tolerances are observed by others, and it is rational to be nice. An increase in *q*, indeed, enlarges the attractive basins of the unanimity of tag users (*p*_{0}=1 and *p*_{−1}=*p*_{1}=0) and that of generous players (*p*_{1}=1 and *p*_{−1}=*p*_{0}=0). By contrast, the attractive basin of *p*_{−1}=1 shrinks. For intermediate *q*, the attractive basin of *p*_{0}=1 (shaded areas in figure 1*a–c*) is large. Particularly, when *q*=2*c*/(*b*+*c*) (namely, *A*=0.5), the attractive basin of *p*_{0}=1 is largest and twice as large as those of *p*_{−1}=1 and *p*_{1}=1 (figure 1*b*). When *q* is close to 2*c*/(*b*+*c*), tag users are likely to outperform generous players and defectors to establish tag-based altruism for many initial conditions. If *q* is even larger, being generous is optimal for many configurations (figure 1*c*).

Numerically calculated replicator dynamics for *q*=0.5 are shown in figure 1*d*,*e* for two values of *h*. The results for *h*=0 (non-assortative encountering) resemble figure 1*b*. When *h* is large (assortative encountering), *μ*=0 and 1 are not so distinct because players do not often play with others with the opposite tag. Then, the trajectories inside the simplex are more or less symmetric about *p*_{0}=*p*_{1} (figure 1*e*). By contrast, if players with the opposite tags are more likely to interact, *μ*=−1 and 0 are less distinguished. Note that assortativity (*h*>0) is not necessary for tag-based reciprocity.

### (b) Asymmetric strategies

The assumption that *w*^{a}- and *w*^{b}-subpopulations share the identical number of players and the identical tolerance distributions is too restrictive. Here, we relax *t*=0.5 and ${p}_{i}^{a}={p}_{i}^{b}$ (*i*=0,1,2). Now the population dynamics described by equations (3.7) and (3.8) are four dimensional. Since analytical treatment of the four-dimensional system is intractable, we look for pure-strategy ESSs. If a pure strategy of the *w*^{a}-subpopulation is the best response to the *w*^{b}-subpopulation with a pure strategy and vice versa, this pair of tolerances is an ESS. Therefore, we investigate compatibility of 3×3=9 pure strategies.

Let us fix the tolerance distribution of the *w*^{b}-players. Then, the best response for *w*^{a}-players can be derived from equations (3.1)–(3.3). The best response depending on the tolerance density of *w*^{a}-players is shown in figure 2*a* for ${p}_{1}^{b}>A$. Since $X\equiv \left[A-\left(1-\left(t+h-ht\right)\right){p}_{1}^{b}\right]/\left(t+h-ht\right)<A$ for ${p}_{1}^{b}>A$, the region defined by ${p}_{1}^{a}>A$ belongs to the basin of ${p}_{1}^{a}=1$. The same conclusion obeys with *a* and *b* swapped and *t*+*h*−*ht* replaced by 1−*t*+*ht*. Starting from ${p}_{1}^{a}>A$ and ${p}_{1}^{b}>A$, generous players will eventually eradicate the other two strategies in both subpopulations. This extends the result for the symmetric case.

*w*

^{a}-players when (

*a*) ${p}_{1}^{b}>A$ and (

*b*) ${p}_{1}^{b}<A$. We set $X=\left[A-\left(1-\left(t+h-ht\right)\right){p}_{1}^{b}\right]/\left(t+h-ht\right)$.

The best response for *w*^{a}-players when ${p}_{1}^{b}<A$ is shown in figure 2*b*. Since ${p}_{1}^{b}<A$, possible pure strategies for the *w*^{b}-subpopulation are *μ*=−1 and 0. Figure 2*b* implies that the pure strategy ${p}_{1}^{a}=1$ is incompatible with ${p}_{-1}^{b}=1$ and ${p}_{0}^{b}=1$. Similarly, ${p}_{1}^{b}=1$ is incompatible with ${p}_{-1}^{a}=1$ and ${p}_{0}^{a}=1$. Therefore, we exclude $({p}_{-1}^{a},{p}_{0}^{a},{p}_{1}^{a},{p}_{-1}^{b},{p}_{0}^{b},{p}_{1}^{b})=(1,0,0,0,0,1),\phantom{\rule{0.75em}{0ex}}(0,1,0,0,0,1),\phantom{\rule{0.75em}{0ex}}(0,0,1,1,0,0)\phantom{\rule{0.75em}{0ex}}\text{and}(0,0,1,0,1,0)$. By contrast, four pure strategies: $({p}_{-1}^{a},{p}_{0}^{a},{p}_{1}^{a},{p}_{-1}^{b},{p}_{0}^{b},{p}_{1}^{b})={S}_{1}:(1,0,0,1,0,0),\phantom{\rule{0.45em}{0ex}}{S}_{2}:(1,0,0,0,1,0),\phantom{\rule{0.75em}{0ex}}{S}_{3}:(0,1,0,1,0,0)\phantom{\rule{0.75em}{0ex}}\text{and}\phantom{\rule{0.75em}{0ex}}{S}_{4}:(0,1,0,0,1,0)$ are self-consistent stable steady states. Solutions *S*_{1} (all defectors) and *S*_{4} (all tag users) are symmetric, as discovered in §3*a*. With *S*_{2} (*w*^{a}-defectors and *w*^{b}-tag users) or *S*_{3} (*w*^{a}-tag users and *w*^{b}-defectors), players do not cooperate with dissimilar others. Consequently, how nice a *w*^{a}- (*w*^{b}-) player is to other *w*^{a}- (*w*^{b}-) players does not affect the pay-off and the tolerances of *w*^{b}- (*w*^{a}-) players. Since two groups are independent of each other, asymmetric solutions are allowed.

To investigate which of the five ESSs identified above is more feasible, we perform 3000 runs of evolutionary dynamics with different initial conditions. The initial tolerance density is chosen according to the uniform distribution on the simplex: ${p}_{-1}^{x}+{p}_{0}^{x}+{p}_{1}^{x}=1$ (${p}_{-1}^{x},{p}_{0}^{x},{p}_{1}^{x}\ge 0$) independently for *x*=*a* and *b*, respectively. The initial tolerance of each player is picked according to the tolerance distributions generated in this way. We set *n*=300, *b*=1.0 and *c*=0.3. One generation consists of 8*n* rounds so that each player appears eight times as donor and eight times as recipient on average. Since 8×2*n*, a specific pair of players hardly meet more than once, and direct reciprocity is ruled out. We study asynchronous updating: after each generation, we randomly choose a small number of players (equal to 8). For each of these players, the tolerance is replaced by that of player *i* with probability $\left({\pi}_{i}-{\text{min}}_{j=1}^{n}{\pi}_{j}\right)/{\sum}_{i=1}^{n}\left({\pi}_{i}-{\text{min}}_{j=1}^{n}{\pi}_{j}\right)$, where *π*_{i} (1≤*i*≤*n*) is the *i*th player's generation pay-off. Players with larger *π*_{i} are more likely to bear offspring. The players with the minimal pay-off ($={\text{min}}_{j=1}^{n}{\pi}_{j}$) do not reproduce themselves.

The final proportion of each pure strategy is shown in figure 3. For small *q*, *μ*=−1 results from most initial conditions for both subpopulations (dotted line). By contrast, *μ*=1 is reached for both subpopulations when *q* is large (dash-dotted line). For intermediate *q*, three pure strategies comprising tag users can evolve. The probability that any of the three ESSs, with tag users fixates is pretty large (≥0.5) for 0.4≤*q*≤0.55. This range of *q* agrees with *q*=2*c*/(*b*+*c*)0.46, which is the optimal *q* for prosperity of tag users in the symmetric population (§3*a*). The probability that any of the other four pure strategies dominates the population is tiny and is thus omitted in figure 3.

## 4. Continuous-tag model

Let us consider the case of continuously distributed tags, which is more realistic in some practical situations. Similar to Riolo *et al*. (2001), we assume that tags are uniformly distributed on [0,1], where 0 and 1 are identified, and that the interaction is non-assortative. The difference between two tags is naturally defined as the distance on the ring. Formally, *l*(*w*_{i},*w*_{j})=min(|*w*_{i}−*w*_{j}|, 1−|*w*_{i}−*w*_{j}|) and 0≤*l*(*w*_{i},*w*_{j})≤*l*_{max}0.5. We set *μ*_{i}[*μ*_{min},*μ*_{max}], where *μ*_{min}=−10^{−6} and *μ*_{max}=0.5. Players with negative *μ*_{i} are unconditional defectors. Along the lines of the two-tag model, it is straightforward to show that a population of players with *μ*_{i}=*μ* (1≤*i*≤*n*) for any *μ* is not invaded by a player with a different tolerance, given *bq*>*c*. Such a unanimous population is an ESS for any *μ*. We are interested in which *μ* will evolve.

To see this, imagine that each player initially owns a tolerance value taken from the uniform distribution. Without losing generality, we assume that a reference player *P*_{1} has tag *w*=0.5 and tolerance *μ*. Player *P*_{1} potentially donates to others whose tag *w*′[0.5−*μ*, 0.5+*μ*]. Owing to the uniform tag density, a proportion of 2Δ*l* players, or players with tag *w*′[0.5+*l*, 0.5+*l*+Δ*l*] or *w*′[0.5−*l*−Δ*l*, 0.5−*l*], are in the range of distance [*l*,*l*+Δ*l*] from *P*_{1}.

Since encounters occur in a uniform manner, *P*_{1}’s pay-off is equal to

up to normalization. The first term indicates that, when the tag is visible with probability *q*, the tolerance of both *P*_{1} (*μ*) and the opponents (*μ*′) must be larger than *l* for altruism. The second term corresponds to the situation in which *P*_{1} sneaks benefit *b* without exposing that *P*_{1} is selfish. The third term represents the opposite situation. Since *μ*_{min}0, equation (4.1) is reduced to

If *bq*≤*c*, the optimal tolerance is *μ*=0*μ*_{min}, namely, unconditional defection; the same conclusion as that for the two-tag model. If *bq*>*c*, the optimal tolerance is given by

which is smaller than *μ*_{max} unless *q*=1. Now, it is beneficial to use tags. As a function of *q*, the best response to the uniform tolerance density is plotted in figure 4 for the three values of *b*/*c*. The optimal tolerance increases with *q*. Tag users prosper for a wider range of *q* in more benign environments (for *b*/*c* large).

*b*/

*c*=1.2 (thinnest line), 2 (moderate line) and 4 (thickest line).

We numerically simulate the continuous-tag model to confirm the above prediction. Initially, the tag and the tolerance of *n* players are uniformly and independently distributed on [0,1] and [*μ*_{min}, *μ*_{max}], respectively. We set *n*=800, *b*=1.0 and *c*=0.3. Noiseless simulations are performed as described in §3*b*. In noisy simulations, after each generation with natural selection, eight randomly chosen players are mutated so that their tolerances are drifted by a random amount chosen uniformly from [−0.03,0.03].

The population average of the tolerance after transient is shown in figure 5*a* with mutation present (filled squares) and absent (open circles). The estimation based on equation (4.3) (solid line, figure 5*a*) agrees with the stationary tolerance pretty well. Tag users can evolve for intermediate values of *q*, whereas extreme values of *q* foster unconditional defectors or generous players. The pay-off averaged over the population increases linearly with the final *μ* (results not shown). The standard deviations of the tolerance based on *n* players stay small in each run (figure 5*b*). This indicates that *μ*_{i} tends to align even under continuously distributed tags and dynamical noise.

## 5. Discussion

We have shown that partial visibility of the tolerance (strategy) leads to tag-based indirect reciprocity. Our results extend Roberts & Sherratt's (2002) work, which showed that unconditional defection is the evolutionary outcome when the strategy of each player is completely sealed from others' eyes. If the tolerance is always observed by others, intolerant players reveal themselves to be uncooperative and cannot receive altruism from others. Thus, being generous is optimal; no room to cheat. When *bq*>*c*, a population in which all the players share an identical tolerance (whatever the value) is an ESS. This implies that a homogeneous population entirely of tag users with a shared tolerance level, as well as one of generous players and one of defectors, can be an ESS. Which ESS is more likely to be reached is sensitive to *q*, the probability that a player observes tolerance of an opponent. For an intermediate *q*, the unanimity of tag users results from a wide range of initial tolerance distributions including the uniform distribution. Naturally, the ultimate tolerance of the population monotonically increases with *q*. More generous players prosper for larger *q* (Nowak & Sigmund 1998*a*,*b*; Nowak *et al*. 2000; Brandt & Sigmund 2004, 2005).

The range of *q* hosting evolution of tag users, which is determined by *b* and *c*, is not necessarily large. However, starting from a population with more tag users than expected from the uniform density, the unanimity of tag users is more easily reached. In addition, our social lives are full of partial information. We usually have some but not complete information about others. Therefore, it is plausible to assume an intermediate value of *q*, for which we showed the possibility of tag-based altruism.

The mechanism that we have identified is that for indirect reciprocity, not direct reciprocity. Suppose that the player *P*_{1} is tolerant enough to behave altruistically to player *P*_{2}. Even so, *P*_{1} is discouraged to be nice to *P*_{2} if *P*_{2} is revealed to be intolerant with probability *q*. In other words, *P*_{1} is not motivated to cooperate with those who will not cooperate back to *P*_{1}. Although it may sound like direct reciprocity, every decision is based on tags and tolerances, and *P*_{1} and *P*_{2} do not have to interact in repetition.

In the previous models of tag-based indirect reciprocity, unconditional defectors are eliminated by definition (Riolo *et al*. 2001; Traulsen & Schuster 2003). Tag users can easily evolve under this condition. By contrast, our model allows unconditional defectors. Furthermore, the boundary condition was such that random mutation somehow tended to make players more cooperative. Our model is without such a drift. A positive tolerance drift would induce a cyclic-dominance relation between different phenotypes (Riolo *et al*. 2001; Traulsen & Schuster 2003), which is not the case in our model. With unconditional defectors and without the positive drift, survival of tag users is more challenging. We have related tag-based actions to altruism by introducing the probability that tolerances are observed by others.

The proposed mechanism of indirect reciprocity does not necessitate the memory of previous acts or strategies, in contrast to image scoring (Nowak & Sigmund 1998*a*,*b*; Brandt & Sigmund 2004, 2005; Ohtsuki & Iwasa 2004) and direct reciprocity (Trivers 1971; Axelrod & Hamilton 1981; Axelrod 1984). In our model, the action depends only on present tags and tolerances. In addition, assortative mixing has small effects on the conclusions (compare figure 1*d*,*e*). Assortativity is not mandatory for our mechanism. Neither do we need population viscosity (Nowak & May 1992; Hammond & Axelrod 2006; Jansen & van Baalen 2006), which is one way to implement assortativity. Furthermore, tag-based reciprocity does not necessitate weak coupling of tag and tolerance (Jansen & van Baalen 2006). This study is a first important step towards understanding the evolution of the tag-based indirect reciprocity.

## Acknowledgments

We thank Mayuko Nakamaru for helpful comments on this work. N.M. acknowledges support from the Special Postdoctoral Researchers Program of RIKEN, and H.O. acknowledges support from the Japan Society for the Promotion of Science (JSPS).

## Appendix A.

We examine the stability of the steady-state (*p*_{−1},*p*_{0},*p*_{1})=(1−*A*,0,*A*) of the symmetric two-tag model. By eliminating (*p*_{−1}=1−*p*_{0}−*p*_{1}), we derive the two-dimensional linear dynamics around the steady state up to the second order,

Here, Δ*p*_{1} and Δ*p*_{2} are perturbations to the steady state, and *o*(Δ*p*_{1},Δ*p*_{2}) summarizes small quantities relative to Δ*p*_{1} and Δ*p*_{0}. The first-order terms in the r.h.s. of equations (A1) and (A2) indicate that the Jacobian has eigenvalues *λ*_{1}=(*bq*−*c*)*A* and *λ*_{2}=0 with corresponding eigenvectors (*p*_{1},*p*_{0})=(1,0) and ((1+*h*)/2, −1). Provided *A*>0 (equivalently, *bq*>*c*), *λ*_{1}>0 follows, and the steady-state (1−*A*,0,*A*) is unstable along the boundary of the simplex (*p*_{0}=0). When the perturbation along the first eigenmode is small (Δ*p*_{1}0), the eigenmode with *λ*_{2}=0 matters for stability. When Δ*p*_{1}0, equation (A1) indicates that $\mathrm{\Delta}{\dot{p}}_{0}>0$ inside the simplex (Δ*p*_{0}>0). Therefore, the steady state is a degenerated source.

## References

- Axelrod R. Basic Books; New York, NY: 1984. Evolution of cooperation.
- Axelrod R, Hamilton W.D. The evolution of cooperation. Science. 1981;211:1390–1396. doi:10.1126/science.7466396 [PubMed]
- Bergstrom T.C. The algebra of assortative encounters and the evolution of cooperation. Int. Game Theory Rev. 2003;5:211–228. doi:10.1142/S0219198903001021
- Brandt H, Sigmund K. The logic of reprobation: assessment and action rules for indirect reciprocation. J. Theor. Biol. 2004;231:475–486. doi:10.1016/j.jtbi.2004.06.032 [PubMed]
- Brandt H, Sigmund K. Indirect reciprocity, image scoring, and moral hazard. Proc. Natl Acad. Sci. USA. 2005;102:2666–2670. doi:10.1073/pnas.0407370102 [PMC free article] [PubMed]
- Eshel I, Cavalli-Sforza L.L. Assortment of encounters and evolution of cooperativeness. Proc. Natl Acad. Sci. USA. 1982;79:1331–1335. doi:10.1073/pnas.79.4.1331 [PMC free article] [PubMed]
- Fehr E, Fischbacher U. The nature of human altruism. Nature. 2003;425:785–791. doi:10.1038/nature02043 [PubMed]
- Hamilton W.D. The genetical evolution of social behaviour I and II. J. Theor. Biol. 1964;7:1–16. doi:10.1016/0022-5193(64)90038-4 see also 17–52. [PubMed]
- Hammond R.A, Axelrod R. Evolution of contingent altruism when cooperation is expensive. Theor. Popul. Biol. 2006;69:333–338. doi:10.1016/j.tpb.2005.12.002 [PubMed]
- Hofbauer J, Sigmund K. Cambridge University Press; Cambridge, UK: 1998. Evolutionary games and population dynamics.
- Jansen V.A.A, van Baalen M. Altruism through beard chromodynamics. Nature. 2006;440:663–666. doi:10.1038/nature04387 [PubMed]
- Marsden P.V. Homogeneity in confiding relations. Soc. Netw. 1988;10:57–76. doi:10.1016/0378-8733(88)90010-X
- McPherson J.M, Smith-Lovin L. Homophily in voluntary organizations: status distance and the composition of face-to-face groups. Am. Sociol. Rev. 1987;52:370–379. doi:10.2307/2095356
- McPherson M, Smith-Lovin L, Cook J.M. Birds of a feather: homophily in social networks. Annu. Rev. Sociol. 2001;27:415–444. doi:10.1146/annurev.soc.27.1.415
- Newman M.E.J. Mixing patterns in networks. Phys. Rev. E. 2003;67:026126. doi:10.1103/PhysRevE.67.026126 [PubMed]
- Nowak M.A, May R.M. Evolutionary games and spatial chaos. Nature. 1992;359:826–829. doi:10.1038/359826a0
- Nowak M.A, Sigmund K. Evolution of indirect reciprocity by image scoring. Nature. 1998a;393:573–577. doi:10.1038/31225 [PubMed]
- Nowak M.A, Sigmund K. The dynamics of indirect reciprocity. J. Theor. Biol. 1998b;194:561–574. doi:10.1006/jtbi.1998.0775 [PubMed]
- Nowak M.A, Sigmund K. Evolution of indirect reciprocity. Nature. 2005;437:1291–1298. doi:10.1038/nature04131 [PubMed]
- Nowak M.A, Page K.M, Sigmund K. Fairness versus reason in the ultimatum game. Science. 2000;289:1773–1775. doi:10.1126/science.289.5485.1773 [PubMed]
- Ohtsuki H, Iwasa Y. How should we define goodness?—reputation dynamics in indirect reciprocity. J. Theor. Biol. 2004;231:107–120. doi:10.1016/j.jtbi.2004.06.005 [PubMed]
- Riolo R.L, Cohen M.D, Axelrod R. Evolution of cooperation without reciprocity. Nature. 2001;414:441–443. doi:10.1038/35106555 [PubMed]
- Roberts G, Sherratt T.N. Does similarity breed cooperation? Nature. 2002;418:499–500. doi:10.1038/418499b [PubMed]
- Taylor P.D, Jonker L.B. Evolutionary stable strategies and game dynamics. Math. Biosci. 1978;40:145–156. doi:10.1016/0025-5564(78)90077-9
- Traulsen A, Schuster H.G. Minimal model for tag-based cooperation. Phys. Rev. E. 2003;68:046129. doi:10.1103/PhysRevE.68.046129 [PubMed]
- Trivers R. The evolution of reciprocal altruism. Q. Rev. Biol. 1971;46:35–57. doi:10.1086/406755

**The Royal Society**

## Formats:

- Article |
- PubReader |
- ePub (beta) |
- PDF (332K)

- Upstream reciprocity in heterogeneous networks.[J Theor Biol. 2010]
*Iwagami A, Masuda N.**J Theor Biol. 2010 Aug 7; 265(3):297-305. Epub 2010 May 15.* - A tale of two defectors: the importance of standing for evolution of indirect reciprocity.[J Theor Biol. 2003]
*Panchanathan K, Boyd R.**J Theor Biol. 2003 Sep 7; 224(1):115-26.* - Evolution of indirect reciprocity by image scoring.[Nature. 1998]
*Nowak MA, Sigmund K.**Nature. 1998 Jun 11; 393(6685):573-7.* - The evolution of cooperation and altruism--a general framework and a classification of models.[J Evol Biol. 2006]
*Lehmann L, Keller L.**J Evol Biol. 2006 Sep; 19(5):1365-76.* - Evolution of indirect reciprocity.[Nature. 2005]
*Nowak MA, Sigmund K.**Nature. 2005 Oct 27; 437(7063):1291-8.*

- Powering up with indirect reciprocity in a large-scale field experiment[Proceedings of the National Academy of Scie...]
*Yoeli E, Hoffman M, Rand DG, Nowak MA.**Proceedings of the National Academy of Sciences of the United States of America. 2013 Jun 18; 110(Suppl 2)10424-10429* - Groupwise information sharing promotes ingroup favoritism in indirect reciprocity[BMC Evolutionary Biology. ]
*Nakamura M, Masuda N.**BMC Evolutionary Biology. 12213* - Evolution of in-group favoritism[Scientific Reports. ]
*Fu F, Tarnita CE, Christakis NA, Wang L, Rand DG, Nowak MA.**Scientific Reports. 2460* - Reputation-Based Conditional Interaction Supports Cooperation in Well-Mixed Prisoner's Dilemmas[PLoS ONE. ]
*Chen X, Schick A, Doebeli M, Blachford A, Wang L.**PLoS ONE. 7(5)e36260* - Moral assessment in indirect reciprocity[Journal of Theoretical Biology. 2012]
*Sigmund K.**Journal of Theoretical Biology. 2012 Apr 21; 299(5)25-30*