Without any mechanism for the evolution of cooperation, natural selection favors defectors. In a mixed population defectors, D, have a higher payoff (=fitness) than cooperators, C. Therefore, natural selection continuously reduces the abundance, i, of cooperators until they are extinct. The average fitness of the population also declines under natural selection. The total population size is given by N. There are i cooperators and N − i defectors. The fitness of cooperators and defectors is respectively given by f_C = b(i − 1)/(N − 1) − c and f_D = bi/(N − 1). The average fitness of the population is given by f̅ = (b − c)i/N.

Evolutionary dynamics of cooperators and defectors. The red and blue arrows indicate selection favoring defectors and cooperators, respectively. (a) Without any mechanism for the evolution of cooperation, defectors dominate. A mechanism for evolution of cooperation can allow cooperators to be the evolutionarily stable strategy (ESS), risk dominant (RD) or advantageous (AD) in comparison with defectors. (b) Cooperators are ESS if they can resist invasion by defectors. (c) Cooperators are RD if the basin of attraction of defectors is less than 1/2. (d) Cooperators are AD if the basin of attraction of defectors is less that 1/3. In this case, the fixation probability of a single cooperator in a finite population of defectors is greater than the inverse of the population size (for weak selection). (e) Some mechanisms allow cooperators to dominate defectors.

Five mechanisms for the evolution of cooperation. Kin selection operates when the donor and the recipient of an altruistic act are genetic relatives. Direct reciprocity requires repeated encounters between the same two individuals. Indirect reciprocity is based on reputation; a helpful individual is more likely to receive help. Network reciprocity means that clusters of cooperators outcompete defectors. Group selection is the idea that competition is not only between individuals but also between groups.