Send to

Choose Destination
Proc Natl Acad Sci U S A. 2016 Jul 19;113(29):8002-9. doi: 10.1073/pnas.1601074113. Epub 2016 Jul 18.

The probability of monophyly of a sample of gene lineages on a species tree.

Author information

Department of Biology, Stanford University, Stanford, CA 94305;
Department of Mathematics and Statistics, University of Otago, Dunedin 9054, New Zealand.
Department of Biology, Stanford University, Stanford, CA 94305;


Monophyletic groups-groups that consist of all of the descendants of a most recent common ancestor-arise naturally as a consequence of descent processes that result in meaningful distinctions between organisms. Aspects of monophyly are therefore central to fields that examine and use genealogical descent. In particular, studies in conservation genetics, phylogeography, population genetics, species delimitation, and systematics can all make use of mathematical predictions under evolutionary models about features of monophyly. One important calculation, the probability that a set of gene lineages is monophyletic under a two-species neutral coalescent model, has been used in many studies. Here, we extend this calculation for a species tree model that contains arbitrarily many species. We study the effects of species tree topology and branch lengths on the monophyly probability. These analyses reveal new behavior, including the maintenance of nontrivial monophyly probabilities for gene lineage samples that span multiple species and even for lineages that do not derive from a monophyletic species group. We illustrate the mathematical results using an example application to data from maize and teosinte.


coalescent theory; monophyly; phylogeography

[Indexed for MEDLINE]
Free PMC Article

Supplemental Content

Full text links

Icon for HighWire Icon for PubMed Central
Loading ...
Support Center