Send to

Choose Destination
Ecol Evol. 2016 May 21;6(12):4115-28. doi: 10.1002/ece3.2154. eCollection 2016 Jun.

Using simulations to evaluate Mantel-based methods for assessing landscape resistance to gene flow.

Author information

Department of Environmental Conservation University of Massachusetts Amherst Massachusetts 01003.
Department of Fisheries and Wildlife Oregon State University Corvallis Oregon 97331.
Department of Biology McGill University Montreal Quebec H3A 1B1 Canada.
Field Conservation Program Woodland Park Zoo Seattle Washington 98103.
Department of Ecology and Evolutionary Biology University of Toronto Mississauga Ontario L5L 1C6 Canada.
Department of Wildlife Sciences University of Göttingen Büsgenweg 3 37077 Göttingen Germany.
Division of Biological Sciences University of Montana Missoula Montana 59846.


Mantel-based tests have been the primary analytical methods for understanding how landscape features influence observed spatial genetic structure. Simulation studies examining Mantel-based approaches have highlighted major challenges associated with the use of such tests and fueled debate on when the Mantel test is appropriate for landscape genetics studies. We aim to provide some clarity in this debate using spatially explicit, individual-based, genetic simulations to examine the effects of the following on the performance of Mantel-based methods: (1) landscape configuration, (2) spatial genetic nonequilibrium, (3) nonlinear relationships between genetic and cost distances, and (4) correlation among cost distances derived from competing resistance models. Under most conditions, Mantel-based methods performed poorly. Causal modeling identified the true model only 22% of the time. Using relative support and simple Mantel r values boosted performance to approximately 50%. Across all methods, performance increased when landscapes were more fragmented, spatial genetic equilibrium was reached, and the relationship between cost distance and genetic distance was linearized. Performance depended on cost distance correlations among resistance models rather than cell-wise resistance correlations. Given these results, we suggest that the use of Mantel tests with linearized relationships is appropriate for discriminating among resistance models that have cost distance correlations <0.85 with each other for causal modeling, or <0.95 for relative support or simple Mantel r. Because most alternative parameterizations of resistance for the same landscape variable will result in highly correlated cost distances, the use of Mantel test-based methods to fine-tune resistance values will often not be effective.


CDPOP; landscape fragmentation; landscape genetics; landscape resistance; simulations

Supplemental Content

Full text links

Icon for Wiley Icon for PubMed Central
Loading ...
Support Center