The invention of agriculture is widely assumed to have driven recent human population growth. However, direct genetic evidence for population growth after independent agricultural origins has been elusive. We estimated population sizes through time from a set of globally distributed whole mitochondrial genomes, after separating lineages associated with agricultural populations from those associated with hunter-gatherers. The coalescent-based analysis revealed strong evidence for distinct demographic expansions in Europe, southeastern Asia, and sub-Saharan Africa within the past 10,000 y. Estimates of the timing of population growth based on genetic data correspond neatly to dates for the initial origins of agriculture derived from archaeological evidence. Comparisons of rates of population growth through time reveal that the invention of agriculture facilitated a fivefold increase in population growth relative to more ancient expansions of hunter-gatherers.