A central and classic question in ecology is what causes populations to fluctuate in abundance. Understanding the interaction between natural drivers of fluctuating populations and human exploitation is an issue of paramount importance for conservation and natural resource management. Three main hypotheses have been proposed to explain fluctuations: (i) species interactions, such as predator-prey interactions, cause fluctuations, (ii) strongly nonlinear single-species dynamics cause fluctuations, and (iii) environmental variation cause fluctuations. We combine a general fisheries model with data from a global sample of fish species to assess how two of these hypothesis, nonlinear single-species dynamics and environmental variation, interact with human exploitation to affect the variability of fish populations. In contrast with recent analyses that suggest fishing drives increased fluctuations by changing intrinsic nonlinear dynamics, we show that single-species nonlinear dynamics alone, both in the presence and absence of fisheries, are unlikely to drive deterministic fluctuations in fish; nearly all fish populations fall into regions of stable dynamics. However, adding environmental variation dramatically alters the consequences of exploitation on the temporal variability of populations. In a variable environment, (i) the addition of mortality from fishing leads to increased temporal variability for all species examined, (ii) variability in recruitment rates of juveniles contributes substantially more to fluctuations than variation in adult mortality, and (iii) the correlation structure of juvenile and adult vital rates plays an important and underappreciated role in determining population fluctuations. Our results are robust to alternative model formulations and to a range of environmental autocorrelation.