Theoretical accounts of grammatical limitations in specific language impairment (SLI) have been polarized between those that postulate problems with domain-specific grammatical knowledge, and those that regard grammatical deficits as downstream consequences of perceptual or memory limitations. Here we consider an alternative view that grammatical deficits arise when the learning system is biased towards memorization of exemplars, and is poor at extracting statistical dependencies from the input. We examine evidence that SLI involves deficits in extracting nonadjacent dependencies from input, leading to reliance on rote learning, and consider how far this may be part of a limitation of procedural learning, or a secondary consequence of memory limitations.