Nervous systems adapt to the prevailing sensory environment, and the consequences of this adaptation can be observed in the responses of single neurons and in perception. Given the variety of timescales underlying events in the natural world, determining the temporal characteristics of adaptation is important to understanding how perception adjusts to its sensory environment. Previous work has shown that neural adaptation can occur on a timescale of milliseconds, but perceptual adaptation has generally been studied over relatively long timescales, typically on the order of seconds. This disparity raises important questions. Can perceptual adaptation be observed at brief, functionally relevant timescales? And if so, how do its properties relate to the rapid adaptation seen in cortical neurons? We address these questions in the context of visual motion processing, a perceptual modality characterized by rapid temporal dynamics. We demonstrate objectively that 25 ms of motion adaptation is sufficient to generate a motion aftereffect, an illusory sensation of movement experienced when a moving stimulus is replaced by a stationary pattern. This rapid adaptation occurs regardless of whether the adapting motion is perceived. In neurophysiological recordings from the middle temporal area of primate visual cortex, we find that brief motion adaptation evokes direction-selective responses to subsequently presented stationary stimuli. A simple model shows that these neural responses can explain the consequences of rapid perceptual adaptation. Overall, we show that the motion aftereffect is not merely an intriguing perceptual illusion, but rather a reflection of rapid neural and perceptual processes that can occur essentially every time we experience motion.