In humans, horizontal sound localization of low-frequency sounds is mainly based on interaural time differences (ITDs). Traditionally, it was assumed that ITDs are converted into a topographic (or rate-place) code, supported by an array of neurons with parametric tuning to ITDs within the behaviorally relevant range. Although this topographic model has been confirmed in owls, its applicability to mammals has been challenged by recent physiological results suggesting that, at least in small-headed species, ITDs are represented by a nontopographic population rate code, which involves only two opponent (left and right) channels, broadly tuned to ITDs from the two auditory hemifields. The current study investigates which of these two models of ITD processing is more likely to apply to humans. For that, evoked responses to abrupt changes in the ITDs of otherwise continuous sounds were measured with electroencephalography. The ITD change was either away from ("outward" change) or toward the midline ("inward" change). According to the opponent-channel model, the response to an outward ITD change should be larger than the response to the corresponding inward change, whereas the topographic model would predict similar response sizes for both conditions. The measured response sizes were highly consistent with the predictions of the opponent-channel model and contravened the predictions of the topographic model, suggesting that, in humans, ITDs are coded nontopographically. The hemispheric distributions of the ITD change responses suggest that the majority of ITD-sensitive neurons in each hemisphere are tuned to ITDs from the contralateral hemifield.