This study investigated the effects of irrelevant sounds on the serial recall performance of visually presented digits in a sample of children diagnosed with (central) auditory processing disorders [(C)APD] and age- and span-matched control groups. The irrelevant sounds used were samples of tones and speech. Memory performance was significantly disrupted in the presence of irrelevant sounds in all three groups of children. While irrelevant speech was more disruptive than irrelevant tones in the two control groups, children diagnosed with (C)APD did not show larger disruption from irrelevant speech compared to irrelevant tones. Children diagnosed with (C)APD appear to process speech differently from their typically developing peers, and this may be remediated with auditory training procedures and the placement of these children in smaller classes.