Recent research has revealed that the presence of irrelevant visual information during retrieval of long-term memories diminishes recollection of task-relevant visual details. Here, we explored the impact of irrelevant auditory information on remembering task-relevant visual details by probing recall of the same previously viewed images while participants were in complete silence, exposed to white noise, or exposed to ambient sounds recorded at a busy café. The presence of auditory distraction diminished objective recollection of goal-relevant details, relative to the silence and white noise conditions. Critically, a comparison with results from a previous study using visual distractors showed equivalent effects for auditory and visual distraction. These findings suggest that disruption of recollection by external stimuli is a domain-general phenomenon produced by interference between resource-limited, top-down mechanisms that guide the selection of mnemonic details and control processes that mediate our interactions with external distractors.