A crucial question in cognitive science is how linguistic and visual information are integrated. Previous research has shown that eye movements to objects in the visual environment are locked to linguistic input. More surprisingly, listeners fixate on now-empty regions that had previously been occupied by relevant objects. This 'looking at nothing' phenomenon has been linked to the claim that the visual system constructs sparse representations of the external world and relies on saccades and fixations to extract information in a just-in-time manner. Our model provides a different explanation: based on recent work in visual cognition and memory, it assumes that the visual system creates and stores detailed internal memory representations, and that looking at nothing facilitates retrieval of those representations.