Men and women have different cognitive abilities that might reflect sex-specific neural organization. Here we studied sex effects on brain function using functional magnetic resonance imaging (fMRI) with variable acoustic noise (AN) to modulate the cognitive challenge and enhance the sensitivity for the detection of sex differences in brain activation. During the performance of a visual attention (VA) task that requires the tracking of multiple moving objects and has graded levels of difficulty, women (n=15) but not men (n=13) had shorter reaction times for "Loud" than for "Quiet" scans. Men activated more than women in the superior prefrontal and occipital cortices and the anterior thalamus. The latent connectivity of the prefrontal cortex was higher with the anterior thalamus but lower with the auditory cortex for men than for women. Increases in activation with visual attention load were larger for men than for women in the superior parietal and auditory cortices. Increased AN reduced brain activation in the parietal cortex and the anterior thalamus for men but not for women. Together, these sex-specific differences in brain activation during the VA task, at different cognitive and acoustic levels suggest differences in auditory gating of the thalamus for men and women.