The present investigation sought to identify which brain regions distinguish pedophilic from nonpedophilic men, using unbiased, automated analyses of the whole brain. T1-weighted magnetic resonance images (MRIs) were acquired from men who demonstrated illegal or clinically significant sexual behaviors or interests (n = 65) and from men who had histories of nonsexual offenses but no sexual offenses (n = 62). Sexual interest in children was assessed by participants' admissions of pedophilic interest, histories of committing sexual offenses against children, and psychophysiological responses in the laboratory to erotic stimuli depicting children or adults. Automated parcellation of the MRIs revealed significant negative associations between pedophilia and white matter volumes of the temporal and parietal lobes bilaterally. Voxel-based morphometry corroborated the associations and indicated that the regions of lower white matter volumes followed, and were limited to, two major fiber bundles: the superior fronto-occipital fasciculus and the right arcuate fasciculus. No significant differences were found in grey matter or in cerebrospinal fluid (CSF). Because the superior fronto-occipital and arcuate fasciculi connect the cortical regions that respond to sexual cues, these results suggest (1) that those cortical regions operate as a network for recognizing sexually relevant stimuli and (2) that pedophilia results from a partial disconnection within that network.