Autism and Williams syndrome are genetically based neurodevelopmental disorders that present strikingly different social phenotypes. Autism involves fundamental impairments in social reciprocity and communication, whereas people with Williams syndrome are highly sociable and engaging. This article reviews the behavioral and neuroimaging literature that has explored the neurocognitive mechanisms that underlie these contrasting social phenotypes, focusing on studies of face processing. The article concludes with a discussion of how the social phenotypes of both syndromes may be characterized by impaired connectivity between the amygdala and other critical regions in the 'social brain'.