The semantic variant of primary progressive aphasia (PPA) is characterized by the combination of word comprehension deficits, fluent aphasia and a particularly severe anomia. In this study, two novel tasks were used to explore the factors contributing to the anomia. The single most common factor was a blurring of distinctions among members of a semantic category, leading to errors of overgeneralization in word-object matching tasks as well as in word definitions and object descriptions. This factor was more pronounced for natural kinds than artifacts. In patients with the more severe anomias, conceptual maps were more extensively disrupted so that inter-category distinctions were as impaired as intra-category distinctions. Many objects that could not be named aloud could be matched to the correct word in patients with mild but not severe anomia, reflecting a gradual intensification of the semantic factor as the naming disorder becomes more severe. Accurate object descriptions were more frequent than accurate word definitions and all patients experienced prominent word comprehension deficits that interfered with everyday activities but no consequential impairment of object usage or face recognition. Magnetic resonance imaging revealed three characteristics: greater atrophy of the left hemisphere; atrophy of anterior components of the perisylvian language network in the superior and middle temporal gyri; and atrophy of anterior components of the face and object recognition network in the inferior and medial temporal lobes. The left sided asymmetry and perisylvian extension of the atrophy explains the more profound impairment of word than object usage and provides the anatomical basis for distinguishing the semantic variant of primary progressive aphasia from the partially overlapping group of patients that fulfil the widely accepted diagnostic criteria for semantic dementia.