We performed a comprehensive cognitive, neuroimaging, and genetic study of 31 patients with primary progressive aphasia (PPA), a decline in language functions that remains isolated for at least 2 years. Detailed speech and language evaluation was used to identify three different clinical variants: nonfluent progressive aphasia (NFPA; n = 11), semantic dementia (SD; n = 10), and a third variant termed logopenic progressive aphasia (LPA; n = 10). Voxel-based morphometry (VBM) on MRIs showed that, when all 31 PPA patients were analyzed together, the left perisylvian region and the anterior temporal lobes were atrophied. However, when each clinical variant was considered separately, distinctive patterns emerged: (1) NFPA, characterized by apraxia of speech and deficits in processing complex syntax, was associated with left inferior frontal and insular atrophy; (2) SD, characterized by fluent speech and semantic memory deficits, was associated with anterior temporal damage; and (3) LPA, characterized by slow speech and impaired syntactic comprehension and naming, showed atrophy in the left posterior temporal cortex and inferior parietal lobule. Apolipoprotein E epsilon4 haplotype frequency was 20% in NFPA, 0% in SD, and 67% in LPA. Cognitive, genetic, and anatomical features indicate that different PPA clinical variants may correspond to different underlying pathological processes.