Synesthesia, a neurological condition affecting between 0.05%-1% of the population, is characterized by anomalous sensory perception and associated alterations in cognitive function due to interference from synesthetic percepts. A stimulus in one sensory modality triggers an automatic, consistent response in either another modality or a different aspect of the same modality. Familiality studies show evidence of a strong genetic predisposition; whereas initial pedigree analyses supported a single-gene X-linked dominant mode of inheritance with a skewed F:M ratio and a notable absence of male-to-male transmission, subsequent analyses in larger samples indicated that the mode of inheritance was likely to be more complex. Here, we report the results of a whole-genome linkage scan for auditory-visual synesthesia with 410 microsatellite markers at 9.05 cM density in 43 multiplex families (n = 196) with potential candidate regions fine-mapped at 5 cM density. Using NPL and HLOD analysis, we identified four candidate regions. Significant linkage at the genome-wide level was detected to chromosome 2q24 (HLOD = 3.025, empirical genome-wide p = 0.047). Suggestive linkage was found to chromosomes 5q33, 6p12, and 12p12. No support was found for linkage to the X chromosome; furthermore, we have identified two confirmed cases of male-to-male transmission of synesthesia. Our results demonstrate that auditory-visual synesthesia is likely to be an oligogenic disorder subject to multiple modes of inheritance and locus heterogeneity. This study comprises a significant step toward identifying the genetic substrates underlying synesthesia, with important implications for our understanding of the role of genes in human cognition and perception.