Traits that are attractive to the opposite sex are often positively correlated when scaled such that scores increase with attractiveness, and this correlation typically has a genetic component. Such traits can be genetically correlated due to genes that affect both traits ("pleiotropy") and/or because assortative mating causes statistical correlations to develop between selected alleles across the traits ("gametic phase disequilibrium"). In this study, we modeled the covariation between monozygotic and dizygotic twins, their siblings, and their parents (total N = 7,905) to elucidate the nature of the correlation between two potentially sexually selected traits in humans: height and IQ. Unlike previous designs used to investigate the nature of the height-IQ correlation, the present design accounts for the effects of assortative mating and provides much less biased estimates of additive genetic, non-additive genetic, and shared environmental influences. Both traits were highly heritable, although there was greater evidence for non-additive genetic effects in males. After accounting for assortative mating, the correlation between height and IQ was found to be almost entirely genetic in nature. Model fits indicate that both pleiotropy and assortative mating contribute significantly and about equally to this genetic correlation.