We model selection at a locus affecting a quantitative trait (QTL) in the presence of genetic variance due to other loci. The dynamics at the QTL are related to the initial genotypic value and to the background genetic variance of the trait, assuming that background genetic values are normally distributed, under three different forms of selection on the trait. Approximate dynamics are derived under the assumption of small mutation effect. For similar strengths of selection on the trait (i.e, gradient of directional selection beta) the way background variation affects the dynamics at the QTL critically depends on the shape of the fitness function. It generally causes the strength of selection on the QTL to decrease with time. The resulting neutral heterozygosity pattern resembles that of a selective sweep with a constant selection coefficient corresponding to the early conditions. The signature of selection may also be blurred by mutation and recombination in the later part of the sweep. We also study the race between the QTL and its genetic background toward a new optimum and find the conditions for a complete sweep. Overall, our results suggest that phenotypic traits exhibiting clear-cut molecular signatures of selection may represent a biased subset of all adaptive traits.