To determine whether population differentiation in flowering time is consistent with differences in current selection, we quantified phenotypic selection acting through female reproductive success on flowering phenology and floral display in two Scandinavian populations of the outcrossing, perennial herb Arabidopsis lyrata in two years. One population was located in an alpine environment strongly affected by grazing, whereas the other was close to sea level and only moderately affected by herbivory. Multiple regression models indicated directional selection for early end of flowering in one year in the lowland population, and directional selection for early start of flowering in one year in the alpine population. As expected, there was selection for more inflorescences in the lowland population. However, in the alpine population, plants with many inflorescences were selectively grazed and the number of inflorescences produced was negatively related to female fitness in one year and not significantly related to female fitness in the second year. The results are consistent with the hypothesis that genetic differentiation in flowering phenology between the study populations is adaptive, and indicate that interactions with selective grazers may strongly influence selection on floral display in A. lyrata.