In the last decades, the rapid spread of diseases, such as varroosis and nosemosis, associated with massive honey bee colonies mortality around the world has significantly decreased the number and size of honey bee populations and possibly their genetic diversity. Here, we compare the genetic diversity of Iberian honey bee colonies in two samplings performed in 2006 and 2010 in relation to the presence of the pathogenic agents Nosema apis, Nosema ceranae, and Varroa destructor in order to determine whether parasite and pathogen spread in honey bee colonies reflects changes in genetic diversity. We found that the genetic diversity remained similar, while the incidence of N. ceranae increased and the incidence of N. apis and V. destructor decreased slightly. These results indicate that the genetic diversity was not affected by the presence of these pathogenic agents in the analyzed period. However, the two groups of colonies with and without Nosema/Varroa detected showed significant genetic differentiation (G test). A detailed analysis of the allelic segregation of microsatellite loci in Nosema/Varroa-negative colonies and parasitized ones revealed two outlier loci related to genes involved in immune response.