Numerous studies of ecological genetics have found that alleles contributing to local adaptation sometimes cluster together, forming "genomic islands of divergence." Divergence hitchhiking theory posits that these clusters evolve by the preferential establishment of tightly linked locally adapted mutations, because such linkage reduces the rate that recombination breaks up locally favorable combinations of alleles. Here, I use calculations based on previously developed analytical models of divergence hitchhiking to show that very few clustered mutations should be expected in a single bout of adaptation, relative to the number of unlinked mutations, suggesting that divergence hitchhiking theory alone may often be insufficient to explain empirical observations. Using individual-based simulations that allow for the transposition of a single genetic locus from one position on a chromosome to another, I then show that tight clustering of the loci involved in local adaptation tends to evolve on biologically realistic time scales. These results suggest that genomic rearrangements may often be an important component of local adaptation and the evolution of genomic islands of divergence. More generally, these results suggest that genomic architecture and functional neighborhoods of genes may be actively shaped by natural selection in heterogeneous environments. Because small-scale changes in gene order are relatively common in some taxa, comparative genomic studies could be coupled with studies of adaptation to explore how commonly such rearrangements are involved in local adaptation.