Four experiments examined attentional flexibility in listening to polyrhythmic patterns. Musically trained and untrained listeners detected changes in timing of 1 tone (the lower tone) in a 3:2 polyrhythm in which high and low tones varied in frequency separation. Experiment 1 encouraged integrative attending; all listeners performed significantly poorer in conditions with wide as opposed to narrow frequency separations. Experiment 2, which encouraged selective attending to low tones, reversed these results: Performance was poorer in the narrow frequency conditions. In neither experiment did skill interact with frequency separation. Experiments 3 and 4 extended these findings to moderate frequency separations. Over all experiments, musically trained listeners exhibited an enhanced ability to detect timing variations, but not flexibility of perceptual organization as it applies to detection of timing changes. Instead, pattern structure (e.g., frequency and time relation) decisively influenced perception for both levels of skill.