The prevalent view of binocular rivalry holds that it is a competition between the two eyes mediated by reciprocal inhibition among monocular neurons. This view is largely due to the nature of conventional rivalry-inducing stimuli, which are pairs of dissimilar images with coherent patterns within each eye's image. Is it the eye of origin or the coherency of patterns that determines perceptual alternations between coherent percepts in binocular rivalry? We break the coherency of conventional stimuli and replace them by complementary patchworks of intermingled rivalrous images. Can the brain unscramble the pieces of the patchwork arriving from different eyes to obtain coherent percepts? We find that pattern coherency in itself can drive perceptual alternations, and the patchworks are reassembled into coherent forms by most observers. This result is in agreement with recent neurophysiological and psychophysical evidence demonstrating that there is more to binocular rivalry than mere eye competition.