The efficient and robust extraction of invariant patterns from an image is a long-standing problem in computer vision. Invariant structures are often related to repetitive or near-repetitive patterns. The perception of repetitive patterns in an image is strongly linked to the visual interpretation and composition of textures. Repetitive patterns are products of both repetitive structures as well as repetitive reflections or color patterns. In other words, patterns that exhibit near-stationary behavior provide rich information about objects, their shapes, and their texture in an image. In this paper, we propose a new algorithm for repetitive pattern detection and grouping. The algorithm follows the classical region growing image segmentation scheme. It utilizes a mean-shift-like dynamic to group local image patches into clusters. It exploits a continuous joint alignment to: 1) match similar patches, and 2) refine the subspace grouping. We also propose an algorithm for inferring the composition structure of the repetitive patterns. The inference algorithm constructs a data-driven structural completion field, which merges the detected repetitive patterns into specific global geometric structures. The result of higher level grouping for image patterns can be used to infer the geometry of objects and estimate the general layout of a crowded scene.