Atlas construction generally includes first an image registration step to normalize all images into a common space and then an atlas building step to fuse all the aligned images. Although numerous atlas construction studies have been performed to improve the accuracy of image registration step, simple averaging or weighted averaging is often used for the atlas building step. In this paper, we propose a novel patch-based sparse representation method for atlas construction, especially for the atlas building step. By taking advantage of local sparse representation, more distinct anatomical details can be revealed in the built atlas. Also, together with the constraint on group structure of representations and the use of overlapping patches, anatomical consistency between neighboring patches can be ensured. The proposed method has been applied to 73 neonatal MR images with poor spatial resolution and low tissue contrast, for building unbiased neonatal brain atlas. Experimental results demonstrate that the proposed method can enhance the quality of built atlas by discovering more anatomical details especially in cortical regions, and perform better in a neonatal data normalization application, compared to other existing start-of-the-art nonlinear neonatal brain atlases.