Many structural properties of conventional passive materials are known to arise from the symmetries of their microscopic constituents. By contrast, it is largely unclear how the interplay between particle shape and self-propulsion controls the meso- and macroscale behavior of active matter. Here we use large-scale simulations of homo- and heterogeneous self-propelled particle systems to identify generic effects of broken particle-shape symmetry on collective motion. We find that even small violations of fore-aft symmetry lead to fundamentally different collective behaviors, which may facilitate demixing of differently shaped species as well as the spontaneous formation of stable microrotors. These results suggest that variation of particle shape yields robust physical mechanisms to control self-assembly of active matter, with possibly profound implications for biology and materials design.