Predicting structure from the attributes of a material's building blocks remains a challenge and central goal for materials science. Isolating the role of building block shape for self-assembly provides insight into the ordering of molecules and the crystallization of colloids, nanoparticles, proteins, and viruses. We investigated 145 convex polyhedra whose assembly arises solely from their anisotropic shape. Our results demonstrate a remarkably high propensity for thermodynamic self-assembly and structural diversity. We show that from simple measures of particle shape and local order in the fluid, the assembly of a given shape into a liquid crystal, plastic crystal, or crystal can be predicted.