Heat shock protein 40s (Hsp40s) and heat shock protein 70s (Hsp70s) form chaperone partnerships that are key components of cellular chaperone networks involved in facilitating the correct folding of a broad range of client proteins. While the Hsp40 family of proteins is highly diverse with multiple forms occurring in any particular cell or compartment, all its members are characterized by a J domain that directs their interaction with a partner Hsp70. Specific Hsp40-Hsp70 chaperone partnerships have been identified that are dedicated to the correct folding of distinct subsets of client proteins. The elucidation of the mechanism by which these specific Hsp40-Hsp70 partnerships are formed will greatly enhance our understanding of the way in which chaperone pathways are integrated into finely regulated protein folding networks. From in silico analyses, domain swapping and rational protein engineering experiments, evidence has accumulated that indicates that J domains contain key specificity determinants. This review will critically discuss the current understanding of the structural features of J domains that determine the specificity of interaction between Hsp40 proteins and their partner Hsp70s. We also propose a model in which the J domain is able to integrate specificity and chaperone activity.