Double chromodomains occur in CHD proteins, which are ATP-dependent chromatin remodeling factors implicated in RNA polymerase II transcription regulation. Biochemical studies suggest important differences in the histone H3 tail binding of different CHD chromodomains. In human and Drosophila, CHD1 double chromodomains bind lysine 4-methylated histone H3 tail, which is a hallmark of transcriptionally active chromatin in all eukaryotes. Here, we present the crystal structure of the yeast CHD1 double chromodomains, and pinpoint their differences with that of the human CHD1 double chromodomains. The most conserved residues in these double chromodomains are the two chromoboxes that orient adjacently. Only a subset of CHD chromoboxes can form an aromatic cage for methyllysine binding, and methyllysine binding requires correctly oriented inserts. These factors preclude yeast CHD1 double chromodomains from interacting with the histone H3 tail. Despite great sequence similarity between the human CHD1 and CHD2 chromodomains, variation within an insert likely prevents CHD2 double chromodomains from binding lysine 4-methylated histone H3 tail as efficiently as in CHD1. By using the available structural and biochemical data we highlight the evolutionary specialization of CHD double chromodomains, and provide insights about their targeting capacities.