Histone H3 lysine 4 (H3K4) methylation is catalyzed by the highly evolutionarily conserved multiprotein complex known as Set1/COMPASS or MLL/COMPASS-like complexes from yeast to human, respectively. Here we have reconstituted fully functional yeast Set1/COMPASS and human MLL/COMPASS-like complex in vitro and have identified the minimum subunit composition required for histone H3K4 methylation. These subunits include the methyltransferase C-terminal SET domain of Set1/MLL, Cps60/Ash2L, Cps50/RbBP5, Cps30/WDR5, and Cps25/Dpy30, which are all common components of the COMPASS family from yeast to human. Three-dimensional (3D) cryo-EM reconstructions of the core yeast complex, combined with immunolabeling and two-dimensional (2D) EM analysis of the individual subcomplexes reveal a Y-shaped architecture with Cps50 and Cps30 localizing on the top two adjacent lobes and Cps60-Cps25 forming the base at the bottom. EM analysis of the human complex reveals a striking similarity to its yeast counterpart, suggesting a common subunit organization. The SET domain of Set1 is located at the juncture of Cps50, Cps30, and the Cps60-Cps25 module, lining the walls of a central channel that may act as the platform for catalysis and regulative processing of various degrees of H3K4 methylation. This structural arrangement suggested that COMPASS family members function as exo-methylases, which we have confirmed by in vitro and in vivo studies.