The crucial roles of Sec1/Munc18 (SM)-like proteins in membrane fusion have been evidenced in genetic and biochemical studies. SM proteins interact directly with SNAREs and contribute to SNARE pairing by a yet unclear mechanism. Here, we show that the SM protein, Sly1, interacts directly with the conserved oligomeric Golgi (COG) tethering complex. The Sly1-COG interaction is mediated by the Cog4 subunit, which also interacts with Syntaxin 5 through a different binding site. We provide evidence that disruption of Cog4-Sly1 interaction impairs pairing of SNAREs involved in intra-Golgi transport thereby markedly attenuating Golgi-to-ER retrograde transport. These results highlight the mechanism by which SM proteins link tethering to SNAREpin assembly.