Brown fat can increase energy expenditure and protect against obesity through a specialized program of uncoupled respiration. Here we show by in vivo fate mapping that brown, but not white, fat cells arise from precursors that express Myf5, a gene previously thought to be expressed only in the myogenic lineage. We also demonstrate that the transcriptional regulator PRDM16 (PRD1-BF1-RIZ1 homologous domain containing 16) controls a bidirectional cell fate switch between skeletal myoblasts and brown fat cells. Loss of PRDM16 from brown fat precursors causes a loss of brown fat characteristics and promotes muscle differentiation. Conversely, ectopic expression of PRDM16 in myoblasts induces their differentiation into brown fat cells. PRDM16 stimulates brown adipogenesis by binding to PPAR-gamma (peroxisome-proliferator-activated receptor-gamma) and activating its transcriptional function. Finally, Prdm16-deficient brown fat displays an abnormal morphology, reduced thermogenic gene expression and elevated expression of muscle-specific genes. Taken together, these data indicate that PRDM16 specifies the brown fat lineage from a progenitor that expresses myoblast markers and is not involved in white adipogenesis.