Sex hormones are essential for neural circuit development and sex-specific behaviors. Male behaviors require both testosterone and estrogen, but it is unclear how the two hormonal pathways intersect. Circulating testosterone activates the androgen receptor (AR) and is also converted into estrogen in the brain via aromatase. We demonstrate extensive sexual dimorphism in the number and projections of aromatase-expressing neurons. The masculinization of these cells is independent of AR but can be induced in females by either testosterone or estrogen, indicating a role for aromatase in sexual differentiation of these neurons. We provide evidence suggesting that aromatase is also important in activating male-specific aggression and urine marking because these behaviors can be elicited by testosterone in males mutant for AR and in females subjected to neonatal estrogen exposure. Our results suggest that aromatization of testosterone into estrogen is important for the development and activation of neural circuits that control male territorial behaviors.