Gender differences are an important factor regulating our daily interactions. Using functional magnetic resonance imaging we show that brain areas involved in processing social signals are activated differently by threatening signals send from male and female facial and bodily expressions and that their activation patterns are different for male and female observers. Male participants pay more attention to the female face as shown by increased amygdala activity. But a host of other areas show selective sensitivity for male observers attending to male threatening bodily expressions (extrastriate body area, superior temporal sulcus, fusiform gyrus, pre-supplementary motor area, and premotor cortex). This is the first study investigating gender differences in processing dynamic female and male facial and bodily expressions and it illustrates the importance of gender differences in affective communication.