Everyday language is replete with descriptions of emotional events that people have experienced and wish to share with others. Such descriptions presumably rely on pairings of affective words and visual information (such as events and pictures) that have been learnt throughout one's development. To study this kind of affective language learning in the brain, we used functional neuroimaging during associative learning of emotional words and pictures. Brain imaging revealed increased activation of both primary emotional areas such as the amygdala and of higher cognitive areas such as the inferior frontal gyrus (IFG) and medial frontal gyrus. The dynamic causal modeling with Bayesian model selection suggested that the IFG first receives the input and that the connections are bidirectional, suggesting that during such emotional picture-word pair learning, the frontal cortex drives the amygdala activation. Specifically, the interaction between the frontal regions and the amygdala was enhanced by active learning involving both negative and positive emotional stimuli as compared with neutral stimuli. This circuit (especially for negative stimuli) converges with emotion regulation circuits. The enhancement in the connectivity might be responsible for the emotional memory effect in this type of learning.