Lesion and neuroimaging studies have implicated the medial frontal lobes as playing an important role in our ability to predict other people's behavior by attributing to them mental states, such as beliefs, intention and emotion (termed "Theory of Mind"; ToM). However, recent studies have challenged these findings by highlighting the role of the temporal-parietal junction (TPJ) in ToM. In the present study, we tested the hypothesis that the ventromedial (VM) prefrontal cortex plays a unique role in affective ToM reasoning rather than a general role in ToM. We compared the performance of patients with lesions localized either in the VM, dorsolateral, TPJ, or superior parietal to healthy controls, with a battery of naturalistic affective and cognitive ToM stories (about false beliefs, false attribution, irony and lies). Patients with VM damage were impaired at providing appropriate mental state explanations for the affective ToM stories, compared to healthy controls and patients with posterior damage. In the VM group, performance in the affective ToM was significantly impaired as compared to cognitive ToM stories. Furthermore, in the VM group, ratings of levels of emotionality of each story suggested that levels of affective load correlated with number of errors in the stories, indicating that the more the emotional load involved in the story the greater the difficulty posed for the subjects in this group.