The purpose of the present study was to compare the responses of 27 incarcerated rapists and 27 incarcerated nonsexual offenders using the Rapist Empathy Measure (targeting victim specific empathy deficits) and to examine the relationship between empathy with self-esteem and psychopathy for both groups. The Social Self-Esteem Inventory was used as a measure of perceived social competence and the Psychopathy Checklist-Revised (PCL-R; Hare, 1991) was used as a measure of psychopathy. All participants completed the two self-report questionnaires on empathy and self-esteem; in addition, the rapists were required to complete an extra section of the empathy measure that assessed their empathic responses to their own victims. Demographic information and psychopathy scores were obtained by reviewing institutional files. When psychopathy scores were not available, subjects participated in a semi-structured interview and were scored on the Psychopathy Checklist-Revised by the researcher. Rapists demonstrated more empathy than the nonsexual offenders toward women in general and the same degree of empathy as the nonsexual offenders toward a woman who had been a victim of a sexual assault by another male. Of particular importance were the within-group comparisons across victim type for the rapists which revealed significant empathy deficits toward their own victim(s). Interestingly, no differences were found between the rapists and nonsexual offenders in terms of self-esteem and psychopathy, and neither self-esteem nor psychopathy significantly predicted empathy for either group. It was concluded from the present study that rapists may suppress empathy primarily toward their own victim rather than suffer from a generalized empathy deficit. It is suggested that empathy deficits in rapists might better be construed as cognitive distortions specific to their victims and should be addressed in that manner in treatment.