This study was designed to investigate how emotion category, characterized by distinct musical structures (happiness, sadness, threat) and expressiveness (mechanical, expressive) may influence overt and covert behavioral judgments and physiological responses in musically trained and untrained listeners. Mechanical and expressive versions of happy, sad and scary excerpts were presented while physiological measures were recorded. Participants rated the intensity of the emotion they felt. In addition, they monitored excerpts for the presence of brief breaths. Results showed that the emotion categories were rated higher in the expressive than in the mechanical versions and that this effect was larger in musicians. Moreover, expressive excerpts were found to increase skin conductance level more than the mechanical ones, independently of their arousal value, and to slow down response times in the breath detection task relative to the mechanical versions, suggesting enhanced capture of attention by expressiveness. Altogether, the results support the key role of the performer's expression in the listener's emotional response to music.