Although some evidence suggests that exercise may improve mental health status, information regarding the intensity and duration of exercise is incomplete. The purpose of this study was to elucidate the association between physical activities in both leisure time and commuting to work and mental health. A questionnaire survey was conducted at three municipal offices in Japan. A total of 670 men and women completed the questionnaire. Mental health status was assessed by the 28-item General Health Questionnaire (GHQ). Respondents were asked about weekly hours of leisure-time exercise for four levels of exercise, and the volume of exercise was assessed using a metabolic equivalent task index. Information about commuting to work on foot or by bicycle was also obtained. The mean GHQ scores were calculated according to physical activity levels using analysis of covariance with adjustment of potential confounders including job stress. In men, the GHQ score decreased steadily with increasing levels of leisure-time exercise, and an inverse association was evident even for mild intensity exercise. Moreover, the GHQ score decreased according to increasing duration of time on commuting to work by either walking or cycling in men, but not in women. These relations did not materially change after adjustment for potential confounding factors. In women, there was no significant association between any of the indices of physical activity. In conclusion, leisure-time exercise and walking or cycling during commuting to work may be associated with better mental health in men.