To determine the relationship between age and gray matter structure and how interactions between gender and hemisphere impact this relationship, we examined correlations between global or regional gray matter volume and age, including interactions of gender and hemisphere, using a general linear model with voxel-based and region-of-interest analyses. Brain magnetic resonance images were collected from 1460 healthy individuals aged 20-69 years; the images were linearly normalized and segmented and restored to native space for analysis of global gray matter volume. Linearly normalized images were then non-linearly normalized and smoothed for analysis of regional gray matter volume. Analysis of global gray matter volume revealed a significant negative correlation between gray matter ratio (gray matter volume divided by intracranial volume) and age in both genders, and a significant interaction effect of age × gender on the gray matter ratio. In analyzing regional gray matter volume, the gray matter volume of all regions showed significant main effects of age, and most regions, with the exception of several including the inferior parietal lobule, showed a significant age × gender interaction. Additionally, the inferior temporal gyrus showed a significant age × gender × hemisphere interaction. No regional volumes showed significant age × hemisphere interactions. Our study may contribute to clarifying the mechanism(s) of normal brain aging in each brain region.