We examined the effect of age on working memory with a reading span task (RST) together with other verbal span tasks. Sixty-two participants were divided into three subgroups (young, middle-aged, and elderly). The RST performances were significantly different among all the subgroups. To elucidate which component of the working memory system is affected by age, we performed an analysis of covariance with the scores of simple and complex verbal span tasks as covariates. From the results, we conclude that the difference of the RST performance between the middle-aged and elderly groups reflects a decline in the capacity of the phonological loop, and the difference between the young and middle-aged groups reflects malfunctioning of the central executive system.