The audiometric and self-assessed handicap data of 180 elderly persons with presbyacusic-type hearing impairment were submitted to principal components analysis in order to study the factor structure underlying auditory processing. Audiometric data included pure-tone threshold sensitivity, speech understanding scores for monosyllabic words without competition, synthetic sentences within competition, and dichotic synthetic sentence identification. The Hearing Handicap Inventory for the Elderly (HHIE) was used to quantify self-assessed handicap both by the subjects and by their significant others. Six factors emerged from the analysis: (1) low-frequency sensitivity loss, (2) high-frequency sensitivity loss, (3) general speech understanding ability, (4) self-assessed handicap, (5) the central processing of left ear input, and (6) the central processing of right ear input. Five conclusions are highlighted: (1) degree of hearing sensitivity loss, which determines speech audibility, remains a key concept in understanding the speech understanding problems of elderly persons; (2) low-frequency and high-frequency sensitivity loss emerge as separate factors; (3) a factor of general speech understanding ability, not strongly related to audibility, is confirmed; (4) the central processing of right and left ear verbal inputs emerge as separate factors; and (5) perceived auditory handicap emerges as a separate factor, apparently not strongly related to either audibility, general speech understanding, or the central processing of verbal input.