Although degeneration of brain white matter (WM) in aging is a well-recognized problem, its quantification has mainly relied on volumetric measurements, which lack detail in describing the degenerative adaptation. In this study, WM structural complexity was evaluated in healthy old and young adults by analyzing the three-dimensional fractal dimension (FD) of WM segmented from magnetic resonance images of brain. FDs detected in the old were significantly smaller than in the young subjects. Specifically, WM interior structure complexity degenerated in the left hemisphere in old men but in the right hemisphere in old women. Men showed more complex WM patterns than women. An asymmetrical (right-greater-than-left-hemisphere) complexity pattern was observed in the interior and general structures of WM, yet the surface complexity was symmetrical across WM structures of the two hemispheres. WM volumes were also measured, but no significant decline was found with aging. These results suggest that the deterioration of WM complexity is not uniformly distributed between the genders and across brain hemispheres.