We report a correlation between the dielectric property and structure of stretched poly(lactic acid) (PLA) films, revealed by polarization-sensitive terahertz time-domain spectroscopy and two-dimensional (2D) wide-angle X-ray scattering (WAXS). The experiments evidence that the dielectric function of the PLA film becomes more anisotropic with increasing draw ratio (DR). This behavior is explained by a classical Lorentz oscillator model assuming polarization-dependent absorption. The birefringence can be systematically altered from 0 to 0.13 by controlling DR. The combination of terahertz spectroscopy and 2D WAXS measurement reveals a clear correlation between the birefringence in the terahertz frequency domain and the degree of orientation of the PLA molecular chains. These findings imply that the birefringence is a result of the orientation of the PLA chains with anisotropic macromolecular vibration modes. Because of a good controllability of the birefringence, polymer-based materials will provide an attractive materials system for phase retarders in the terahertz frequency range.