Motion of the sample arm fiber in optical coherence tomography (OCT) systems can dynamically alter the polarization state of light incident on tissue during imaging, with consequences for both conventional and polarization-sensitive (PS-)OCT. Endoscopic OCT is particularly susceptible to polarization-related effects, since in most cases, the transverse scanning mechanism involves motion of the sample arm optical fiber to create an image. We investigated the effects of a scanning sample arm fiber on the polarization state of light in an OCT system, and demonstrate that by referencing the state backscattered from within a sample to the measured state at the surface, changes in polarization state due to sample fiber motion can be isolated. The technique is demonstrated by high-speed PS-OCT imaging at 1 frame per second, with both linear and rotary scanning fiber-optic probes. Measurements were made on a calibrated wave plate, and endoscopic PS-OCT images of ex-vivo human tissues are also presented, allowing comparison with features in histologic sections.