Orbital angular momentum (OAM) is an important resource in high-dimensional quantum information processing, as its quantum number can be infinite. Dove prism (DP) is a most common tool to manipulate OAM light. However, the Dove prism changes the polarization of the photon states and decreases the sorting fidelity of the interferometer. In this work, we analyze the polarization-dependent effect of the DP on OAM light manipulation in the normal single-path Sagnac interferometers (SPSIs) with beam splitter (BS) and polarizing beam splitter (PBS). The results demonstrate that the BS SPSI is more sensitive to the input polarization and the specific parameters of the DP. We have also proposed and realized a modified BS SPSI, of which the sorting fidelity can be 100% in principle and is independent on the input polarization and the transmission matrix of the DP. The experiments demonstrate that the fidelity of the modified BS SPSI is about 5%~10% higher than that of the normal one. The modified BS SPSI is easy to implement (only two more half-wave plates are required) and is stable for free running at the scale of several hours. These merits make the structure suitable for applications in critical quantum information processing tasks, such as quantum cryptography.