To date, no phase-shifting method can accurately retrieve the phase map from a small set of noisy interferograms with low phase-shifts. In this Letter, we develop a novel approach to resolve this limitation under such harsh conditions. The proposed new method is based on constructing a set of connected interferograms by means of simple subtraction and addition operations, in which all the subset of interferograms have the same phase-shift interval of π/2. According to this characteristic, this set of connected interferograms can be processed with conventional phase retrieval methods as PCA or AIA obtaining accurate results. The reduction in the RMS errors after using our method reaches as high as 93.7% and 89.3% respectively comparing with conventional PCA and AIA methods under harsh conditions. Both simulation and experiment results demonstrate that the new proposed method provides an effective way, with high precision and robustness against noise, for phase retrieval.