To improve the photocatalytic activity of a transition metal coordination polymer modified polyoxometalate (TMCP/POM), polypyrrole (PPy) was loaded on its surface through a facile in situ chemical oxidation polymerization process. Under the irradiation of visible light, the PPy loaded TMCP/POM composite material exhibited better photocatalytic activity than TMCP/POM, PPy and their mechanically blended products with respect to the degradation of Rhodamine B (RhB). Optical and electrochemical tests illustrated that the enhancement of photocatalytic activity can be attributed to the high separation efficiency of the photogenerated electron-hole pair on the interface of PPy and TMCP/POM, which originates from a synergy effect between them. Furthermore, the influence of reaction temperature on the morphology, wettability, conductivity and photocatalytic performance of the resulting composite material was discussed and an optical temperature to fabricate the photocatalyst with high efficiency has been obtained. These results suggest that the loading of PPy on the surface of TMCP/POM would be a feasible strategy to enhance its photocatalytic activity.