Due to ever increasing precision and automation demands in robotic grinding, the automatic and robust robotic grinding workstation has become a research hot-spot. This work proposes a grinding workstation constituting of machine vision and an industrial manipulator to solve the difficulty of positioning rough metal cast objects and automatic grinding. Faced with the complex characteristics of industrial environment, such as weak contrast, light nonuniformity and scarcity, a coarse-to-fine two-step localization strategy was used for obtaining the object position. The deep neural network and template matching method were employed for determining the object position precisely in the presence of ambient light. Subsequently, edge extraction and contour fitting techniques were used to measure the position of the contour of the object and to locate the main burr on its surface after eliminating the influence of burr. The grid method was employed for detecting the main burrs, and the offline grinding trajectory of the industrial manipulator was planned with the guidance of the coordinate transformation method. The system greatly improves the automaticity through the entire process of loading, grinding and unloading. It can determine the object position and target the robotic grinding trajectory by the shape of the burr on the surface of an object. The measurements indicate that this system can work stably and efficiently, and the experimental results demonstrate the high accuracy and high efficiency of the proposed method. Meanwhile, it could well overcome the influence of the materials of grinding work pieces, scratch and rust.