This paper proposes a pixelwise photometric stereo method for object surfaces with general bidirectional reflectance distribution functions (BRDFs) via appropriate reflection modeling. The modeling is based on three general characteristics of reflection components, i.e., the smooth variation of diffuse reflection, the concentration of specular reflection, and the low-intensity nature of shadow. A graph, whose nodes are light directions, is introduced to model these characteristics. In the graph, the neighborhood of nodes is determined by finding the light sources with close directions. The smoothness of the diffuse component is termed as the summation of local variations under all light sources. The specular reflection is modeled by group sparsity, and the shadow is determined via weighted l1 -norm modeling. The optimization problem, which incorporates these three modeling terms, is cast as a second-order cone programming problem. The proposed method is evaluated on both synthetic and real-world scenes with both isotropic and anisotropic materials. The experimental results show that the method is effective for object surfaces with general BRDFs and outperforms the state-of-the-arts.