Nanoindentation has been recently used to measure the mechanical properties of polycrystalline graphene. However, the measured failure loads are found to be scattered widely and vary from lab to lab. We perform molecular dynamics simulations of nanoindentation on polycrystalline graphene at different sites including grain center, grain boundary (GB), GB triple junction, and holes. Depending on the relative position between the indenter tip and defects, significant scattering in failure load is observed. This scattering is found to arise from a combination of the non-uniform stress state, varied and weakened strengths of different defects, and the relative location between the indenter tip and the defects in polycrystalline graphene. Consequently, the failure behavior of polycrystalline graphene by nanoindentation is critically dependent on the indentation site, and is thus distinct from uniaxial tensile loading. Our work highlights the importance of the interaction between the indentation tip and defects, and the need to explicitly consider the defect characteristics at and near the indentation site in polycrystalline graphene during nanoindentation.