Identifying the structures of high-pressure hydrogen has been one of the central goals in high-pressure physics; however, it still presents a fundamental challenge because of the lack of an effective measure for distinguishing the structures. Herein, we address this issue by focusing on the potential candidates of phases II and III of high-pressure hydrogen. We find that the anisotropic dielectric constants of the different hydrogen solids and their responses to pressure behave differently depending on the atomic structures, corresponding to the different polarization responses of the structures to the external electric field. These findings are robust regardless of the quantum and thermal motion of hydrogen solids. Therefore, the anisotropic dielectric property can serve as a potential measure for probing the structures of high-pressure hydrogen as well as other high-pressure materials.