We investigate the out-of-equilibrium behavior of a dissipative gas of Rydberg atoms that features a dynamical transition between two stationary states characterized by different excitation densities. We determine the structure and properties of the phase diagram and identify the universality class of the transition, both for the statics and the dynamics. We show that the proper dynamical order parameter is in fact not the excitation density and find evidence that the dynamical transition is in the "model A" universality class; i.e., it features a nontrivial Z2 symmetry and a dynamics with nonconserved order parameter. This sheds light on some relevant and observable aspects of dynamical transitions in Rydberg gases. In particular it permits a quantitative understanding of a recent experiment [C. Carr, Phys. Rev. Lett. 111, 113901 (2013)] which observed bistable behavior as well as power-law scaling of the relaxation time. The latter emerges not due to critical slowing down in the vicinity of a second order transition, but from the nonequilibrium dynamics near a so-called spinodal line.