We generalize the standard coupled mode equations describing interactions between forward and backward propagating waves in a nonlinear optical Bragg grating. Including the lowest order corrections of the grating depth, we obtain a Hamiltonian system that can be regarded as an extension of the usual coupled mode equations for shallow gratings. The results are consistent with existing results based on a Bloch wave expansion. We also obtain exact traveling solitary wave solutions, that can be regarded as a generalized gap soliton, modified by the grating's depth.