Receptive fields in the retina indicate the first measurements taken over the (discrete) visual image. Why are they circular surround with an excitatory/inhibitory structure? We hypothesize that this provides a representation of the visual information in a form suitable for transmission over the optic nerve, a rather limited channel, that can then be extended into a variety of representations at the cortex. These cortical representations span a range of sizes and functionally separate positive and negative contrast data, precisely as is required for further processing. Our scheme is both physiologically and psychophysically plausible. In particular, we derive an explicit formula for constructing large receptive fields from small ones, and introduce the notion of de-blurring to derive interpolation filters for hyperacuity. A mathematical requirement of our scheme is a form of separation between positive and negative contrast data, a nonlinearity that we predict will agree with observations. Furthermore, the mathematics that we utilize are more naturally applicable to physiological models based on Gaussian than are (Fourier) spatial frequencies.