Color-graphemic synesthetes perceive colors when viewing alphanumeric characters. Theories of color-graphemic synesthesia posit that synesthetic color experience arises from activation of neural mechanisms also involved in ordinary color vision. To learn how early in visual processing those mechanisms exist, we performed several experiments. In one experiment, real colors were altered in appearance by the lightness of their backgrounds, but the appearance of synesthetic colors was immune to surrounding light levels. In the second experiment using a hue cancellation technique, adaptation to synesthetic color had no subsequent effect on the amount of cancelling light to achieve equilibrium yellow, whereas adaptation to real colors did. In the third experiment, vivid synesthetic color had no influence on equilibrium yellow settings of the actual color of the characters evoking synesthesia. Because brightness contrast and chromatic adaptation are putatively mediated by neural mechanisms early in visual processing including retina and primary visual cortex, our results imply that neural events responsible for synesthetic color emerge subsequent to these early visual stages.