The magnetic compass sense of animals is currently thought to be based on light-dependent processes like the proposed radical pair mechanism. In accordance, many animals show orientation responses that depend on light. However, the orientation responses depend on the wavelength and irradiance of monochromatic light in rather complex ways that cannot be explained directly by the radical pair mechanism. Here, a radically different model is presented that can explain a vast majority of the complex observed light-dependent responses. The model put forward an integration process consisting of simple lateral inhibition between a normal functioning, light-independent magnetic compass (e.g. magnetite based) and a vision based skylight color gradient compass that misperceives compass cues in monochromatic light. Integration of the misperceived color compass cue and the normal magnetic compass not only explains most of the categorically different light-dependent orientation responses, but also shows a surprisingly good fit to how well the animals are oriented (r-values) under light of different wavelength and irradiance. The model parsimoniously suggests the existence of a single magnetic sense in birds (probably based on magnetic crystals).