Luminescent silicon nanocrystals (Si NCs) have attracted tremendous research interest. Their size dependent photoluminescence (PL) shows great promise in various optoelectronic and biomedical applications and devices. However, it remains unclear why the exciton emission is limited to energy below 2.1 eV, no matter how small the nanocrystal is. Here we interpret a nanosecond transient yellow emission band at 590 nm (2.1 eV) as a critical limit of the wavelength tunability in colloidal silicon nanocrystals. In the "large size" regime (d > ~3 nm), quantum confinement dominantly determines the PL wavelength and thus the PL peak blue shifts upon decreasing the Si NC size. In the "small size" regime (d < ~2 nm) the effect of the yellow band overwhelms the effect of quantum confinement with distinctly increased nonradiative trapping. As a consequence, the photoluminescence peak does not exhibit any additional blue shift and the quantum yield drops abruptly with further decreasing the size of the Si NCs. This finding confirms that the PL originating from the quantum confined core states can only exist in the red/near infrared with energy below 2.1 eV; while the blue/green PL originates from surface related states and exhibits nanosecond transition.