An analysis of specific-ion effects in aqueous and nonaqueous solvents using energy decomposition analysis is presented. Specific-ion effects induce or influence physicochemical phenomena in a way that is determined by the identity of the ions present, and not merely by their charge or concentration. Such effects have been known since the seminal work of Hofmeister and are often categorized according to the well-known Hofmeister series. Examples of specific-ion effects are ubiquitous throughout chemistry and biology and are traditionally explained in terms of the influence ions have on the structure of water. However, this explanation is unsatisfactory because it is unable to adequately explain and predict frequently observed series reversals and anomalies. Further, recent experiments have shown that specific-ion effects are observed in nonaqueous solvents. By modeling solvated ion-N-isopropylacrylamide (NIPAM) complexes, we show here that specific-ion effects on ion-NIPAM interaction free energies are observed not only in water, but also in several nonaqueous solvents (methanol, acetonitrile, DMSO) in correspondence with the ions' Lewis Strengths. Interestingly, the same trends are observed in the absence of a solvent environment altogether. Counterion effects on ion-NIPAM interaction free energies are negligible for dissociated ion pairs but are evident in associated ion pairs because of the modulation of repulsive ion-NIPAM interactions. We propose a mechanism for explaining reversals in specific-ion effects, based on the competing strengths of the ion-solvent and ion-NIPAM interaction and their relative Lewis strengths. This extends existing theories regarding specific-ion effect reversals in aqueous solutions, as we show that solvent properties must also be taken in to account for specific-ion effects to be predicted in arbitrary solvent environments.