Regulatory T (Treg) cells play an essential role in maintaining immunological tolerance. The discovery of FoxP3 as a key Treg transcription factor combined with recent advances in the development of functional reporter mice has enabled new insights into Treg biology and revealed unexpected features of this lineage. In this review, we address the stability of this population, focusing on studies that suggest that Tregs can downregulate FoxP3, lose regulatory activity and, under some conditions, become memory T cells capable of recognizing self-antigens and expressing effector cell activities including the production of IL-17 and IFNgamma. The presence of these 'exTregs' in multiple inflammatory settings suggests a potential role for these cells in a variety of disease settings ranging from autoimmunity to cancer and infectious disease.