Testicular germ cell tumors (TGCTs) are the most frequent malignancies in adolescents and young adults. The incidence of TGCTs has doubled over the last few decades and the mechanisms underlying their pervasive growth are still poorly understood. Among them, seminomatous and non-seminomatous tumors have carcinoma in situ of the testis (CIS) as a common precursor lesion. It is currently accepted that the acquisition of genetic alterations and/or exposure to environmental factors are involved in the transition from CIS to invasive tumors. Nevertheless, although several TGCT-associated genetic aberrations have been identified, the mechanisms mediating their effects on TGCT development are still largely unknown. The aim of this review is to analyze the potential role of testicular microenvironmental factors, such as hypoxia and stroma cell-derived factors, in the acquisition by TGCT cells of an aggressive phenotype and the importance of these factors as potential therapeutic targets.