Smad proteins have been demonstrated to be key components in the transforming growth factor beta signaling cascade. Here we demonstrate that Smad4, together with Smad3, can interact with the androgen receptor (AR) in the DNA-binding and ligand-binding domains, which may result in the modulation of 5alpha-dihydrotestosterone-induced AR transactivation. Interestingly, in the prostate PC3 and LNCaP cells, addition of Smad3 can enhance AR transactivation, and co-transfection of Smad3 and Smad4 can then repress AR transactivation in various androgen response element-promoter reporter assays as well as Northern blot and reverse transcription-PCR quantitation assays with prostate-specific antigen mRNA expression. In contrast, in the SW480.C7 cells, lacking endogenous functional Smad4, the influence of Smad3 on AR transactivation is dependent on the various androgen response element-promoters. The influence of Smad3/Smad4 on the AR transactivation may involve the acetylation since the treatment of trichostatin A or sodium butyrate can reverse Smad3/Smad4-repressed AR transactivation and Smad3/Smad4 complex can also decrease the acetylation level of AR. Together, these results suggest that the interactions between AR, Smad3, and Smad4 may result in the differential regulation of the AR transactivation, which further strengthens their roles in the prostate cancer progression.