Vasospasm occurs both in patients and animal models after angioplasty and may be associated with early closure of the dilated vessel. To investigate the mechanism of angioplasty-induced vasospasm, the effect of serotonin-receptor blockade with two serotonin2 (S2) antagonists, LY53857 and sergolexole, was examined in rabbits with focal femoral artery atherosclerosis. In preliminary studies, local infusion of 1-100 micrograms serotonin caused significant femoral artery vasoconstriction (p less than 0.05) in both normal and atherosclerotic rabbits. There was no significant difference in the degree of vasoconstriction induced by equal doses of serotonin in normal and atherosclerotic animals. Infusion of 10 micrograms serotonin produced a 23 +/- 5% decrease in luminal diameter in atherosclerotic femoral arteries. This was blocked by pretreatment with both S2 inhibitors given separately in different animals before serotonin infusion (p less than 0.002). In contrast, LY53857 (sergolexole was not tested) had no significant effect on phenylephrine-induced vasoconstriction, confirming its specificity as an S2-receptor antagonist. Balloon angioplasty of atherosclerotic vessels caused a significant increase in vessel diameter at the angioplasty site (45% increase from baseline diameter, p less than 0.05). This was associated with significant luminal narrowing both proximal (21% reduction from baseline, p less than 0.05) and distal (17% reduction from baseline, p less than 0.03) to the angioplasty site. These proximal and distal changes are most likely due to vasospasm, as there was no histological evidence of thrombus or dissection at these sites to explain the luminal narrowing. Pretreatment of animals with 10 mg LY53857 or 20 mg sergolexole blocked the proximal vasospasm (2.6 +/- 0.4 before versus 2.2 +/- 0.1mm after angioplasty for LY53857, 2.1 +/- 0.4 before versus 2.1 +/- 0.4 mm after angioplasty for sergolexole; p = NS). Treatment with 20 mg LY53857 inhibited both proximal (2.3 +/- 0.1 before versus 2.2 +/- 0.2 mm after angioplasty, p = NS) and distal (1.7 +/- 0.1 before versus 1.6 +/- 0.2 mm after angioplasty, p = NS) vasospasm after angioplasty. Proximal (2.3 +/- 0.5 before versus 2.5 +/- 0.3 mm after) and distal (1.7 +/- 0.2 before versus 1.7 +/- 0.4 mm after) vasospasm was also prevented by pretreatment with 40 mg sergolexole.(ABSTRACT TRUNCATED AT 400 WORDS)