Executive mechanisms involved in task switching were studied in 18 brain injured patients. The patients had to rapidly switch back and forth between two visual classification tasks and the analyses focused on switch costs, (i.e., performance differences between switch and no-switch trials), and on interference effects, (i.e., processing costs imposed by the presence of interfering stimulus attributes). The patients were grouped according to the side of the brain lesion. Patients with left brain damage (LBD) showed higher switch costs than patients with right brain damage (RBD). These group differences were attributable to disproportionally high switch costs in patients with LBD and language or speech disorders. This result suggests that the efficiency of suppressing internal interference from a recently activated task set depends on the availability of verbal representations of the upcoming task. Patients with RBD showed higher interference from external task sets. This effect was not affected by language or speech disorders. The overall results argue for a fractionation of executive functions to protect against interference from internal and external sources in task switching.