Many nonstarch polysaccharides (NSPs) classified as dietary fibers have been reported to possess immunoregulatory properties. The fibers reported to activate or by other means modulate immune responses originate from both plant, fungal, and microbial sources and constitute highly distinct structures. In order to enhance our understanding of factors important for the immunoregulatory activities, this article addresses the importance of chemical structure, origin, and purity of fibers for their capacity to interact with key regulatory immune cells. Furthermore, we assess bioavailability, and discuss possible mechanisms involved. The binding of some NSPs to carbohydrate receptors on immune cells is well established and this event leads to activation or other changes. Especially, certain beta-glucans and some mannans have demonstrated immunomodulatory capacity with the specific structure being important for the activity. Within beta-glucans the activity varies according to structure, molecular weight, and solubility. As many of the preparations tested constitute crude extracts or partly purified NSPs, the risk of contaminants holding immunoregulatory activities should not be ignored. To what extent NSPs enter systemic circulation has been difficult to assess, partly due to lack of sensitive analytical methods. The presence of NSPs in blood and Peyer's patches in the gut has been demonstrated, supporting encounter between NSPs and immune cells, but bioavailability studies still constitute a major challenge. Studies demonstrating in vivo effects of beta-glucans on microbial infections and cancer treatment strongly indicate an immunoregulatory mechanism behind the effects. However, the potential of NSPs as immunoregulatory food ingredients is still far from fully explored.