NCBI Bookshelf. A service of the National Library of Medicine, National Institutes of Health.

Murray MM, Wallace MT, editors. The Neural Bases of Multisensory Processes. Boca Raton (FL): CRC Press/Taylor & Francis; 2012.

Cover of The Neural Bases of Multisensory Processes

The Neural Bases of Multisensory Processes.

Show details

Chapter 2Cortical and Thalamic Pathways for Multisensory and Sensorimotor Interplay

, , and .

2.1. INTRODUCTION

Numerous studies in both monkey and human provided evidence for multisensory integration at high-level and low-level cortical areas. This chapter focuses on the anatomical pathways contributing to multisensory integration. We first describe the anatomical connections existing between different sensory cortical areas, briefly concerning the well-known connections between associative cortical areas and the more recently described connections targeting low-level sensory cortical areas. Then we focus on the description of the connections of the thalamus with different sensory and motor areas and their potential role in multisensory and sensorimotor integration. Finally, we discuss the several possibilities for the brain to integrate the environmental world with the different senses.

2.2. CORTICAL AREAS IN MULTISENSORY PROCESSES

2.2.1. Multisensory Association Cortices

Parietal, temporal, and frontal cortical regions of primates have been reported to be polysensory cortical areas, i.e., related to more than a single sensory modality. We describe here several important features about these regions, focusing on the superior temporal sulcus (STS), the intraparietal sulcus, and the frontal cortex.

2.2.1.1. Superior Temporal Sulcus

Desimone and Gross (1979) found neurons responsive to visual, auditory, and somatosensory stimuli in a temporal region of the STS referred to as superior temporal plane (STP) (see also Bruce et al. 1981; Baylis et al. 1987; Hikosaka et al. 1988). The rostral part of the STS (Bruce et al. 1981; Benevento et al. 1977) appears to contain more neurons with multisensory properties than the caudal part (Hikosaka et al. 1988). The connections of the STP include higher-order visual cortical areas as posterior parietal visual areas (Seltzer and Pandya 1994; Cusick et al. 1995) and temporal lobe visual areas (Kaas and Morel 1993), auditory cortical areas (Pandya and Seltzer 1982), and posterior parietal cortex (Seltzer and Pandya 1994; Lewis and Van Essen 2000). The STS region also has various connections with the prefrontal cortex (Cusick et al. 1995). In humans, numerous neuroimaging studies have shown multisensory convergence in the STS region (see Barraclough et al. 2005 for a review).

Recently, studies have focused on the role of the polysensory areas of the STS and their interactions with the auditory cortex in processing primate communications (Ghazanfar 2009). The STS is probably one of the origins of visual inputs to the auditory cortex (Kayser and Logothetis 2009; Budinger and Scheich 2009; Cappe et al. 2009a; Smiley and Falchier 2009) and thus participates in the multisensory integration of conspecific face and vocalizations (Ghazanfar et al. 2008) that occurs in the auditory belt areas (Ghazanfar et al. 2005; Poremba et al. 2003). These findings support the hypothesis of general roles for the STS region in synthesizing perception of speech and general biological motion (Calvert 2001).

2.2.1.2. Intraparietal Sulcus

The posterior parietal cortex contains a number of different areas including the lateral intraparietal (LIP) and ventral intraparietal (VIP) areas, located in the intraparietal sulcus. These areas seem to be functionally related and appear to encode the location of objects of interest (Colby and Goldberg 1999). These areas are thought to transform sensory information into signals related to the control of hand and eye movements via projections to the prefrontal, premotor, and visuomotor areas of the frontal lobe (Rizzolatti et al. 1997). Neurons of the LIP area present multisensory properties (Cohen et al. 2005; Russ et al. 2006; Gottlieb 2007). Similarly, neurons recorded in the VIP area exhibit typical multisensory responses (Duhamel et al. 1998; Bremmer et al. 2002; Schlack et al. 2005; Avillac et al. 2007). Anatomically, LIP and VIP are connected with cortical areas of different sensory modalities (Lewis and Van Essen 2000). In particular, VIP receives inputs from posterior parietal areas 5 and 7 and insular cortex in the region of S2, and few inputs from visual regions such as PO and MST (Lewis and Van Essen 2000). Although it is uncertain whether neurons in VIP are responsive to auditory stimuli, auditory inputs may originate from the dorsolateral auditory belt and parabelt (Hackett et al. 1998). The connectivity pattern of LIP (Andersen et al. 1990; Blatt et al. 1990; Lewis and Van Essen 2000) is consistent with neuronal responses related to eye position and visual inputs. Auditory and somatosensory influences appear to be very indirect and visuomotor functions dominate, as the connection pattern suggests. In particular, the ventral part of the LIP is connected with areas dealing with spatial information (Andersen et al. 1997) as well as with the frontal eye field (Schall et al. 1995), whereas the dorsal part of the LIP is connected with areas responsible for the processing of visual information related to the form of objects in the inferotemporal cortex (ventral “what” visual pathway). Both LIP and VIP neurons exhibit task-dependent responses (Linden et al. 1999; Gifford and Cohen 2004), although the strength of this dependence and its rules remain to be determined.

2.2.1.3. Frontal and Prefrontal Cortex

The premotor cortex, located in the frontal lobe, contains neurons with responses to somatosensory, auditory, and visual signals, especially its ventral part as shown in monkeys (Fogassi et al. 1996; Graziano et al. 1994, 1999). Somatosensory responses may be mediated by connections with somatosensory area S2 and parietal ventral (PV) somatosensory area (Disbrow et al. 2003) and with the posterior parietal cortex, such as areas 5, 7a, 7b, anterior intraparietal area (AIP), and VIP (see Kaas and Collins 2004). Visual inputs could also come from the posterior parietal region. The belt and parabelt auditory areas project to regions rostral to the premotor cortex (Hackett et al. 1999; Romanski et al. 1999) and may contribute to auditory activation, as well as connections from the trimodal portion of area 7b to the premotor cortex (Graziano et al. 1999).

Anterior to the premotor cortex, the prefrontal cortex plays a key role in temporal integration and is related to evaluative and cognitive functions (Milner et al. 1985; Fuster 2001). Much of this cortex has long been considered to be multisensory (Bignall 1970) but some regions are characterized by some predominance in one sensory modality, such as an auditory domain in the ventral prefrontal region (Suzuki 1985; Romanski and Goldman-Rakic 2002; Romanski 2004). This region receives projections from auditory, visual, and multisensory cortical regions (e.g., Gaffan and Harrison 1991; Barbas 1986; Romanski et al. 1999; Fuster et al. 2000), which are mediated through different functional streams ending separately in the dorsal and ventral prefrontal regions (Barbas and Pandya 1987; Kaas and Hackett 2000; Romanski et al. 1999). This cortical input arising from different modalities confer to the prefrontal cortex a role in cross-modal association (see Petrides and Iversen 1976; Joseph and Barone 1987; Barone and Joseph 1989; Ettlinger and Wilson 1990) as well as in merging sensory information especially in processing conspecific auditory and visual communication stimuli (Romanski 2007; Cohen et al. 2007).

2.2.2. Low-Level Sensory Cortical Areas

Several studies provide evidence that anatomical pathways between low-level sensory cortical areas may represent the anatomical support for early multisensory integration. We will detail these patterns of connections in this part according to sensory interactions.

2.2.2.1. Auditory and Visual Connections and Interactions

Recently, the use of anterograde and retrograde tracers in the monkey brain made it possible to highlight direct projections from the primary auditory cortex (A1), the caudal auditory belt and parabelt, and the polysensory area of the temporal lobe (STP) to the periphery of the primary visual cortex (V1, area 17 of Brodmann) (Falchier et al. 2002), as well as from the associative auditory cortex to the primary and secondary visual areas (Rockland and Ojima 2003). These direct projections of the auditory cortex toward the primary visual areas would bring into play connections of the feedback type and may play a role in the “foveation” of a peripheral auditory sound source (Heffner and Heffner 1992). The reciprocity of these connections from visual areas to auditory areas was also tested in a recent study (Falchier et al. 2010) that revealed the existence of projections from visual areas V2 and prostriata to auditory areas, including the caudal medial and lateral belt area, the caudal parabelt area, and the temporoparietal area. Furthermore, in the marmoset, a projection from the high-level visual areas to the auditory cortex was also reported (Cappe and Barone 2005). More precisely, an area anterior to the STS (corresponding to the STP) sends connections toward the auditory core with a pattern of feedback connections. Thus, multiple sources can provide visual input to the auditory cortex in monkeys (see also Smiley and Falchier 2009; Cappe et al. 2009a).

Direct connections between the primary visual and auditory areas have been found in rodents, such as in the gerbil (Budinger et al. 2006) or the prairie vole (Campi et al. 2010) as well as in carnivores. For example, the primary auditory cortex of the ferret receives a sparse projection from the visual areas including the primary visual cortex (Bizley et al. 2007). Similarly, in the adult cat, visual and auditory cortices are interconnected but the primary sensory fields are not the main areas involved. Only a minor projection is observed from A1 toward the visual areas A17/18 (Innocenti et al. 1988), the main component arising from the posterior auditory field (Hall and Lomber 2008). It is important to note that there is probably a tendency for a decrease in the density of these auditory– visual interconnections when going from rodents to carnivore to primates. This probably means a higher incidence of cross-modal responses in unisensory areas of the rodents (Wallace et al. 2004), whereas such responses are not present in the primary visual or auditory cortex of the monkey (Lakatos et al. 2007; Kayser et al. 2008; Wang et al. 2008).

On the behavioral side, in experiments conducted in animals, multisensory integration dealt in most cases with spatial cues, for instance, the correspondence between the auditory space and the visual space. These experiments were mainly conducted in cats (Stein et al. 1989; Stein and Meredith 1993; Gingras et al. 2009). For example, Stein and collaborators (1989) trained cats to move toward visual or auditory targets with weak salience, resulting in poor performance that did not exceed 25% on average. When the same stimuli were presented in spatial and temporal congruence, the percentage of correct detections increased up to nearly 100%. In monkeys, only few experiments have been conducted on behavioral facilitation induced by multimodal stimulation (Frens and Van Opstal 1998; Bell et al. 2005). In line with human studies, simultaneous presentation in monkeys of a sound during a visually guided saccade induced a reduction of about 10% to 15% of saccade latency depending on the visual stimulus contrast level (Wang et al. 2008). Recently, we have shown behavioral evidence for multisensory facilitation between vision and hearing in macaque monkeys (Cappe et al. 2010). Monkeys were trained to perform a simple detection task to stimuli, which were auditory (noise), visual (flash), or auditory–visual (noise and flash) at different intensities. By varying the intensity of individual auditory and visual stimuli, we observed that, when the stimuli are of weak saliency, the multisensory condition had a significant facilitatory effect on reaction times, which disappeared at higher intensities (Cappe et al. 2010). We applied to the behavioral data the “race model” (Raab 1962) that supposes that the faster unimodal modality should be responsible for the shortening in reaction time (“the faster the winner”), which would correspond to a separate activation model (Miller 1982). It turns out that the multisensory benefit at low intensity derives from a coactivation mechanism (Miller 1982) that implies a convergence of hearing and vision to produce multisensory interactions and a reduction in reaction time. The anatomical studies previously described suggest that such a convergence may take place at the lower levels of cortical sensory processing.

In humans, numerous behavioral studies, using a large panel of different paradigms and various types of stimuli, showed the benefits of auditory–visual combination stimuli compared to unisensory stimuli (see Calvert et al. 2004 for a review; Romei et al. 2007; Cappe et al. 2009b as recent examples).

From a functional point of view, many studies have shown multisensory interactions early in time and in different sensory areas with neuroimaging and electrophysiological methods. Auditory–visual interactions have been revealed in the auditory cortex or visual cortex using electrophysiological or neuroimaging methods in cats and monkeys (Ghazanfar et al. 2005; Bizley et al. 2007; Bizley and King 2008; Cappe et al. 2007; Kayser et al. 2007, 2008; Lakatos et al. 2007; Wang et al. 2008). More specifically, electrophysiological studies in monkeys, revealing multisensory interactions in primary sensory areas such as V1 or A1, showed that cross-modal stimuli (i.e., auditory or visual stimuli, respectively) are rather modulatory on the non-“sensory-specific” response, and/ or acting on the oscillatory activity (Lakatos et al. 2007; Kayser et al. 2008) or on the latency of the neuronal responses (Wang et al. 2008). These mechanisms can enhance the speed of sensory processing and induce a reduction of the reaction times (RTs) during a multisensory stimulation. Neurons recorded in the primary visual cortex showed a significant reduction in visual response latencies, specifically in suboptimal conditions (Wang et al. 2008). It is important to mention that, in the primary sensory areas of the primate, authors have reported the absence of nonspecific sensory responses at the spiking level (Wang et al. 2008; Lakatos et al. 2007; Kayser et al. 2008). These kinds of interactions between hearing and vision were also reported in humans using neuroimaging techniques (Giard and Peronnet 1999; Molholm et al. 2002; Lovelace et al. 2003; Laurienti et al. 2004; Martuzzi et al. 2007).

2.2.2.2. Auditory and Somatosensory Connections and Interactions

The advantage of being able to use a number of distinct tracers allows us to identify connections between several cortical areas. Indeed, we made injections of retrograde tracers into early visual (V2 and MT), somatosensory (1/3b), and auditory (core) cortical areas in marmosets (Cappe and Barone 2005) allowing us to exhibit connections between cortical areas considered as unisensory areas. Projections from visual areas, such as the STP, to the core auditory cortex have been found (Cappe and Barone 2005), as described in Section 2.2.2. Other corticocortical projections, and in particular from somatosensory to auditory cortex, were found, supporting the view that inputs from different modalities are sent to cortical areas that are classically considered to be unimodal (Cappe and Barone 2005). More precisely, our study revealed projections from somatosensory areas S2/PV to the primary auditory cortex. Another study conducted in gerbils also showed connections between the primary somatosensory cortex and the primary auditory cortex (Budinger et al. 2006). In marmosets and macaques, projections from the retroinsular area of the somatosensory cortex to the caudiomedial belt auditory area were also reported (de la Mothe et al. 2006a; Smiley et al. 2007).

Intracranial recordings in the auditory cortex of monkeys have shown the modulation of auditory responses by somatosensory stimuli, consistent with early multisensory convergence (Schroeder et al. 2001; Schroeder and Foxe 2002; Fu et al. 2003). These findings have been extended by a functional magnetic resonance imaging (fMRI) study in anesthetized monkeys, which showed auditory–somatosensory interactions in the caudal lateral belt area (Kayser et al. 2005).

In humans, there have been previous demonstrations of a redundant signal effect between auditory and tactile stimuli (Murray et al. 2005; Zampini et al. 2007; Hecht et al. 2008). Functional evidence was mainly found with EEG and fMRI techniques (Foxe et al. 2000, 2002; Murray et al. 2005). In particular, Murray and collaborators (2005) reported in humans that neural responses showed an initial auditory–somatosensory interaction in auditory association areas.

2.2.2.3. Visual and Somatosensory Connections and Interactions

Limited research has been focused on interactions between vision and touch. In our experiments, using multiple tracing methods in marmoset monkeys (Cappe and Barone 2005), we found direct projections from visual cortical areas to somatosensory cortical areas. More precisely, after an injection of retrograde tracer in the primary somatosensory cortex (areas 1/3b), we observed projections originating from visual areas (the ventral and dorsal fundus of the superior temporal area, and the middle temporal crescent).

On a functional point of view, electrophysiological recordings in the somatosensory cortex of macaque monkeys showed modulations of responses by auditory and visual stimuli (Schroeder and Foxe 2002). Behavioral results in humans demonstrated gain in performance when visual and tactile stimuli were combined (Forster et al. 2002; Hecht et al. 2008). Evidence of functional interactions between vision and touch was observed with neuroimaging techniques in humans (Amedi et al. 2002, 2007; James et al. 2002). In particular, it has been shown that the perception of motion could activate the MT complex in humans (Hagen et al. 2002). It has also been demonstrated that the extrastriate visual cortex area 19 is activated during tactile perception (see Sathian and Zangaladze 2002 for review).

2.2.2.4. Heteromodal projections and Sensory Representation

In somatosensory (Krubitzer and Kaas 1990; Huffman and Krubitzer 2001) and visual systems (Kaas and Morel 1993; Schall et al. 1995; Galletti et al. 2001; Palmer and Rosa 2006), there is evidence for the existence of different connectivity patterns according to sensory representation, especially in terms of the density of connections between areas. This observation also applies to heteromodal connections. We found that the visual projections to areas 1/3b are restricted to the representation of certain body parts (Cappe and Barone 2005). Some visual projections selectively target the face (middle temporal crescent) or the arm (dorsal fundus of the superior temporal area) representations in areas 1/3b. Similarly, auditory and multimodal projections to area V1 are prominent toward the representation of the peripheral visual field (Falchier et al. 2002, 2010; Hall and Lomber 2008), and only scattered neurons in the auditory cortex send a projection to foveal V1. The fact that heteromodal connections are coupling specific sensory representations across modalities probably reflects an adaptive process for behavioral specialization. This is in agreement with human and monkey data showing that the neuronal network involved in multisensory integration, as well as its expression at the level of the neuronal activity, is highly dependent on the perceptual task in which the subject is engaged. In humans, the detection or discrimination of bimodal objects, as well as the perceptual expertise of subjects, differentially affect both the temporal aspects and the cortical areas at which multisensory interactions occur (Giard and Peronnet 1999; Fort et al. 2002). Similarly, we have shown that the visuo–auditory interactions observed at the level of V1 neurons are observed only in behavioral situations during which the monkey has to interact with the stimuli (Wang et al. 2008).

Such an influence of the perceptual context on the neuronal expression of multisensory interaction is also present when analyzing the phenomena of cross-modal compensation after sensory deprivation in human. In blind subjects (Sadato et al. 1996), the efficiency of somatosensory stimulation on the activation of the visual cortex is at maximum during an active discrimination task (Braille reading). This suggests that the mechanisms of multisensory interaction, at early stages of sensory processing and the cross-modal compensatory mechanisms, are probably mediated through common neuronal pathways involving the heteromodal connections described previously.

2.3. THALAMUS IN MULTISENSORY PROCESSES

2.3.1. Thalamocortical and Corticothalamic Connections

Although the cerebral cortex and the superior colliculus (Stein and Meredith 1993) have been shown to be key structures for multisensory interactions, the idea that the thalamus could play a relay role in multisensory processing has been frequently proposed (Ghazanfar and Schroeder 2006 for review; Hackett et al. 2007; Cappe et al. 2009c; see also Cappe et al. 2009a for review).

By using anatomical multiple tracing methods in the macaque monkey, we were able to test this hypothesis recently and looked at the relationship and the distribution of the thalamocortical and the corticothalamic (CT) connections between different sensory and motor cortical areas and thalamic nuclei (Cappe et al. 2009c). In this study, we provided evidence for the convergence of different sensory modalities in the thalamus. Based on different injections in somatosensory [in the posterior parietal somatosensory cortex (PE/PEa in area 5)], auditory [in the rostral (RAC) and caudal auditory cortex (CAC)], and premotor cortical areas [dorsal and ventral premotor cortical areas (PMd and PMv)] in the same animal, we were able to assess how connections between the cortex and the different thalamic nuclei are organized.

We demonstrated for the first time the existence of overlapping territories of thalamic projections to different sensory and motor areas. We focus our review on thalamic nuclei that are projecting into more than two areas of different attributes rather than on sensory-specific thalamocortical projections. Thalamocortical projections were found from the central lateral (CL) nucleus and the mediodorsal (MD) nucleus to RAC, CAC, PEa, PE, PMd, and PMv. Common territories of projection were observed from the nucleus LP to PMd, PMv, PEa, and PE. The ventroanterior nucleus (VA), known as a motor thalamic nucleus, sends projections to PE and to PEa. Interestingly, projections distinct from the ones arising from specific unimodal sensory nuclei were observed from auditory thalamic nuclei, such as projections from the medial geniculate nucleus to the parietal cortex (PE in particular) and the premotor cortex (PMd/PMv). Last but not least, the medial pulvinar nucleus (PuM) exhibits the most significant overlap across modalities, with projections from superimposed territories to all six cortical areas injected with tracers. Projections from PuM to the auditory cortex were also described by de la Mothe and colleagues (2006b). Hackett and collaborators (2007) showed that somatosensory inputs may reach the auditory cortex (CM and CL) through connections coming from the medial part of the medial geniculate nucleus (MGm) or the multisensory nuclei [posterior, suprageniculate, limitans, and medial pulvinar (PuM)]. All these thalamocortical projections are consistent with the presence of thalamic territories possibly integrating different sensory modalities with motor attributes.

We calculated the degree of overlap between thalamocortical and CT connections in the thalamus to determine the projections to areas of a same modality, as previously described (Tanné-Gariépy et al. 2002; Morel et al. 2005; Cappe et al. 2009c). The degree of overlap may range between 0% when two thalamic territories projecting to two distinct cortical areas are spatially completely segregated and 100% when the two thalamic territories fully overlap (considering a spatial resolution of 0.5 mm, further details in Cappe et al. 2009c). Thalamic nuclei with spatially intermixed thalamocortical cells projecting to auditory or premotor cortices were located mainly in the PuM, VA, and CL nuclei. The overlap between the projections to the auditory and parietal cortical areas concerned different thalamic nuclei such as PuM, CL, and to a lesser extent, LP and PuL. The projections to the premotor and posterior parietal cortex overlapped primarily in PuM, LP, MD, and also in VA, VLpd, and CL. Quantitatively, we found that projections from the thalamus to the auditory and motor cortical areas overlapped to an extent ranging from 4% to 12% through the rostral thalamus and increased up to 30% in the caudal part of the thalamus. In PuM, the degree of overlap between thalamocortical projections to auditory and premotor cortex ranged from 14% to 20%. PuM is the thalamic nucleus where the maximum of overlap between thalamocortical projections was found.

Aside from the thalamocortical connections, CT connections were also investigated in the same study, concerning, in particular, the parietal areas PE and PEa injected with a tracer with anterograde properties (biotinylated dextran amine; Cappe et al. 2007). Indeed, areas PE and PEa send CT projections to the thalamic nuclei PuM, LP, and to a lesser extent, VPL, CM, CL, and MD (PEa only for MD). These thalamic nuclei contained both the small and giant CT endings. The existence of these two different types of CT endings reflect the possibility for CT connections to represent either feedback or feedforward projections (for review, see Rouiller and Welker 2000; Sherman and Guillery 2002, 2005; Sherman 2007). In contrast to the feedback CT projection originating from cortical layer VI, the feedforward CT projection originates from layer V and terminates in the thalamus in the form of giant endings, which can ensure highly secure and rapid synaptic transmission (Rouiller and Welker 2000). Considering the TC and CT projections, some thalamic nuclei (PuM, LP, VPL, CM, CL, and MD) could play a role in the integration of different sensory information with or without motor attributes (Cappe et al. 2007, 2009c). Moreover, parietal areas PE and PEa may send, via the giant endings, feedforward CT projection and transthalamic projections to remote cortical areas in the parietal, temporal, and frontal lobes contributing to polysensory and sensorimotor integration (Cappe et al. 2007, 2009c).

2.3.2. Role of Thalamus in Multisensory Integration

The interconnections between the thalamus and the cortex described in the preceding section suggest that the thalamus could play the role of early sensory integrator. An additional role for the thalamus in multisensory interplay may derive from the organization of its CT and thalamocortical connections/loops as evoked in Section 2.3.1 (see also Crick and Koch 1998). Indeed, the thalamus could also have a relay role between different sensory and/or premotor cortical areas. In particular, the pulvinar, mainly its medial part, contains neurons which project to the auditory cortex, the somatosensory cortex, the visual cortex, and the premotor cortex (Romanski et al. 1997; Hackett et al. 1998; Gutierrez et al. 2000; Cappe et al. 2009c; see also Cappe et al. 2009a for a review). The feedforward CT projection originating from different sensory or motor cortical areas, combined with a subsequent TC projection, may allow a transfer of information between remote cortical areas through a “cortico–thalamo–cortical” route (see, e.g., Guillery 1995; Rouiller and Welker 2000; Sherman and Guillery 2002, 2005; Sherman 2007; Cappe et al. 2009c). As described in Section 2.3.1, the medial part of the pulvinar nucleus is the main candidate (although other thalamic nuclei such as LP, VPL, MD, or CL may also play a role) to represent an alternative to corticocortical loops by which information can be transferred between cortical areas belonging to different sensory and sensorimotor modalities (see also Shipp 2003). On a functional point of view, neurons in PuM respond to visual stimuli (Gattass et al. 1979) and auditory stimuli (Yirmiya and Hocherman 1987), which is consistent with our hypothesis.

Another point is that, as our injections in the different sensory and motor areas included cortical layer I (Cappe et al. 2009c), it is likely that some of these projections providing multimodal information to the cortex originate from the so-called “matrix” calbindin-immunoreactive neurons distributed in all thalamic nuclei and projecting diffusely and relatively widely to the cortex (Jones 1998).

Four different mechanisms of multisensory and sensorimotor interplay can be proposed based on the pattern of convergence and divergence of thalamocortical and CT connections (Cappe et al. 2009c). First, some restricted thalamic territories sending divergent projections to cortical areas afford different sensory and/or motor inputs which can be mixed simultaneously. Although such a multimodal integration in the temporal domain cannot be excluded (in case the inputs reach the cerebral cortex at the exact same time), it is less likely to provide massive multimodal interplay than an actual spatial convergence of projections. More convincingly, this pattern could support a temporal coincidence mechanism as a synchronizer between remote cortical areas, allowing a higher perceptual saliency of multimodal stimuli (Fries et al. 2001). Second, thalamic nuclei could be an integrator of multisensory information, rapidly relaying this integrated information to the cortex by their multiple thalamocortical connections. In PuM, considerable mixing of territories projecting to cortical areas belonging to several modalities is in line with previously reported connections with several cortical domains, including visual, auditory, somatosensory, and prefrontal and motor areas. Electrophysiological recordings showed visual and auditory responses in this thalamic nucleus (see Cappe et al. 2009c for an extensive description). According to our analysis, PuM, LP, MD, MGm, and MGd could play the role of integrator (Cappe et al. 2009c). Third, the spatial convergence of different sensory and motor inputs at the cortical level coming from thalamocortical connections of distinct thalamic territories suggests a fast multisensory interplay. In our experiments (Cappe et al. 2009c), the widespread distribution of thalamocortical inputs to the different cortical areas injected could imply that this mechanism of convergence plays an important role in multisensory and motor integration. By their cortical connection patterns, thalamic nuclei PuM and LP, for instance, could play this role for auditory–somatosensory interplay in area 5 (Cappe et al. 2009c). Fourth, the cortico–thalamo–cortical route can support rapid and secure transfer from area 5 (PE/PEa; Cappe et al. 2007) to the premotor cortex via the giant terminals of these CT connections (Guillery 1995; Rouiller and Welker 2000; Sherman and Guillery 2002, 2005; Sherman 2007). These giant CT endings, consistent with this principle of transthalamic loop, have been shown to be present in different thalamic nuclei (e.g., Schwartz et al. 1991; Rockland 1996; Darian-Smith et al. 1999; Rouiller et al. 1998, 2003; Taktakishvili et al. 2002; Rouiller and Durif 2004) and may well also apply to PuM, as demonstrated by the overlap between connections to the auditory cortex and to the premotor cortex, allowing an auditory–motor integration (Cappe et al. 2009c).

Thus, recent anatomical findings at the thalamic level (Komura et al. 2005; de la Mothe 2006b; Hackett et al. 2007; Cappe et al. 2007, 2009c) may represent the anatomical support for multisensory behavioral phenomenon as well as multisensory integration at the functional level. Indeed, some nuclei in the thalamus, such as the medial pulvinar, receive either mixed sensory inputs or projections from different sensory cortical areas and project to sensory and premotor areas (Cappe et al. 2009c). Sensory modalities may thus already be fused at the thalamic level before being directly conveyed to the premotor cortex and consequently participating in the redundant signal effect expressed by faster reaction times in response to auditory–visual stimulation (Cappe et al. 2010).

2.4. HIGHER-ORDER, LOWER-ORDER CORTICAL AREAS AND/OR THALAMUS?

When applying the race model to behavioral performance for multisensory tasks, results showed that this model cannot account for the shorter reaction times in auditory–visual conditions (see Cappe et al. 2010 for data in monkeys), a result that imposes a “coactivation” model and implies a convergence of the sensory channels (Miller 1982). The anatomical level at which the coactivation occurs is still under debate (Miller et al. 2001), as it has been suggested to occur early at the sensory level (Miller et al. 2001; Gondan et al. 2005) or late at the motor stage (Giray and Ulrich 1993). However, in humans, analysis of the relationships between behavioral and neuronal indices (Molholm et al. 2002; Sperdin et al. 2009; Jepma et al. 2009) seems to suggest that this convergence of the sensory channels occurs early in sensory processing, before the decision at motor levels (Mordkoff et al. 1996; Gondan et al. 2005), as shown in monkeys (Lamarre et al. 1983; Miller et al. 2001; Wang et al. 2008). Determining the links between anatomic, neurophysiologic, and behavioral indices of multisensory processes is necessary to understand the conditions under which a redundant signal effect is observable.

The reality of direct connections from a cortical area considered as unisensory to another one of different modality is a paradox for hierarchical models of sensory processing (Maunsell and Van Essen 1983; Felleman and Van Essen 1991). The most recent findings provided evidence that multisensory interactions can occur shortly after response onset, at the lowest processing stages (see previous paragraphs). These new elements have to be considered and included in view of the sensory system organization. Obviously, it is possible that some connections mediating early-stage multisensory connections have not yet been identified by anatomical methods.

Inside a sensory system, the hierarchy relationship between cortical areas have been defined by the nature of the connections in terms of feedforward or feedback although the role of these connections is only partially understood (Salin and Bullier 1995; Bullier 2006). Recent results suggest that multisensory convergence in unisensory areas can intervene with stages of information processing of low levels, through feedback and feedforward circuits (Schroeder et al. 2001; Schroeder and Foxe 2002; Fu et al. 2003; Cappe and Barone 2005). Accordingly, anatomical methods alone are not sufficient to definitely determine the functional distinction of any connections in terms of feedforward–feedback nature, and cannot be used to establish a hierarchy between functional areas of different systems.

This review highlights that both higher-order association areas and lower-order cortical areas are multisensory in nature and that the thalamus could also play a role in multisensory processing. Figure 2.1 summarizes and represents schematically the possible scenarios for multisensory integration through anatomical pathways. First, as traditionally proposed, information is processed from the primary “unisensory” cortical areas to “multisensory” association cortical areas, and finally, the premotor and motor cortical areas in a hierarchical way (Figure 2.1a). In these multisensory association areas, the strength and the latencies of neuronal responses are affected by the nature of the stimuli (e.g., Avillac et al. 2007; Romanski 2007; Bizley et al. 2007). Second, recent evidence demonstrated the existence of multisensory interaction at the first level of cortical processing of the information (Figure 2.1b). Third, as we described in this review, the thalamus by its numerous connections could play a role in this processing (Figure 2.1c). Altogether, this model represents the different alternative pathways for multisensory integration. These multiple pathways, which coexist (Figure 2.1d), may be useful to allow different paths according to the task and/or to mediate information of different natures (see Wang et al. 2008 for recent evidence of the influence of a perceptual task on neuronal responses).

FIGURE 2.1. Hypothetical scenarios for multisensory and motor integration through anatomically identified pathways.

FIGURE 2.1

Hypothetical scenarios for multisensory and motor integration through anatomically identified pathways. (a) High-level cortical areas as a pathway for multisensory and motor integration. (b) Low-level cortical areas as a pathway for multisensory integration. (more...)

Taken together, the data reviewed here provide evidence for anatomical pathways possibly involved in multisensory integration at low levels of information processing in the primate and argue against a strict hierarchical model. An alternative for multisensory integration appears to be the thalamus. Indeed, as demonstrated in this chapter, the thalamus, thanks to its multiple connections, appears to belong to a cortico–thalamo–cortical loop. This allows us to consider that it may have a key role in multisensory integration. Finally, higher order association cortical areas, lower order cortical areas, as well as the thalamus have now been shown to be part of multisensory integration. The question is now to determine how this system of multisensory integration is organized and how the different parts of the system communicate to allow a unified view of the perception of the world.

2.5. CONCLUSIONS

Obviously, we are just beginning to understand the complexity of interactions in the sensory systems and between the sensory and the motor systems. More work is needed in both the neural and perceptual domains. At the neural level, additional studies are needed to understand the extent and hierarchical organization of multisensory interactions. At the perceptual level, further experiments should explore the conditions necessary for cross-modal binding and plasticity, and investigate the nature of the information transfer between sensory systems. Such studies will form the basis for a new comprehension of how the different sensory and/or motor systems function together.

ACKNOWLEDGMENTS

This study was supported by the following grants: the CNRS ATIP program (to P.B.), the Swiss National Science Foundation, grants 31-61857.00 (to E.M.R.) and 310000-110005 (to E.M.R.), the Swiss National Science Foundation Center of Competence in Research on “Neural Plasticity and Repair” (to E.M.R.).

REFERENCES

  1. Amedi A, Jacobson G, Hendler T, Malach R, Zohary E. Convergence of visual and tactile shape processing in the human lateral occipital complex. Cerebral Cortex. 2002;12:1202–12. [PubMed: 12379608]
  2. Amedi A, Stern W.M, Camprodon J.A, editors. et al. Shape conveyed by visual-to-auditory sensory substitution activates the lateral occipital complex. Nature Neuroscience. 2007;10:687–9. [PubMed: 17515898]
  3. Andersen R.A, Asanuma C, Essick G, Siegel R.M. Corticocortical connections of anatomically and physiologically defined subdivisions within the inferior parietal lobule. Journal of Comparative Neurology. 1990;296:65–113. [PubMed: 2358530]
  4. Andersen R.A, Snyder L.H, Bradley D.C, Xing J. Multimodal representation of space in the posterior parietal cortex and its use in planning movements. Annual Review of Neuroscience. 1997;20:303–30.. [PubMed: 9056716]
  5. Avillac M, Ben Hamed S, Duhamel J.R. Multisensory integration in the ventral intraparietal area of the macaque monkey. Journal of Neuroscience. 2007;27:1922–32. [PubMed: 17314288]
  6. Barbas H. Pattern in the laminar origin of corticocortical connections. Journal of Comparative Neurology. 1986;252:415–22. [PubMed: 3793985]
  7. Barbas H, Pandya D.N. Architecture and frontal cortical connections of the premotor cortex (area 6) in the rhesus monkey. Journal of Comparative Neurology. 1987;256:211–28. [PubMed: 3558879]
  8. Barone P, Joseph J.P. Role of the dorsolateral prefrontal cortex in organizing visually guided behavior. Brain, Behavior and Evolution. 1989;33:132–5. [PubMed: 2758291]
  9. Barraclough N.E, Xiao D, Baker C.I, Oram M.W, Perrett D.I. Integration of visual and auditory information by superior temporal sulcus neurons responsive to the sight of actions. Journal of Cognitive Neuroscience. 2005;17:377–91. [PubMed: 15813999]
  10. Baylis G.C, Rolls E.T, Leonard C.M. Functional subdivisions of the temporal lobe neocortex. Journal of Neuroscience. 1987;7:330–42. [PubMed: 3819816]
  11. Bell A.H, Meredith M.A, Van Opstal A.J, Munoz D.P. Crossmodal integration in the primate superior colliculus underlying the preparation and initiation of saccadic eye movements. Journal of Neurophysiology. 2005;93:3659–73. [PubMed: 15703222]
  12. Benevento L.A, Fallon J, Davis B.J, Rezak M. Auditory–visual interaction in single cells in the cortex of the superior temporal sulcus and the orbital frontal cortex of the macaque monkey. Experimental Neurology. 1977;57:849–72. [PubMed: 411682]
  13. Bignall K.E. Auditory input to frontal polysensory cortex of the squirrel monkey: Possible pathways. Brain Research. 1970;19:77–86. [PubMed: 4985445]
  14. Bizley J.K, King A.J. Visual–auditory spatial processing in auditory cortical neurons. Brain Research. 2008;1242:24–36. [PMC free article: PMC4340571] [PubMed: 18407249]
  15. Bizley J.K, Nodal F.R, Bajo V.M, Nelken I, King A.J. Physiological and anatomical evidence for multisensory interactions in auditory cortex. Cerebral Cortex. 2007;17:2172–89. [PubMed: 17135481]
  16. Blatt G.J, Andersen R.A, Stoner G.R. Visual receptive field organization and cortico-cortical connections of the lateral intraparietal area (area LIP) in the macaque. Journal of Comparative Neurology. 1990;299:421–45. [PubMed: 2243159]
  17. Bremmer F, Klam F, Duhamel J.R, Ben Hamed S, Graf W. Visual-vestibular interactive responses in the macaque ventral intraparietal area (VIP) European Journal of Neuroscience. 2002;16:1569–86. [PubMed: 12405971]
  18. Bruce C, Desimone R, Gross C.G. Visual properties of neurons in a polysensory area in superior temporal sulcus of the macaque. Journal of Neurophysiology. 1981;46:369–84. [PubMed: 6267219]
  19. Budinger E, Scheich H. Anatomical connections suitable for the direct processing of neuronal information of different modalities via the rodent primary auditory cortex (review) Hearing Research. 2009;258:16–27. [PubMed: 19446016]
  20. Budinger E, Heil P, Hess A, Scheich H. Multisensory processing via early cortical stages: Connections of the primary auditory cortical field with other sensory systems. Neuroscience. 2006;143:1065–83. [PubMed: 17027173]
  21. Bullier J. What is feed back? In: van Hemmen J.L, Sejnowski T.J, editors. 23 Problems in Systems Neuroscience, New York: Oxford University Press; 2006. pp. 103–132.
  22. Calvert G.A. Crossmodal processing in the human brain: Insights from functional neuroimaging studies (review) Cerebral Cortex. 2001;11:1110–23. [PubMed: 11709482]
  23. Calvert G, Spence C, Stein B.E, editors. The Handbook of Multisensory Processes. Cambridge, MA: MIT Press; 2004.
  24. Campi K.L, Bales K.L, Grunewald R, Krubitzer L. Connections of auditory and visual cortex in the prairie vole (Microtus ochrogaster): Evidence for multisensory processing in primary sensory areas. Cerebral Cortex. 2010;20:89–108. [PMC free article: PMC2792189] [PubMed: 19395525]
  25. Cappe C, Barone P. Heteromodal connections supporting multisensory integration at low levels of cortical processing in the monkey. European Journal of Neuroscience. 2005;22:2886–902. [PubMed: 16324124]
  26. Cappe C, Morel A, Rouiller E.M. Thalamocortical and the dual pattern of corticothalamic projections of the posterior parietal cortex in macaque monkeys. Neuroscience. 2007;146:1371–87. [PubMed: 17395383]
  27. Cappe C, Rouiller E.M, Barone P. Multisensory anatomic pathway (review) Hearing Research. 2009a;258:28–36. [PubMed: 19410641]
  28. Cappe C, Thut G, Romei V, Murray M.M. Selective integration of auditory-visual looming cues by humans. Neuropsychologia. 2009b;47:1045–52. [PubMed: 19041883]
  29. Cappe C, Morel A, Barone P, Rouiller E.M. The thalamocortical projection systems in primate: An anatomical support for multisensory and sensorimotor integrations. Cerebral Cortex. 2009c;19:2025–37. [PMC free article: PMC2722423] [PubMed: 19150924]
  30. Cappe C, Murray M.M, Barone P, Rouiller E.M. Multisensory facilitation of behavior in monkeys: Effects of stimulus intensity. Journal of Cognitive Neuroscience. 2010;22:2850–63. [PubMed: 20044892]
  31. Cohen Y.E, Russ B.E, Gifford G.W. Auditory processing in the posterior parietal cortex (review) Behavioral and Cognitive Neuroscience Reviews. (3rd) 2005;4:218–31. [PubMed: 16510894]
  32. Cohen Y.E, Theunissen F, Russ B.E, Gill P. Acoustic features of rhesus vocalizations and their representation in the ventrolateral prefrontal cortex. Journal of Neurophysiology. 2007;97:1470–84. [PubMed: 17135477]
  33. Colby C.L, Goldberg M.E. Space and attention in parietal cortex (review) Annual Review of Neuroscience. 1999;22:319–49. [PubMed: 10202542]
  34. Crick F, Koch C. Constraints on cortical and thalamic projections: The no-strong-loops hypothesis. Nature. 1998;391:245–50. [PubMed: 9440687]
  35. Cusick C.G, Seltzer B, Cola M, Griggs E. Chemoarchitectonics and corticocortical terminations within the superior temporal sulcus of the rhesus monkey: Evidence for subdivisions of superior temporal polysensory cortex. Journal of Comparative Neurology. 1995;360:513–35. [PubMed: 8543656]
  36. Darian-Smith C, Tan A, Edwards S. Comparing thalamocortical and corticothalamic microstructure and spatial reciprocity in the macaque ventral posterolateral nucleus (VPLc) and medial pulvinar. Journal of Comparative Neurology. 1999;410:211–34. [PubMed: 10414528]
  37. de la Mothe L.A, Blumell S, Kajikawa Y, Hackett T.A. Cortical connections of the auditory cortex in marmoset monkeys: Core and medial belt regions. Journal of Comparative Neurology. 2006a;496:27–71. [PubMed: 16528722]
  38. de la Mothe L.A, Blumell S, Kajikawa Y, Hackett T.A. Thalamic connections of the auditory cortex in marmoset monkeys: Core and medial belt regions. Journal of Comparative Neurology. 2006b;496:72–96. [PMC free article: PMC4419740] [PubMed: 16528728]
  39. Desimone R, Gross C.G. Visual areas in the temporal cortex of the macaque. Brain Research. 1979;178:363–80. [PubMed: 116712]
  40. Disbrow E, Litinas E, Recanzo G.H, Padberg J, Krubitzer L. Cortical connections of the second somatosensory area and the parietal ventral area in macaque monkeys. Journal of Comparative Neurology. 2003;462:382–99. [PubMed: 12811808]
  41. Duhamel J.R, Colby C.L, Goldberg M.E. Ventral intraparietal area of the macaque: Congruent visual and somatic response properties. Journal of Neurophysiology. 1998;79:126–36. [PubMed: 9425183]
  42. Ettlinger G, Wilson W.A. Cross-modal performance: Behavioural processes, phylogenetic considerations and neural mechanisms (review) Behavioural Brain Research. 1990;40:169–92. [PubMed: 2178346]
  43. Falchier A, Clavagnier S, Barone P, Kennedy H. Anatomical evidence of multimodal integration in primate striate cortex. Journal of Neuroscience. 2002;22:5749–59. [PubMed: 12097528]
  44. Falchier A, Schroeder C.E, Hackett T.A, editors. et al. Low level intersensory connectivity as a fundamental feature of neocortex. Cerebral Cortex. 2010;20:1529–38. [PMC free article: PMC2882821] [PubMed: 19875677]
  45. Felleman D.J, Van Essen D.C. Distributed hierarchical processing in the primate cerebral cortex. Cerebral Cortex. 1991;1:1–47. [PubMed: 1822724]
  46. Fogassi L, Gallese V, Fadiga L, Luppino G, Matelli M, Rizzolatti G. Coding of peripersonal space in inferior premotor cortex (area F4) Journal of Neurophysiology. 1996;76:141–57. [PubMed: 8836215]
  47. Fort A, Delpuech C, Pernier J, Giard M.H. Dynamics of corticosubcortical cross-modal operations involved in audio-visual object detection in humans. Cerebral Cortex. 2002;12:1031–39. [PubMed: 12217966]
  48. Forster B, Cavina-Pratesi C, Aglioti S.M, Berlucchi G. Redundant target effect and intersensory facilitation from visual-tactile interactions in simple reaction time. Experimental Brain Research. 2002;143:480–487. [PubMed: 11914794]
  49. Foxe J.J, Morocz I.A, Murray M.M, Higgins B.A, Javitt D.C, Schroeder C.E. Multisensory auditory-somatosensory interactions in early cortical processing revealed by high-density electrical mapping. Brain Research. Cognitive Brain Research. 2000;10:77–83. [PubMed: 10978694]
  50. Foxe J.J, Wylie G.R, Martinez A, editors. et al. Auditory–somatosensory multisensory processing in auditory association cortex: An fMRI study. Journal of Neurophysiology. 2002;88:540–3. [PubMed: 12091578]
  51. Frens M.A, Van Opstal A.J. Visual–auditory interactions modulate saccade-related activity in monkey superior colliculus. Brain Research Bulletin. 1998;46:211–24. [PubMed: 9667814]
  52. Fries P, Neuenschwander S, Engel A.K, Goebel R, Singer W. Rapid feature selective neuronal synchronization through correlated latency shifting. Nature Neuroscience. 2001;4:194–200. [PubMed: 11175881]
  53. Fu K.M, Johnston T.A, Shah A.S, editors. et al. Auditory cortical neurons respond to somatosensory stimulation. Journal of Neuroscience. 2003;23:7510–5. [PubMed: 12930789]
  54. Fuster J.M. The prefrontal cortex—an update: Time is of the essence (review) Neuron. 2001;30:319–33. [PubMed: 11394996]
  55. Fuster J.M, Bodner M, Kroger J.K. Cross-modal and cross-temporal association in neurons of frontal cortex. Nature. 2000;405:347–51. [PubMed: 10830963]
  56. Gaffan D, Harrison S. Auditory–visual associations, hemispheric specialization and temporal- frontal interaction in the rhesus monkey. Brain. 1991;114:2133–44. [PubMed: 1933238]
  57. Galletti C, Gamberini M, Kutz D.F, Fattori P, Luppino G, Matelli M. The cortical connections of area V6: An occipito-parietal network processing visual information. European Journal of Neuroscience. 2001;13:1572–88. [PubMed: 11328351]
  58. Gattass R, Oswaldo-Cruz E, Sousa A.P. Visual receptive fields of units in the pulvinar of cebus monkey. Brain Research. 1979;160:413–30. [PubMed: 217477]
  59. Ghazanfar A.A. The multisensory roles for auditory cortex in primate vocal communication (review) Hearing Research. 2009;258:113–20. [PMC free article: PMC2787678] [PubMed: 19371776]
  60. Ghazanfar A.A, Schroeder C.E. Is neocortex essentially multisensory? (review) Trends in Cognitive Sciences. 2006;10:278–85. [PubMed: 16713325]
  61. Ghazanfar A.A, Maier J.X, Hoffman K.L, Logothetis N.K. Multisensory integration of dynamic faces and voices in rhesus monkey auditory cortex. Journal of Neuroscience. 2005;25:5004–12. [PubMed: 15901781]
  62. Ghazanfar A.A, Chandrasekaran C, Logothetis N.K. Interactions between the superior temporal sulcus and auditory cortex mediate dynamic face/voice integration in rhesus monkeys. Journal of Neuroscience. 2008;28:4457–69. [PMC free article: PMC2663804] [PubMed: 18434524]
  63. Giard M.H, Peronnet F. Auditory–visual integration during multimodal object recognition in humans: A behavioral and electrophysiological study. Journal of Cognitive Neuroscience. 1999;11:473–90. [PubMed: 10511637]
  64. Gifford G.W, Cohen Y.E. Effect of a central fixation light on auditory spatial responses in area LIP. Journal of Neurophysiology. (3rd) 2004;91:2929–33. [PubMed: 14749308]
  65. Gingras G, Rowland B.A, Stein B.E. The differing impact of multisensory and unisensory integration on behavior. Journal of Neuroscience. 2009;29:4897–902. [PMC free article: PMC2678542] [PubMed: 19369558]
  66. Giray M, Ulrich R. Motor coactivation revealed by response force in divided and focused attention. Journal of Experimental Psychology. Human Perception and Performance. 1993;19:1278–91. [PubMed: 8294892]
  67. Gondan M, Niederhaus B, Rösler F, Röder B. Multisensory processing in the redundant-target effect: A behavioral and event-related potential study. Perception & Psychophysics. 2005;67:713–26. [PubMed: 16134464]
  68. Gottlieb J. From thought to action: The parietal cortex as a bridge between perception, action, and cognition (review) Neuron. 2007;53:9–16. [PubMed: 17196526]
  69. Graziano M.S, Yap G.S, Gross C.G. Coding of visual space by premotor neurons. Science. 1994;266:1054–7. [PubMed: 7973661]
  70. Graziano M.S, Reiss L.A, Gross C.G. A neuronal representation of the location of nearby sounds. Nature. 1999;397:428–30. [PubMed: 9989407]
  71. Guillery R.W. Anatomical evidence concerning the role of the thalamus in corticocortical communication: A brief review. Journal of Anatomy. 1995;187:583–92. [PMC free article: PMC1167461] [PubMed: 8586557]
  72. Gutierrez C, Cola M.G, Seltzer B, Cusick C. Neurochemical and connectional organization of the dorsal pulvinar complex in monkeys. Journal of Comparative Neurology. 2000;419:61–86. [PubMed: 10717640]
  73. Hackett T.A, Stepniewska I, Kaas J.H. Thalamocortical connections of the parabelt auditory cortex in macaque monkeys. Journal of Comparative Neurology. 1998;400:271–86. [PubMed: 9766404]
  74. Hackett T.A, Stepniewska I, Kaas J.H. Prefrontal connections of the parabelt auditory cortex in macaque monkeys. Brain Research. 1999;817:45–58. [PubMed: 9889315]
  75. Hackett T.A, de La Mothe L.A, Ulbert I, Karmos G, Smiley J, Schroeder C.E. Multisensory convergence in auditory cortex: II. Thalamocortical connections of the caudal superior temporal plane. Journal of Comparative Neurology. 2007;502:924–52. [PubMed: 17444488]
  76. Hagen M.C, Franzén O, McGlone F, Essick G, Dancer C, Pardo J.V. Tactile motion activates the human middle temporal/V5 (MT/V5) complex. European Journal of Neuroscience. 2002;16:957–64. [PubMed: 12372032]
  77. Hall A.J, Lomber S.G. Auditory cortex projections target the peripheral field representation of primary visual cortex. Experimental Brain Research. 2008;190:413–30. [PubMed: 18641978]
  78. Hecht D, Reiner M, Karni A. Enhancement of response times to bi- and tri-modal sensory stimuli during active movements. Experimental Brain Research. 2008;185:655–65. [PubMed: 17992522]
  79. Heffner R.S, Heffner H.E. Visual factors in sound localization in mammals. Journal of Comparative Neurology. 1992;317:219–32. [PubMed: 1577997]
  80. Hikosaka K, Iwai E, Saito H, Tanaka K. Polysensory properties of neurons in the anterior bank of the caudal superior temporal sulcus of the macaque monkey. Journal of Neurophysiology. 1988;60:1615–37. [PubMed: 2462027]
  81. Huffman K.J, Krubitzer L. Area 3a: topographic organization and cortical connections in marmoset monkeys. Cerebral Cortex. 2001;11:849–67. [PubMed: 11532890]
  82. Innocenti G.M, Berbel P, Clarke S. Development of projections from auditory to visual areas in the cat. Journal of Comparative Neurology. 1988;272:242–59. [PubMed: 2456313]
  83. James T.W, Humphrey G.K, Gati J.S, Servos P, Menon R.S, Goodale M.A. Haptic study of three-dimensional objects activates extrastriate visual areas. Neuropsychologia. 2002;40:1706–14. [PubMed: 11992658]
  84. Jepma M, Wagenmakers E.J, Band G.P, Nieuwenhuis S. The effects of accessory stimuli on information processing: Evidence from electrophysiology and a diffusion model analysis. Journal of Cognitive Neuroscience. 2009;21:847–64. [PubMed: 18702584]
  85. Jones E.G. Viewpoint: The core and matrix of thalamic organization. Neuroscience. 1998;85:331–45. [PubMed: 9622234]
  86. Joseph J.P, Barone P. Prefrontal unit activity during a delayed oculomotor task in the monkey. Experimental Brain Research. 1987;67:460–8. [PubMed: 3653308]
  87. Kaas J.H, Collins C.E. Evolving ideas of brain evolution. Nature. 2001;411:141–2. [PubMed: 11346770]
  88. Kaas J, Collins C.E. The resurrection of multisensory cortex in primates: connection patterns that integrates modalities. In: Calvert G, Spence C, Stein B.E, editors. The Handbook of Multisensory Processes. Cambridge, MA: MIT Press; 2004. pp. 285–93.
  89. Kaas J.H, Hackett T.A. Subdivisions of auditory cortex and processing streams in primates. Proceedings of the National Academy of Sciences of the United States of America. 2000;97:11793–9. [PMC free article: PMC34351] [PubMed: 11050211]
  90. Kaas J.H, Morel A. Connections of visual areas of the upper temporal lobe of owl monkeys: The MT crescent and dorsal and ventral subdivisions of FST. Journal of Neuroscience. 1993;13:534–46. [PubMed: 8381166]
  91. Kayser C, Logothetis N.K. Directed interactions between auditory and superior temporal cortices and their role in sensory integration (review) Frontiers in Integrative Neuroscience. 2009;3:7. doi: 10.3389/neuro.07.007.2009. [PMC free article: PMC2691153] [PubMed: 19503750]
  92. Kayser C, Petkov C.I, Augath M, Logothetis N.K. Integration of touch and sound in auditory cortex. Neuron. 2005;48:373–84. [PubMed: 16242415]
  93. Kayser C, Petkov C.I, Augath M, Logothetis N.K. Functional imaging reveals visual modulation of specific fields in auditory cortex. Journal of Neuroscience. 2007;27:1824–35. [PubMed: 17314280]
  94. Kayser C, Petkov C.I, Logothetis N.K. Visual modulation of neurons in auditory cortex. Cerebral Cortex. 2008;18:1560–74. [PubMed: 18180245]
  95. Komura Y, Tamura R, Uwano T, Nishijo H, Ono T. Auditory thalamus integrates visual inputs into behavioral gains. Nature Neuroscience. 2005;8:1203–9. [PubMed: 16116444]
  96. Krubitzer L.A, Kaas J.H. The organization and connections of somatosensory cortex in marmosets. Journal of Neuroscience. 1990;10:952–74. [PubMed: 2108231]
  97. Lakatos P, Chen C.M, O’Connell M.N, Mills A, Schroeder C.E. Neuronal oscillations and multisensory interaction in primary auditory cortex. Neuron. 2007;53:279–92. [PMC free article: PMC3717319] [PubMed: 17224408]
  98. Lamarre Y, Busby L, Spidalieri G. Fast ballistic arm movements triggered by visual, auditory, and somesthetic stimuli in the monkey: I. Activity of precentral cortical neurons. Journal of Neurophysiology. 1983;50:1343–58. [PubMed: 6663331]
  99. Laurienti P.J, Kraft R.A, Maldjian J.A, Burdette J.H, Wallace M.T. Semantic congruence is a critical factor in multisensory behavioral performance. Experimental Brain Research. 2004;158:405–14. [PubMed: 15221173]
  100. Lewis J.W, Van Essen D.C. Corticocortical connections of visual, sensorimotor, and multimodal processing areas in the parietal lobe of the macaque monkey. Journal of Comparative Neurology. 2000;428:112–37. [PubMed: 11058227]
  101. Linden J.F, Grunewald A, Andersen R.A. Responses to auditory stimuli in macaque lateral intraparietal area: II. Behavioral modulation. Journal of Neurophysiology. 1999;82:343–58. [PubMed: 10400963]
  102. Lovelace C.T, Stein B.E, Wallace M.T. An irrelevant light enhances auditory detection in humans: A psychophysical analysis of multisensory integration in stimulus detection. Brain Research. Cognitive Brain Research. 2003;17:447–53. [PubMed: 12880914]
  103. Martuzzi R, Murray M.M, Michel C.M. et al. Multisensory interactions within human primary cortices revealed by BOLD dynamics. Cerebral Cortex. 2007;17:1672–9. [PubMed: 16968869]
  104. Maunsell J.H, Van Essen D.C. The connections of the middle temporal visual area (MT) and their relationship to a cortical hierarchy in the macaque monkey. Journal of Neuroscience. 1983;3:2563–86. [PubMed: 6655500]
  105. Miller J. Divided attention: Evidence for coactivation with redundant signals. Cognitive Psychology. 1982;14:247–79. [PubMed: 7083803]
  106. Miller J, Ulrich R, Lamarre Y. Locus of the redundant-signals effect in bimodal divided attention: A neurophysiological analysis. Perception & Psychophysics. 2001;63:555–62. [PubMed: 11414141]
  107. Milner B, Petrides M, Smith M.L. Frontal lobes and the temporal organization of memory. Human Neurobiology. 1985;4:137–42. [PubMed: 4066424]
  108. Molholm S, Ritter W, Murray M.M, Javitt D.C, Schroeder C.E, Foxe J.J. Multisensory auditory–visual interactions during early sensory processing in humans: A high-density electrical mapping study. Brain Research. Cognitive Brain Research. 2002;14:115–28. [PubMed: 12063135]
  109. Mordkoff J.T, Miller J, Roch A.C. Absence of coactivation in the motor component: Evidence from psychophysiological measures of target detection. Journal of Experimental Psychology. Human Perception and Performance. 1996;22:25–41. [PubMed: 8742250]
  110. Morel A, Liu J, Wannier T, Jeanmonod D, Rouiller E.M. Divergence and convergence of thalamocortical projections to premotor and supplementary motor cortex: A multiple tracing study in macaque monkey. European Journal of Neuroscience. 2005;21:1007–29. [PubMed: 15787707]
  111. Murray M.M, Molholm S, Michel C.M, editors. et al. Grabbing your ear: Rapid auditory-somatosensory multisensory interactions in low-level sensory cortices are not constrained by stimulus alignment. Cerebral Cortex. 2005;15:963–74. [PubMed: 15537674]
  112. Palmer S.M, Rosa M.G. A distinct anatomical network of cortical areas for analysis of motion in far peripheral vision. European Journal of Neuroscience. 2006;24:2389–405. [PubMed: 17042793]
  113. Pandya D.N, Seltzer B. Intrinsic connections and architectonics of posterior parietal cortex in the rhesus monkey. Journal of Comparative Neurology. 1982;204:196–210. [PubMed: 6276450]
  114. Petrides M, Iversen S.D. Cross-modal matching and the primate frontal cortex. Science. 1976;192:1023–4. [PubMed: 818708]
  115. Poremba A, Saunders R.C, Crane A.M, Cook M, Sokoloff L, Mishkin M. Functional mapping of the primate auditory system. Science. 2003;299:568–72. [PubMed: 12543977]
  116. Raab D.H. Statistical facilitation of simple reaction times. Transactions of the New York Academy of Sciences. 1962;24:574–90. [PubMed: 14489538]
  117. Rizzolatti G, Fogassi L, Gallese V. Parietal cortex: From sight to action (review) Current Opinion in Neurobiology. 1997;7:562–7. [PubMed: 9287198]
  118. Rockland K.S. Two types of corticopulvinar terminations: Round (type 2) and elongate (type 1) Journal of Comparative Neurology. 1996;368:57–87. [PubMed: 8725294]
  119. Rockland K.S, Ojima H. Multisensory convergence in calcarine visual areas in macaque monkey. International Journal of Psychophysiology. 2003;50:19–26. [PubMed: 14511833]
  120. Romanski L.M. Domain specificity in the primate prefrontal cortex (review) Cognitive, Affective & Behavioral Neuroscience. 2004;4:421–9. [PubMed: 15849888]
  121. Romanski L.M. Representation and integration of auditory and visual stimuli in the primate ventral lateral prefrontal cortex. Cerebral Cortex. 2007;17 Suppl. no. 1:i61–9. [PMC free article: PMC2778283] [PubMed: 17634387]
  122. Romanski L.M, Giguere M, Bates J.F, Goldman-Rakic P.S. Topographic organization of medial pulvinar connections with the prefrontal cortex in the rhesus monkey. Journal of Comparative Neurology. 1997;379:313–32. [PubMed: 9067827]
  123. Romanski L.M, Bates J.F, Goldman-Rakic P.S. Auditory belt and parabelt projections to the prefrontal cortex in the rhesus monkey. Journal of Comparative Neurology. 1999;403:141–57. [PubMed: 9886040]
  124. Romanski L.M, Goldman-Rakic P.S. An auditory domain in primate prefrontal cortex. Nature Neuroscience. 2002;5:15–6. [PMC free article: PMC2793092] [PubMed: 11753413]
  125. Romei V, Murray M.M, Merabet L.B, Thut G. Occipital transcranial magnetic stimulation has opposing effects on visual and auditory stimulus detection: Implications for multisensory interactions. Journal of Neuroscience. 2007;27:11465–72. [PubMed: 17959789]
  126. Rouiller E.M, Durif C. The dual pattern of corticothalamic projection of the primary auditory cortex in macaque monkey. Neuroscience Letters. 2004;358:49–52. [PubMed: 15016432]
  127. Rouiller E.M, Tanné J, Moret V, Kermadi I, Boussaoud D, Welker E. Dual morphology and topography of the corticothalamic terminals originating from the primary, supplementary motor, and dorsal premotor cortical areas in macaque monkeys. Journal of Comparative Neurology. 1998;396:169–85. [PubMed: 9634140]
  128. Rouiller E.M, Welker E. A comparative analysis of the morphology of corticothalamic projections in mammals. Brain Research Bulletin. 2000;53:727–41. [PubMed: 11179837]
  129. Rouiller E.M, Wannier T, Morel A. The dual pattern of corticothalamic projection of the premotor cortex in macaque monkeys. Thalamus & Related Systems. 2003;2:189–97.
  130. Russ B.E, Kim A.M, Abrahamsen K.L, Kiringoda R, Cohen Y.E. Responses of neurons in the lateral intraparietal area to central visual cues. Experimental Brain Research. 2006;174:712–27. [PubMed: 16738908]
  131. Sadato N, Pascual-Leone A, Grafman J, editors. et al. Activation of the primary visual cortex by Braille reading in blind subjects. Nature. 1996;380:526–8. [PubMed: 8606771]
  132. Salin P.A, Bullier J. Corticocortical connections in the visual system: Structure and function. Physiological Reviews. 1995;75:107–54. [PubMed: 7831395]
  133. Sathian K, Zangaladze A. Feeling with the mind’s eye: Contribution of visual cortex to tactile perception (review) Behavioural Brain Research. 2002;135:127–32. [PubMed: 12356442]
  134. Schall J.D, Morel A, King D.J, Bullier J. Topography of visual cortex connections with frontal eye field in macaque: Convergence and segregation of processing streams. Journal of Neuroscience. 1995;15:4464–87. [PubMed: 7540675]
  135. Schlack A, Sterbing-D’Angelo S.J, Hartung K, Hoffmann K.P, Bremmer F. Multisensory space representations in the macaque ventral intraparietal area. Journal of Neuroscience. 2005;25:4616–25. [PubMed: 15872109]
  136. Schroeder C.E, Foxe J.J. The timing and laminar profile of converging inputs to multisensory areas of the macaque neocortex. Cognitive Brain Research. 2002;14:187–98. [PubMed: 12063142]
  137. Schroeder C.E, Lindsley R.W, Specht C, Marcovici A, Smiley J.F, Javitt D.C. Somatosensory input to auditory association cortex in the macaque monkey. Journal of Neurophysiology. 2001;85:1322–7. [PubMed: 11248001]
  138. Schwartz M.L, Dekker J.J, Goldman-Rakic P.S. Dual mode of corticothalamic synaptic termination in the mediodorsal nucleus of the rhesus monkey. Journal of Comparative Neurology. 1991;309:289–304. [PubMed: 1918440]
  139. Seltzer B, Pandya D.N. Parietal, temporal, and occipital projections to cortex of the superior temporal sulcus in the rhesus monkey: A retrograde tracer study. Journal of Comparative Neurology. 1994;343:445–63. [PubMed: 8027452]
  140. Sherman S.M. The thalamus is more than just a relay. Current Opinion in Neurobiology. 2007;17:417–22. [PMC free article: PMC2753250] [PubMed: 17707635]
  141. Sherman S.M, Guillery R.W. The role of the thalamus in the flow of information to the cortex. Philosophical Transactions of the Royal Society of London. Series B, Biological Sciences. 2002;357:1695–708. [PMC free article: PMC1693087] [PubMed: 12626004]
  142. Sherman S.M, Guillery R.W. Exploring the Thalamus and Its Role in Cortical Function. Cambridge: MIT Press; 2005.
  143. Shipp S. The functional logic of cortico-pulvinar connections. Philosophical Transactions of the Royal Society of London. Series B, Biological Sciences. 2003;358:1605–24. [PMC free article: PMC1693262] [PubMed: 14561322]
  144. Smiley J.F, Hackett T.A, Ulbert I, Karmas G, Lakatos P, Javitt D.C, Schroeder C.E. Multisensory convergence in auditory cortex, I. Cortical connections of the caudal superior temporal plane in macaque monkeys. Journal of Comparative Neurology. 2007;502:894–923. [PubMed: 17447261]
  145. Smiley J.F, Falchier A. Multisensory connections of monkey auditory cerebral cortex. Hearing Research. 2009;258:37–46. [PMC free article: PMC2788085] [PubMed: 19619628]
  146. Sperdin H, Cappe C, Foxe J.J, Murray M.M. Early, low-level auditory-somatosensory multisensory interactions impact reaction time speed. Frontiers in Integrative Neuroscience. 2009;3:2. doi:10.3389/neuro.07.002.2009. [PMC free article: PMC2659167] [PubMed: 19404410]
  147. Stein B.E, Meredith M.A. The Merging of the Senses. Cambridge, MA: MIT Press; 1993.
  148. Stein B.E, Meredith M.A, Huneycutt W.S, Mcdade L. Behavioral indices of multisensory integration: Orientation to visual cues is affected by auditory stimuli. Journal of Cognitive Neuroscience. 1989;1:12–24. [PubMed: 23968407]
  149. Suzuki H. Distribution and organization of visual and auditory neurons in the monkey prefrontal cortex. Vision Research. 1985;25:465–9. [PubMed: 4024465]
  150. Tanné-Gariépy J, Rouiller E.M, Boussaoud D. Pariinputs to dorsal versus ventral premotor areas in the macaque monkey: Evidence for largely segregated visuomotor pathways. Experimental Brain Research. 2002;145:91–103. [PubMed: 12070749]
  151. Taktakishvili O, Sivan-Loukianova E, Kultas-Ilinsky K, Ilinsky I.A. Posterior parietal cortex projections to the ventral lateral and some association thalamic nuclei in Macaca mulatta. Brain Research Bulletin. 2002;59:135–50. [PubMed: 12379444]
  152. Wallace M.T, Ramachandran R, Stein B.E. A revised view of sensory cortical parcellation. Proceedings of the National Academy of Sciences of the United States of America. 2004;101:2167–72. [PMC free article: PMC357070] [PubMed: 14766982]
  153. Wang Y, Celebrini S, Trotter Y, Barone P. Visuo–auditory interactions in the primary visual cortex of the behaving monkey: Electrophysiological evidence. BMC Neuroscience. 2008;9:79. [PMC free article: PMC2527609] [PubMed: 18699988]
  154. Yirmiya R, Hocherman S. Auditory- and movement-related neural activity interact in the pulvinar of the behaving rhesus monkey. Brain Research. 1987;402:93–102. [PubMed: 3828792]
  155. Zampini M, Torresan D, Spence C, Murray M.M. Auditory-somatosensory multisensory interactions in front and rear space. Neuropsychologia. 2007;45:1869–77. [PubMed: 17291546]
Copyright © 2012 by Taylor & Francis Group, LLC.
Bookshelf ID: NBK92866PMID: 22593888

Views

  • PubReader
  • Print View
  • Cite this Page

Other titles in this collection

Related information

  • PMC
    PubMed Central citations
  • PubMed
    Links to PubMed

Similar articles in PubMed

See reviews...See all...

Recent Activity

Your browsing activity is empty.

Activity recording is turned off.

Turn recording back on

See more...