NCBI Bookshelf. A service of the National Library of Medicine, National Institutes of Health.

Murray MM, Wallace MT, editors. The Neural Bases of Multisensory Processes. Boca Raton (FL): CRC Press/Taylor & Francis; 2012.

Cover of The Neural Bases of Multisensory Processes

The Neural Bases of Multisensory Processes.

Show details

Chapter 3What Can Multisensory Processing Tell Us about the Functional Organization of Auditory Cortex?

and .


The traditional view of sensory processing is that the pooling and integration of information across different modalities takes place in specific areas of the brain only after extensive processing within modality-specific subcortical and cortical regions. This seems like a logical arrangement because our various senses are responsible for transducing different forms of energy into neural activity and give rise to quite distinct perceptions. To a large extent, each of the sensory systems can operate independently. We can, after all, understand someone speaking by telephone or read a book perfectly well without recourse to cues provided by other modalities. It is now clear, however, that multisensory convergence is considerably more widespread in the brain, and particularly the cerebral cortex, than was once thought. Indeed, even the primary cortical areas in each of the main senses have been claimed as part of the growing network of multisensory regions (Ghazanfar and Schroeder 2006).

It is clearly beneficial to be able to combine information from the different senses. Although the perception of speech is based on the processing of sound, what we actually hear can be influenced by visual cues provided by lip movements. This can result in an improvement in speech intelligibility in the presence of other distracting sounds (Sumby and Pollack 1954) or even a subjective change in the speech sounds that are perceived (McGurk and MacDonald 1976). Similarly, the accuracy with which the source of a sound can be localized is affected by the availability of both spatially congruent (Shelton and Searle 1980; Stein et al. 1989) and conflicting (Bertelson and Radeau 1981) visual stimuli. With countless other examples of cross-modal interactions at the perceptual level (Calvert and Thesen 2004), it is perhaps not surprising that multisensory convergence is so widely found throughout the cerebral cortex.

The major challenge that we are now faced with is to identify the function of multisensory integration in different cortical circuits, and particularly at early levels of the cortical hierarchy—the primary and secondary sensory areas—which are more likely to be involved in general-purpose processing relating to multiple sound parameters than in task-specific computational operations (Griffiths et al. 2004; King and Nelken 2009). In doing so, we have to try and understand how other modalities influence the sensitivity or selectivity of cortical neurons in those areas while retaining the modality specificity of the percepts to which the activity of the neurons contributes. By investigating the sources of origin of these inputs and the way in which they interact with the dominant input modality for a given cortical area, we can begin to constrain our ideas about the potential functions of multisensory integration in early sensory cortex.

In this article, we focus on the organization and putative functions of visual inputs to the auditory cortex. Although anatomical and physiological studies have revealed multisensory interactions in visual and somatosensory areas, it is arguably the auditory cortex where most attention has been paid and where we may be closest to answering these questions.


A common feature of all sensory systems is that they comprise multiple cortical areas that can be defined both physiologically and anatomically, and which are collectively involved in the processing of the world around us. Although most studies on the cortical auditory system have focused on the primary area, A1, there is considerable interest in the extent to which different sound features are represented in parallel in distinct functional streams that extend beyond A1 (Griffiths et al. 2004). Research on this question has been heavily influenced by studies of the visual cortex and, in particular, by the proposal that a division of function exists, with separate dorsal and ventral pathways involved in visuomotor control and object identification, respectively. The dorsal processing stream, specialized for detecting object motion and discriminating spatial relationships, includes the middle temporal (MT) and medial superior temporal (MST) areas, whereas the ventral stream comprises areas responsible for color, form, and pattern discrimination. Although the notion of strict parallel processing of information, originating subcortically in the p and m pathways and terminating in temporal and parietal cortical areas, is certainly an oversimplification (Merigan and Maunsell 1993), the perception-action hypothesis is supported by neuroimaging, human neuropsychology, monkey neurophysiology, and human psychophysical experiments (reviewed by Goodale and Westwood 2004).

A popular, if controversial, theory seeks to impose a similar organizational structure onto the auditory cortex. Within this framework, Rauschecker and Tian (2000) proposed that the auditory cortex can be divided into a rostral processing stream, responsible for sound identification, and a caudal processing stream, involved in sound localization. Human functional imaging data provide support for this idea (Alain et al. 2001; Barrett and Hall 2006; Maeder et al. 2001; Warren and Griffiths 2003), and there is evidence for regional differentiation based on the physiological response properties of single neurons recorded in the auditory cortex of nonhuman primates (Tian et al. 2001; Recanzone 2000; Woods et al. 2006; Bendor and Wang 2005). However, the most compelling evidence for a division of labor has been provided by the specific auditory deficits induced by transiently deactivating different cortical areas in cats. Thus, normal sound localization in this species requires the activation of A1, the posterior auditory field (PAF), the anterior ectosylvian sulcus and the dorsal zone of the auditory cortex, whereas other areas, notably the anterior auditory field (AAF), ventral PAF (VPAF), and secondary auditory cortex (A2) do not appear to contribute to this task (Malhotra and Lomber 2007). Moreover, a double dissociation between PAF and AAF in the same animals has been demonstrated, with impaired sound localization produced by cooling of PAF but not AAF, and impaired temporal pattern discrimination resulting from inactivation of AAF but not PAF (Lomber and Malhotra 2008). Lastly, anatomical projection patterns in nonhuman primates support differential roles for rostral and caudal auditory cortex, with each of those areas having distinct prefrontal targets (Hackett et al. 1999; Romanski et al. 1999).

Despite this apparent wealth of data in support of functional specialization within the auditory cortex, there are a number of studies that indicate that sensitivity to both spatial and nonspatial sound attributes is widely distributed across different cortical fields (Harrington et al. 2008; Stecker et al. 2003; Las et al. 2008; Hall and Plack 2009; Recanzone 2008; Nelken et al. 2008; Bizley et al. 2009). Moreover, in humans, circumscribed lesions within the putative “what” and “where” pathways do not always result in the predicted deficits in sound recognition and localization (Adriani et al. 2003). Clearly defined output pathways from auditory cortex to prefrontal cortex certainly seem to exist, but what the behavioral deficits observed following localized deactivation or damage imply about the functional organization of the auditory cortex itself is less clear-cut. Loss of activity in any one part of the network will, after all, affect both upstream cortical areas and potentially the responses of subcortical neurons that receive descending projections from that region of the cortex (Nakamoto et al. 2008). Thus, a behavioral deficit does not necessarily reflect the specialized properties of the neurons within the silenced cortical area per se, but rather the contribution of the processing pathways that the area is integral to.

Can the distribution and nature of multisensory processing in the auditory cortex help reconcile the apparently contrasting findings outlined above? If multisensory interactions in the cortex are to play a meaningful role in perception and behavior, it is essential that the neurons can integrate the corresponding multisensory features of individual objects or events, such as vocalizations and their associated lip movements or the visual and auditory cues originating from the same location in space. Consequently, the extent to which spatial and nonspatial sound features are processed in parallel in the auditory cortex should also be apparent in both the multisensory response properties of the neurons found there and the sources of origin of its visual inputs. Indeed, evidence for task-specific activation of higher cortical areas by different stimulus modalities has recently been provided in humans (Renier et al. 2009). In the next section, we focus on the extent to which anatomical and physiological studies of multisensory convergence and processing in the auditory cortex of the ferret have shed light on this issue. In recent years, this species has gained popularity for studies of auditory cortical processing, in part because of its particular suitability for behavioral studies.


3.3.1. Organization of Ferret Auditory Cortex

Ferret auditory cortex consists of at least six acoustically responsive areas: two core fields, A1 and AAF, which occupy the middle ectosylvian gyrus; two belt areas on the posterior ectosylvian gyrus, the posterior pseudosylvian field (PPF) and posterior suprasylvian field (PSF); plus two areas on the anterior ectosylvian gyrus, the anterior dorsal field (ADF) and the anterior ventral field (AVF) (Bizley et al. 2005; Figure 3.1a). A1, AAF, PPF, and PSF are all tonotopically organized: the neurons found there respond to pure tones and are most sensitive to particular sound frequencies, which vary systemically in value with neuron location within each cortical area. There is little doubt that an equivalent area to the region designated as A1 is found in many different mammalian species, including humans. AAF also appears to be homologous to AAF in other species including the gerbil (Thomas et al. 1993) and the cat (Imaizumi et al. 2004), and is characterized by an underrepresentation of neurons preferring middle frequencies and having shorter response latencies compared to A1.

FIGURE 3.1. Visual inputs to ferret auditory cortex.


Visual inputs to ferret auditory cortex. (a) Ferret sensory cortex. Visual (areas 17–20, PS, SSY, AMLS), posterior parietal (PPr, PPc), somatosensory (S1, SIII, MRSS), and auditory areas (A1, AAF, PPF, PSF, and ADF) have been identified. In addition, LRSS (more...)

Neurons in the posterior fields can be distinguished from those in the primary areas by the temporal characteristics of their responses; discharges are often sustained and they vary in latency and firing pattern in a stimulus-dependent manner. The frequency response areas of posterior field neurons are often circumscribed, exhibiting tuning for sound level as well as frequency. As such, the posterior fields in the ferret resemble PAF and VPAF in the cat (Stecker et al. 2003; Phillips and Orman 1984; Loftus and Sutter 2001) and cortical areas R and RT in the marmoset monkey (Bizley et al. 2005; Bendor and Wang 2008), although whether PPF and PSF actually correspond to these fields is uncertain.

Neurons in ADF also respond to pure tones, but are not tonotopically organized (Bizley et al. 2005). The lack of tonotopicity and the broad, high-threshold frequency response areas that characterize this field are also properties of cat A2 (Schreiner and Cynader 1984). However, given that ferret ADF neurons seem to show relatively greater spatial sensitivity than those in surrounding cortical fields (see following sections), which is not a feature of cat A2, it seems unlikely that these areas are homologous. Ventral to ADF lies AVF. Although many of the neurons that have been recorded there are driven by sound, the high incidence of visually responsive neurons (see Section 3.3.4) makes it likely that AVF should be regarded as a parabelt or higher multisensory field. Given its proximity to the somatosensory area on the medial bank of the rostral suprasylvian sulcus (MRSS) (Keniston et al. 2009), it is possible that AVF neurons might also be influenced by tactile stimuli, but this remains to be determined.

Other studies have also highlighted the multisensory nature of the anterior ectosylvian gyrus. For example, Ramsay and Meredith (2004) described an area surrounding the pseudosylvian sulcus that receives largely segregated inputs from the primary visual and somatosensory cortices, which they termed the pseudosylvian sulcal cortex. Manger et al. (2005) reported that a visually responsive area lies parallel to the pseudosylvian sulcus on the posterolateral half of the anterior ectosylvian gyrus, which also contains bisensory neurons that respond either to both visual and tactile or to visual and auditory stimulation. They termed this area AEV, following the terminology used for the visual region within the cat’s anterior ectosylvian sulcus. Because this region overlaps in part with the acoustically responsive areas that we refer to as ADF and AVF, further research using a range of stimuli will be needed to fully characterize this part of the ferret’s cortex. However, the presence of a robust projection from AVF to the superior colliculus (Bajo et al. 2010) makes it likely that this is equivalent to the anterior ectosylvian sulcus in the cat.

3.3.2. Surrounding Cortical Fields

The different auditory cortical areas described in the previous section are all found on the ectosylvian gyrus (EG), which is enclosed by the suprasylvian sulcus (Figure 3.1a). The somatosensory cortex lies rostral to the EG (Rice et al. 1993; McLaughlin et al. 1998), extrastriate visual areas are located caudally (Redies et al. 1990), and the parietal cortex is found dorsal to the EG (Manger et al. 2002). The suprasylvian sulcus therefore separates the different auditory fields from functionally distinct parts of the cerebral cortex.

Within the suprasylvian sulcus itself, several additional cortical fields have been characterized (Philipp et al. 2006; Manger et al. 2004, 2008; Cantone et al. 2006; Keniston et al. 2008). Beginning at the rostral border between the auditory and somatosensory cortices, field MRSS (Keniston et al. 2009) and the lateral bank of the rostral suprasylvian sulcus (LRSS) (Keniston et al. 2008) form the medial and lateral sides of the suprasylvian sulcus, respectively. Field LRSS has been identified as an auditory–somatosensory area, whereas MRSS is more modality specific and is thought to be a higher somatosensory field. Field MRSS is bordered by the anteromedial lateral suprasylvian visual area (AMLS), which lines the medial or dorsal bank of the suprasylvian sulcus (Manger et al. 2008). Two more visually responsive regions, the suprasylvian visual area (SSY) (Cantone et al. 2006; Philipp et al. 2006) and the posterior suprasylvian area (PS) (Manger et al. 2004) are found on the caudal side of the sulcus. SSY corresponds in location to an area described by Philipp et al. (2005) as the ferret homologue of primate motion-processing area MT. This region has also been described by Manger et al. (2008) as the posteromedial suprasylvian visual area, but we will stay with the terminology used in our previous articles and refer to it as SSY. PS has not been comprehensively investigated and, to our knowledge, neither of these sulcal fields have been tested with auditory or somatosensory stimuli. On the lateral banks of the suprasylvian sulcus, at the dorsal and caudal edges of the EG, remains an area of uninvestigated cortex. On the basis of its proximity to AMLS and SSY, this region has tentatively been divided into the anterolateral lateral suprasylvian visual area (ALLS) and the posterolateral lateral suprasylvian visual area (PLLS) by Manger et al. (2008). However, because these regions of the sulcal cortex lie immediately adjacent to the primary auditory fields, it is much more likely that they are multisensory in nature.

3.3.3. Sensitivity to Complex Sounds

In an attempt to determine whether spatial and nonspatial stimulus attributes are represented within anatomically distinct regions of the ferret auditory cortex, we investigated the sensitivity of neurons in both core and belt areas to stimulus periodicity, timbre, and spatial location (Bizley et al. 2009). Artificial vowel sounds were used for this purpose, as they allowed each of these stimulus dimensions to be varied parametrically. Recordings in our laboratory have shown that ferret vocalizations cover the same frequency range as the sounds used in this study. Vowel identification involves picking out the formant peaks in the spectral envelope of the sound, and is therefore a timbre discrimination task. The periodicity of the sound corresponds to its perceived pitch and conveys information about speaker identity (males tend to have lower pitch voices than females) and emotional state. Neuronal sensitivity to timbre and pitch should therefore be found in cortical areas concerned with stimulus identification.

Neurons recorded throughout the five cortical areas (A1, AAF, PPF, PSF, and ADF) examined were found to be sensitive to the pitch, timbre, and location of the sound source, implying a distributed representation of both spatial and nonspatial sound properties. Nevertheless, significant interareal differences were observed. Sensitivity to sound pitch and timbre was most pronounced in the primary and posterior auditory fields (Bizley et al. 2009). By contrast, relatively greater sensitivity to sound-source location was found in A1 and in the areas around the pseudosylvian sulcus, which is consistent with the finding that the responses of neurons in ADF carry more information about sound azimuth than those in other auditory cortical areas (Bizley and King 2008).

The variance decomposition method used in the study by Bizley et al. (2009) to quantify the effects of each stimulus parameter on the responses of the neurons was very different from the measures used to define a pitch center in marmoset auditory cortex (Bendor and Wang 2005). We did not, for example, test whether pitch sensitivity was maintained for periodic stimuli in which the fundamental frequency had been omitted. Consequently, the distributed sensitivity we observed is not incompatible with the idea that there might be a dedicated pitch-selective area. However, in a subsequent study, we did find that the spiking responses of single neurons and neural ensembles throughout the auditory cortex can account for the ability of trained ferrets to detect the direction of a pitch change (Bizley et al. 2010). Although further research is needed, particularly in awake, behaving animals, these electrophysiological data are consistent with support the results of an earlier intrinsic optical imaging study (Nelken et al. 2008) in providing only limited support for a division of labor across auditory cortical areas in the ferret.

3.3.4. Visual Sensitivity in Auditory Cortex

Visual inputs into auditory cortex have been described in several species, including humans (Calvert et al. 1999; Giard and Peronnet 1999; Molholm et al. 2002), nonhuman primates (Brosch et al. 2005; Ghazanfar et al. 2005; Schroeder and Foxe 2002; Kayser et al. 2007), ferrets (Bizley and King 2008, 2009; Bizley et al. 2007), gerbils (Cahill et al. 1996), and rats (Wallace et al. 2004). In our studies on the ferret, the responses of single neurons and multineuron clusters were recorded to simplistic artificial stimuli presented under anesthesia. Sensitivity to visual stimulation was defined as a statistically significant change in spiking activity after the presentation of light flashes from a light-emitting diode (LED) positioned in the contralateral hemifield or by a significant modulation of the response to auditory stimulation even if the LED by itself was apparently ineffective in driving the neuron.

Although the majority of neurons recorded in the auditory cortex were classified as auditory alone, the activity of more than one quarter was found to be influenced by visual stimulation. Figure 3.2a shows the relative proportion of different response types observed in the auditory cortex as a whole. Bisensory neurons comprised both those neurons whose spiking responses were altered by auditory and visual stimuli and those whose auditory response was modulated by the simultaneously presented visual stimulus. The fact that visual stimuli can drive spiking activity in the auditory cortex has also been described in highly trained monkeys (Brosch et al. 2005). Nevertheless, this finding is unusual, as most reports emphasize the modulatory nature of nonauditory inputs on the cortical responses to sound (Ghazanfar 2009; Musacchia and Schroeder 2009). At least part of the explanation for this is likely to be that we analyzed our data by calculating the mutual information between the neural responses and the stimuli that elicited them. Information (in bits) was estimated by taking into account the temporal pattern of the response rather than simply the overall spike count. This method proved to be substantially more sensitive than a simple spike count measure, and allowed us to detect subtle, but nonetheless significant, changes in the neural response produced by the presence of the visual stimulus.

FIGURE 3.2. Visual–auditory interactions in ferret auditory cortex.


Visual–auditory interactions in ferret auditory cortex. (a) Proportion of neurons (n = 716) that responded to contralaterally presented noise bursts (auditory), to light flashes from an LED positioned in the contralateral visual field (visual), (more...)

Although neurons exhibiting visual–auditory interactions are found in all six areas of the ferret cortex, the proportion of such neurons varies in different cortical areas (Figure 3.2b). Perhaps not surprisingly, visual influences are least common in the primary areas, A1 and AAF. Nevertheless, approximately 20% of the neurons recorded in those regions were found to be sensitive to visual stimulation, and even included some unisensory visual responses. In the fields on the posterior ectosylvian gyrus and ADF, 40% to 50% of the neurons were found to be sensitive to visual stimuli. This rose to 75% in AVF, which, as described in Section 3.3.1, should probably be regarded as a multisensory rather than as a predominantly auditory area.

We found that visual stimulation could either enhance or suppress the neurons’ response to sound and, in some cases, increased the precision in their spike timing without changing the overall firing rate (Bizley et al. 2007). Analysis of all bisensory neurons, including both neurons in which there was a spiking response to each sensory modality and those in which concurrent auditory–visual stimulation modulated the response to sound alone, revealed that nearly two-thirds produced stronger responses to bisensory than to unisensory auditory stimulation. Figure 3.2c shows the proportion of response types in each cortical field. Although the sample size in some areas was quite small, the relative proportions of spiking responses that were either enhanced or suppressed varied across the auditory cortex. Apart from the interactions in A1, the majority of the observed interactions were facilitatory rather than suppressive.

Although a similar trend for a greater proportion of sites to show enhancement as compared with suppression has been reported for local field potential data in monkey auditory cortex, analysis of spiking responses revealed that suppressive interactions are more common (Kayser et al. 2008). This trend was found across four different categories of naturalistic and artificial stimuli, so the difference in the proportion of facilitatory and suppressive interactions is unlikely to reflect the use of different stimuli in the two studies. By systematically varying onset asynchronies between the visual and auditory stimuli, we did observe in a subset of neurons that visual stimuli could have suppressive effects when presented 100 to 200 ms before the auditory stimuli, which were not apparent when the two modalities were presented simultaneously (Bizley et al. 2007). This finding, along with the results of several other studies (Meredith et al. 2006; Dehner et al. 2004; Allman et al. 2008), emphasizes the importance of using an appropriate combination of stimuli to reveal the presence and nature of cross-modal interactions.

Examination of the magnitude of cross-modal facilitation in ferret auditory cortex showed that visual–auditory interactions are predominantly sublinear. In other words, both the mutual information values (in bits) and the spike rates in response to combined auditory–visual stimulation are generally less than the linear sum of the responses to the auditory and visual stimuli presented in isolation, although some notable exceptions to this have been found (e.g., Figure 2E, F of Bizley et al. 2007). This is unsurprising as the stimulus levels used in that study were well above threshold and, according to the “inverse effectiveness principle” (Stein et al. 1988), were unlikely to produce supralinear responses to combined visual–auditory stimulation. Consistent with this is the observation of Kayser et al. (2008), showing that, across stimulus types, multisensory facilitation is more common for those stimuli that are least effective in driving the neurons.

As mentioned above, estimates of the mutual information between the neural responses and each of the stimuli that produce them take into account the full spike discharge pattern. It is then possible to isolate the relative contributions of spike number and spike timing to the neurons’ sensitivity to multisensory stimulation. It has previously been demonstrated in both ferret and cat auditory cortex that the stimulus information contained in the complete spike pattern is conveyed by a combination of spike count and mean spike latency (Nelken et al. 2005). By carrying out a similar analysis of the responses to the brief stimuli used to characterize visual–auditory interactions in ferret auditory cortex, we found that more than half the neurons transmitted more information in the timing of their responses than in their spike counts (Bizley et al. 2007). This is in agreement with the results of Nelken et al. (2005) for different types of auditory stimuli. We found that this was equally the case for unisensory auditory or visual stimuli and for combined visual–auditory stimulation (Figure 3.2d).

3.3.5. Visual Inputs Enhance Processing in Auditory Cortex

To probe the functional significance of the multisensory interactions observed in the auditory cortex, we systematically varied the spatial location of the stimuli and calculated the mutual information between the neural responses and the location of unisensory visual, unisensory auditory, and spatially and temporally coincident auditory–visual stimuli (Bizley and King 2008). The majority of the visual responses were found to be spatially restricted, and usually carried more location-related information than was the case for the auditory responses. The amount of spatial information available in the neural responses varied across the auditory cortex (Figure 3.3). For all three stimulus conditions, spatial sensitivity was found to be highest in ADF, supporting the notion that there is some functional segregation across the auditory cortex, with the anterior fields more involved in spatial processing. Relative to the responses to sound alone, the provision of spatially coincident visual cues frequently altered the amount of information conveyed by the neurons about stimulus location. Bisensory stimulation reduced the spatial information in the response in one third of these cases, but increased it in the remaining two thirds. Thus, overall, visual inputs to the auditory cortex appear to enhance spatial processing.

FIGURE 3.3. Box plots displaying the amount of information transmitted by neurons in each of five ferret cortical fields about LED location (a), sound-source location (b), or the location of temporally and spatially congruent auditory–visual stimuli (c).


Box plots displaying the amount of information transmitted by neurons in each of five ferret cortical fields about LED location (a), sound-source location (b), or the location of temporally and spatially congruent auditory–visual stimuli (c). (more...)

Because of the simple stimuli that were used in these studies, it was not possible to determine whether or how visual inputs might affect the processing of nonspatial information in ferret auditory cortex. However, a number of studies in primates have emphasized the benefits of visual influences on auditory cortex in terms of the improved perception of vocalizations. In humans, lip reading has been shown to activate the auditory cortex (Molholm et al. 2002; Giard and Peronnet 1999; Calvert et al. 1999), and a related study in macaques has shown that presenting a movie of a monkey vocalizing can modulate the auditory cortical responses to that vocalization (Ghazanfar et al. 2005). These effects were compared to a visual control condition in which the monkey viewed a disk that was flashed on and off to approximate the movements of the animal’s mouth. In that study, the integration of face and voice stimuli was found to be widespread in both core and belt areas of the auditory cortex. However, to generate response enhancement, a greater proportion of recording sites in the belt areas required the use of a real monkey face, whereas nonselective modulation of auditory cortical responses was more common in the core areas. Because a number of cortical areas have now been shown to exhibit comparable sensitivity to monkey calls (Recanzone 2008), it would be of considerable interest to compare the degree to which face and non-face visual stimuli can modulate the activity of the neurons found there. This should help us determine the relative extent to which each area might be specialized for processing communication signals.


Characterizing the way in which neurons are influenced by visual stimuli and their distribution within the auditory cortex is only a first step in identifying their possible functions. It is also necessary to know where those visual inputs originate. Potentially, visual information might gain access to the auditory cortex in a number of ways. These influences could arise from direct projections from the visual cortex or they could be inherited from multisensory subcortical nuclei, such as nonlemniscal regions of the auditory thalamus. A third possibility includes feedback connections from higher multisensory association areas in temporal, parietal, or frontal cortex. Anatomical evidence from a range of species including monkeys (Smiley et al. 2007; Hackett et al. 2007a; Cappe et al. 2009), ferrets (Bizley et al. 2007), prairie voles (Campi et al. 2010), and gerbils (Budinger et al. 2006) has shown that subcortical as well as feedforward and feedback corticortical inputs could underpin multisensory integration in auditory cortex. To determine the most likely origins of the nonauditory responses in the auditory cortex, we therefore need to consider studies of anatomical connectivity in conjunction with information about the physiological properties of the neurons, such as tuning characteristics or response latencies.

Previous studies have demonstrated direct projections from core and belt auditory cortex into visual areas V1 and V2 in nonhuman primates (Rockland and Ojima 2003; Falchier et al. 2002) and, more recently, in cats (Hall and Lomber 2008). The reciprocal projection, from V1 to A1, remains to be described in primates, although Hackett et al. (2007b) have found evidence for a pathway terminating in the caudomedial belt area of the auditory cortex from the area prostriata, adjacent to V1, which is connected with the peripheral visual field representations in V1, V2, and MT. Connections between early auditory and visual cortical fields have also been described in gerbils (Budinger et al. 2006, 2008) and prairie voles (Campi et al. 2010).

By placing injections of neural tracer into physiologically identified auditory fields in the ferret, we were able to characterize the potential sources of visual input (Bizley et al. 2007; Figure 3.1b, c). These data revealed a clear projection pattern whereby specific visual cortical fields innervate specific auditory fields. A sparse direct projection exists from V1 to the core auditory cortex (A1 and AAF), which originates from the region of V1 that represents the peripheral visual field. This finding mirrors that of the reciprocal A1 to V1 projection in monkeys and cats, which terminates in the peripheral field representation of V1 (Rockland and Ojima 2003; Falchier et al. 2002; Hall and Lomber 2008). Ferret A1 and AAF are also weakly innervated by area V2. The posterior auditory fields, PPF and PSF, are innervated principally by areas 20a and 20b, thought to be part of the visual form-processing pathway (Manger et al. 2004). In contrast, the largest inputs to the anterior fields, ADF and AVF, come from SSY, which is regarded as part of the visual “where” processing stream (Philipp et al. 2006).

Interestingly, this difference in the sources of cortical visual input, which is summarized in Figure 3.1d, appears to reflect the processing characteristics of the auditory cortical fields concerned. As described above, the fields on the posterior ectosylvian gyrus are more sensitive to pitch and timbre, parameters that contribute to the identification of a sound source, whereas spatial sensitivity for auditory, visual, and multisensory stimuli is greatest in ADF (Figure 3.3). This functional distinction therefore matches the putative roles of the extrastriate areas that provide the major sources of cortical visual input to each of these regions.

These studies appear to support the notion of a division of labor across the nonprimary areas of ferret auditory cortex, but it would be premature to conclude that distinct fields are responsible for the processing of spatial and nonspatial features of the world. Thus, although PSF is innervated by nonspatial visual processing areas 20a and 20b (Figure 3.1c), the responses of a particularly large number of neurons found there show an increase in transmitted spatial information when a spatially congruent visual stimulus is added to the auditory stimulus (Bizley and King 2008). This could be related to a need to integrate spatial and nonspatial cues when representing objects and events in the auditory cortex. The possibility that connections between the visual motion-sensitive area SSY and the fields on the anterior ectosylvian gyrus are involved in processing spatial information provided by different sensory modalities is supported by a magnetoencephalography study in humans showing that audio–visual motion signals are integrated in the auditory cortex (Zvyagintsev et al. 2009). However, we must not forget that visual motion also plays a key role in the perception of communication calls. By making intracranial recordings in epileptic patients, Besle et al. (2008) found that the visual cues produced by lip movements activate MT followed, approximately 10 ms later, by secondary auditory areas, where they alter the responses to sound in ways that presumably influence speech perception. Thus, although the influence of facial expressions on auditory cortical neurons is normally attributed to feedback from the superior temporal sulcus (Ghazanfar et al. 2008), the availability of lower-level visual signals that provide cues to sound onset and offset may be important as well.


The concurrent availability of visual information presumably alters the representation in the auditory cortex of sources that can be seen as well as heard in ways that are relevant for perception and behavior. Obviously, the same argument applies to the somatosensory inputs that have also been described there (Musacchia and Schroeder 2009). By influencing early levels of cortical processing, these nonauditory inputs may play a fairly general processing role by priming the cortex to receive acoustic signals. It has, for example, been proposed that visual and somatosensory inputs can modulate the phase of oscillatory activity in the auditory cortex, potentially amplifying the response to related auditory signals (Schroeder et al. 2008). But, as we have seen, visual inputs can also have more specific effects, changing the sensitivity and even the selectivity of cortical responses to stimulus location and, at least in primates, to vocalizations where communication relies on both vocal calls and facial gestures. The role of multisensory processing in receptive auditory communication is considered in more detail in other chapters in this volume. Here, we will focus on the consequences of merging spatial information across different sensory modalities in the auditory cortex.

3.5.1. Combining Auditory and Visual Spatial Representations in the Brain

There are fundamental differences in the ways in which source location is extracted by the visual and auditory systems. The location of visual stimuli is represented topographically, first by the distribution of activity across the retina and then at most levels of the central visual pathway. By contrast, auditory space is not encoded explicitly along the cochlea. Consequently, sound-source location has to be computed within the brain on the basis of the relative intensity and timing of sounds at each ear (“binaural cues”), coupled with the location-dependent filtering of sounds by the external ear (King et al. 2001). By tuning neurons to appropriate combinations of these cues, a “visual-like” map of auditory space is constructed in the superior colliculus, allowing spatial information from different sensory modalities to be represented in a common format (King and Hutchings 1987; Middlebrooks and Knudsen 1984). This arrangement is particularly advantageous for facilitating the integration of multisensory cues from a common source for the purpose of directing orienting behavior (Stein and Stanford 2008). However, because spatial signals provided by each sensory modality are initially encoded using different reference frames, with visual signals based on eye-centered retinal coordinates and auditory signals being head centered, information about current eye position has to be incorporated into the activity of these neurons in order to maintain map alignment (Hartline et al. 1995; Jay and Sparks 1987).

In contrast to the topographic representation of auditory space in the superior colliculus, there is no space map in the auditory cortex (King and Middlebrooks 2010), posing an even greater challenge for the integration of visual and auditory spatial signals at the cortical level. The integrity of several auditory cortical areas is essential for normal sound localization (Malhotra and Lomber 2007), but we still have a very incomplete understanding of how neural activity in those regions contributes to the percept of where a sound source is located. The spatial receptive fields of individual cortical neurons are frequently very broad and, for the most part, occupy the contralateral side of space. However, several studies have emphasized that sound-source location can also be signaled by the timing of spikes (Jenison 2000; Nelken et al. 2005; Stecker et al. 2003). Our finding that the presence of spatially congruent visual stimuli leads to auditory cortical neurons becoming more informative about the source location, and that this greater spatial selectivity is based on both the timing and number of spikes evoked, is clearly consistent with this. Whatever the relative contributions of different neural coding strategies might be, it seems that sound-source location is signaled by the population response of neurons in the auditory cortex (Woods et al. 2006). The approach used by Allman and colleagues (2009) to estimate the response facilitation produced in a population of cortical neurons by combining visual and auditory stimuli might therefore be useful for characterizing the effects on spatial processing at this level.

We pointed out above that meaningful interactions between different sensory modalities can take place only if the different reference frames used to encode modality-specific spatial signals are brought together. Further evidence for the multisensory representation of spatial signals in the auditory cortex is provided by the demonstration that gaze direction can change the activity of neurons in the auditory cortex (Fu et al. 2004; Werner-Reiss et al. 2003). A modulatory influence of eye position on auditory responses has been observed as early as the inferior colliculus (Groh et al. 2001), indicating that these effects could be inherited from the midbrain rather than created de novo in the auditory cortex. On the other hand, the timing and laminar profile of eye-position effects in the auditory cortex is more consistent with an origin from nonlemniscal regions of the thalamus or via feedback projections from the parietal or frontal cortices (Fu et al. 2004). As in the superior colliculus, varying eye position does not change auditory cortical spatial tuning in a manner consistent with a straightforward transformation into eye-centered coordinates. Rather, spatial tuning seems to take on an intermediate form between eye-centered and head-centered coordinates (Werner-Reiss et al. 2003).

3.5.2. A Role for Auditory Cortex in Spatial Recalibration?

One possibility that has attracted recent attention is that visual–auditory interactions in early sensory cortex could be involved in the visual recalibration of auditory space. The representation of auditory space in the brain is inherently plastic, even in adulthood, and there are several well-documented examples in which the perceived location of sound sources can be altered so as to conform to changes in visual inputs (King 2009; King et al. 2001). The most famous of these is the ventriloquism illusion, whereby synchronous but spatially disparate visual cues can “capture” the location of a sound source, so that it is incorrectly perceived to arise from near the seen location (Bertelson and Radeau 1981). Repeated presentation of consistently misaligned visual and auditory cues results in a shift in the perception of auditory space that can last for tens of minutes once the visual stimulus is removed. This aftereffect has been reported in humans (Recanzone 1998; Radeau and Bertelson 1974; Lewald 2002) and in nonhuman primates (Woods and Recanzone 2004).

Given the widespread distribution of visual–auditory interactions in the cortex, a number of sites could potentially provide the neural substrate for this cross-modal spatial illusion. The finding that the ventriloquism aftereffect does not transfer across sound frequency (Lewald 2002; Recanzone 1998; Woods and Recanzone 2004) implies the involvement of a tonotopically organized region, i.e., early auditory cortex. On the other hand, generalization across frequencies has been observed in another study (Frissen et al. 2005), so this conclusion may not stand. However, neuroimaging results in humans have shown that activity levels in the auditory cortex vary on a trial-by-trial basis according to whether a spatially discrepant visual stimulus is presented at the same time (Bonath et al. 2007). Furthermore, the finding by Passamonti et al. (2009) that patients with unilateral lesions of the visual cortex fail to show the ventriloquism aftereffect in the affected hemifield, whereas patients with parietotemporal lesions still do, is consistent with the possibility that connections between the visual and auditory cortices are involved. On the other hand, the hemianopic patients did show improved sound localization accuracy when visual and auditory stimuli are presented at the same location in space, implying that different neural circuits may underlie these cross-modal spatial effects.

Visual capture of sound-source location is thought to occur because visual cues normally provide more reliable and higher-resolution spatial information. If the visual stimuli are blurred, however, so that this is no longer the case, spatially conflicting auditory cues can then induce systematic errors in visual localization (Alais and Burr 2004). Nothing is known about the neural basis for reverse ventriloquism, but it is tempting to speculate that auditory influences on visual cortex might be involved. Indeed, the influence of sound on perceptual learning in a visual motion discrimination task has been shown to be limited to locations in visual space that match those of the sound source, implying an auditory influence on processing in a visual area that is retinotopically organized (Beer and Watanabe 2009).

Behavioral studies have shown that adult humans and other mammals can adapt substantially to altered auditory spatial cues produced, for example, by reversibly occluding or changing the shape of the external ear (reviewed by Wright and Zhang 2006). Because visual cues provide a possible source of sensory feedback about the accuracy of acoustically guided behavior, one potential role of visual inputs to the auditory cortex is to guide the plasticity observed when localization cues are altered. However, Kacelnik et al. (2006) found that the capacity of adult ferrets to relearn to localize sound accurately after altering binaural cues by reversible occlusion of one ear is not dependent on visual feedback. It has been suggested that instead of being guided by vision, this form of adaptive plasticity could result from unsupervised sensorimotor learning, in which the dynamic acoustic inputs resulting from an animal’s own movements help stabilize the brain’s representation of auditory space (Aytekin et al. 2008). Although vision is not essential for the recalibration of auditory space in monaurally occluded ferrets, it is certainly possible that training with congruent multisensory cues might result in faster learning than that seen with auditory cues alone, as shown in humans for a motion detection task (Kim et al. 2008).


There is now extensive anatomical and physiological evidence from a range of species that multisensory convergence occurs at the earliest levels of auditory cortical processing. These nonauditory influences therefore have to be taken into account in any model of what the auditory cortex actually does. Indeed, one of the consequences of visual, somatosensory, and eye-position effects on the activity of neurons in core and belt areas of the auditory cortex is that those influences will be passed on to each of the brain regions to which these areas project. Multiple sources of input have been implicated in multisensory integration within auditory cortex, and a more detailed characterization of those inputs will help determine the type of information that they provide and what effect this might have on auditory processing. Some of those inputs are likely to provide low-level temporal or spatial cues that enhance auditory processing in a fairly general way, whereas others provide more complex information that is specifically related, for example, to the processing of communication signals. Revealing where those inputs come from and where they terminate will help unravel the relative contributions of different auditory cortical areas to perception. Indeed, the studies that have been carried out to date have provided additional support for the standpoint that there is some functional segregation across the different parts of the auditory cortex. In order to take this further, however, it will also be necessary to examine the behavioral and physiological effects of experimentally manipulating activity in those circuits if we are to understand how visual inputs influence auditory processing and perception.


  1. Adriani M, Maeder P, Meuli R, editors. et al. Sound recognition and localization in man: Specialized cortical networks and effects of acute circumscribed lesions. Experimental Brain Research. 2003;153:591–604. [PubMed: 14504861]
  2. Alain C, Arnott S.R, Hevenor S, Graham S, Grady C.L. “What” and “where” in the human auditory system. Proceedings of the National Academy of Sciences of the United States of America. 2001;98:12301–6. [PMC free article: PMC59809] [PubMed: 11572938]
  3. Alais D, Burr D. The ventriloquist effect results from near-optimal bimodal integration. Current Biology. 2004;14:257–62. [PubMed: 14761661]
  4. Allman B.L, Keniston L.P, Meredith M.A. Subthreshold auditory inputs to extrastriate visual neurons are responsive to parametric changes in stimulus quality: Sensory-specific versus non-specific coding. Brain Research. 2008;1242:95–101. [PMC free article: PMC2645081] [PubMed: 18479671]
  5. Allman B.L, Keniston L.P, Meredith M.A. Adult deafness induces somatosensory conversion of ferret auditory cortex. Proceedings of the National Academy of Sciences of the United States of America. 2009;106:5925–30. [PMC free article: PMC2667075] [PubMed: 19307553]
  6. Aytekin M, Moss C.F, Simon J.Z. A sensorimotor approach to sound localization. Neural Computation. 2008;20:603–35. [PubMed: 18045018]
  7. Bajo V.M, Nodal F.R, Bizley J.K, King A.J. The non-lemniscal auditory cortex in ferrets: Convergence of corticotectal inputs in the superior colliculus. Frontiers in Neuroanatomy. 2010;4:18. [PMC free article: PMC2904598] [PubMed: 20640247]
  8. Barrett D.J, Hall D.A. Response preferences for “what” and “where” in human non-primary auditory cortex. NeuroImage. 2006;32:968–77. [PubMed: 16733092]
  9. Beer A.L, Watanabe T. Specificity of auditory-guided visual perceptual learning suggests cross-modal plasticity in early visual cortex. Experimental Brain Research. 2009;198:353–61. [PMC free article: PMC2865160] [PubMed: 19306091]
  10. Bendor D, Wang X. The neuronal representation of pitch in primate auditory cortex. Nature. 2005;436:1161–5. [PMC free article: PMC1780171] [PubMed: 16121182]
  11. Bendor D, Wang X. Neural response properties of primary, rostral, and rostrotemporal core fields in the auditory cortex of marmoset monkeys. Journal of Neurophysiology. 2008;100:888–906. [PMC free article: PMC2525707] [PubMed: 18525020]
  12. Bertelson P, Radeau M. Cross-modal bias and perceptual fusion with auditory-visual spatial discordance. Perception & Psychophysics. 1981;29:578–84. [PubMed: 7279586]
  13. Besle J, Fischer C, Bidet-Caulet A, Lecaignard F, Bertrand O, Giard M.H. Visual activation and audiovisual interactions in the auditory cortex during speech perception: Intracranial recordings in humans. Journal of Neuroscience. 2008;28:14301–10. [PubMed: 19109511]
  14. Bizley J.K, King A.J. Visual-auditory spatial processing in auditory cortical neurons. Brain Research. 2008;1242:24–36. [PMC free article: PMC4340571] [PubMed: 18407249]
  15. Bizley J.K, King A.J. Visual influences on ferret auditory cortex. Hearing Research. 2009;258:55–63. [PMC free article: PMC2791852] [PubMed: 19595754]
  16. Bizley J.K, Nodal F.R, Nelken I, King A.J. Functional organization of ferret auditory cortex. Cerebral Cortex. 2005;15:1637–53. [PubMed: 15703254]
  17. Bizley J.K, Nodal F.R, Bajo V.M, Nelken I, King A.J. Physiological and anatomical evidence for multisensory interactions in auditory cortex. Cerebral Cortex. 2007;17:2172–89. [PubMed: 17135481]
  18. Bizley J.K, Walker K.M, Silverman B.W, King A.J, Schnupp J.W. Interdependent encoding of pitch, timbre, and spatial location in auditory cortex. Journal of Neuroscience. 2009;29:2064–75. [PMC free article: PMC2663390] [PubMed: 19228960]
  19. Bizley J.K, Walker K.M, King A.J, Schnupp J.W. Neural ensemble codes for stimulus periodicity in auditory cortex. Journal of Neuroscience. 2010;30:5078–91. [PMC free article: PMC2864913] [PubMed: 20371828]
  20. Bonath B, Noesselt T, Martinez A, editors. et al. Neural basis of the ventriloquist illusion. Current Biology. 2007;17:1697–703. [PubMed: 17884498]
  21. Brosch M, Selezneva E, Scheich H. Nonauditory events of a behavioral procedure activate auditory cortex of highly trained monkeys. Journal of Neuroscience. 2005;25:6797–806. [PubMed: 16033889]
  22. Budinger E, Heil P, Hess A, Scheich H. Multisensory processing via early cortical stages: Connections of the primary auditory cortical field with other sensory systems. Neuroscience. 2006;143:1065–83. [PubMed: 17027173]
  23. Budinger E, Laszcz A, Lison H, Scheich H, Ohl F.W. Non-sensory cortical and subcortical connections of the primary auditory cortex in Mongolian gerbils: Bottom-up and top-down processing of neuronal information via field AI. Brain Research. 2008;1220:2–32. [PubMed: 17964556]
  24. Cahill L, Ohl F, Scheich H. Alteration of auditory cortex activity with a visual stimulus through conditioning: a 2-deoxyglucose analysis. Neurobiology of Learning and Memory. 1996;65:213–22. [PubMed: 8616585]
  25. Calvert G.A, Thesen T. Multisensory integration: Methodological approaches and emerging principles in the human brain. Journal of Physiology, Paris. 2004;98:191–205. [PubMed: 15477032]
  26. Calvert G.A, Brammer M.J, Bullmore E.T, Campbell R, Iversen S.D, David A.S. Response amplification in sensory-specific cortices during crossmodal binding. Neuroreport. 1999;10:2619–23. [PubMed: 10574380]
  27. Campi K.L, Bales K.L, Grunewald R, Krubitzer L. Connections of auditory and visual cortex in the prairie vole (Microtus ochrogaster): Evidence for multisensory processing in primary sensory areas. Cerebral Cortex. 2010;20:89–108. [PMC free article: PMC2792189] [PubMed: 19395525]
  28. Cantone G, Xiao J, Levitt J.B. Retinotopic organization of ferret suprasylvian cortex. Visual Neuroscience. 2006;23:61–77. [PubMed: 16597351]
  29. Cappe C, Morel A, Barone P, Rouiller E.M. The thalamocortical projection systems in primate: An anatomical support for multisensory and sensorimotor interplay. Cerebral Cortex. 2009;19:2025–37. [PMC free article: PMC2722423] [PubMed: 19150924]
  30. Dehner L.R, Keniston L.P, Clemo H.R, Meredith M.A. Cross-modal circuitry between auditory and somatosensory areas of the cat anterior ectosylvian sulcal cortex: A ‘new’ inhibitory form of multisensory convergence. Cerebral Cortex. 2004;14:387–403. [PubMed: 15028643]
  31. Falchier A, Clavagnier S, Barone P, Kennedy H. Anatomical evidence of multimodal integration in primate striate cortex. Journal of Neuroscience. 2002;22:5749–59. [PubMed: 12097528]
  32. Frissen I, Vroomen J, De Gelder B, Bertelson P. The aftereffects of ventriloquism: Generalization across sound-frequencies. Acta Psychologica. 2005;118:93–100. [PubMed: 15627411]
  33. Fu K.M, Shah A.S, O'Connell M.N, editors. et al. Timing and laminar profile of eye-position effects on auditory responses in primate auditory cortex. Journal of Neurophysiology. 2004;92:3522–31. [PubMed: 15282263]
  34. Ghazanfar A.A. The multisensory roles for auditory cortex in primate vocal communication. Hearing Research. 2009;258:113–20. [PMC free article: PMC2787678] [PubMed: 19371776]
  35. Ghazanfar A.A, Schroeder C.E. Is neocortex essentially multisensory? Trends in Cognitive Sciences. 2006;10:278–85. [PubMed: 16713325]
  36. Ghazanfar A.A, Maier J.X, Hoffman K.L, Logothetis N.K. Multisensory integration of dynamic faces and voices in rhesus monkey auditory cortex. Journal of Neuroscience. 2005;25:5004–12. [PubMed: 15901781]
  37. Ghazanfar A.A, Chandrasekaran C, Logothetis N.K. Interactions between the superior temporal sulcus and auditory cortex mediate dynamic face/voice integration in rhesus monkeys. Journal of Neuroscience. 2008;28:4457–69. [PMC free article: PMC2663804] [PubMed: 18434524]
  38. Giard M.H, Peronnet F. Auditory-visual integration during multimodal object recognition in humans: A behavioral and electrophysiological study. Journal of Cognitive Neuroscience. 1999;11:473–90. [PubMed: 10511637]
  39. Goodale M.A, Westwood D.A. An evolving view of duplex vision: Separate but interacting cortical pathways for perception and action. Current Opinion in Neurobiology. 2004;14:203–11. [PubMed: 15082326]
  40. Griffiths T.D, Warren J.D, Scott S.K, Nelken I, King A.J. Cortical processing of complex sound: A way forward? Trends in Neuroscience. 2004;27:181–5. [PubMed: 15046876]
  41. Groh J.M, Trause A.S, Underhill A.M, Clark K.R, Inati S. Eye position influences auditory responses in primate inferior colliculus. Neuron. 2001;29:509–18. [PubMed: 11239439]
  42. Hackett T.A, Stepniewska I, Kaas J.H. Prefrontal connections of the parabelt auditory cortex in macaque monkeys. Brain Research. 1999;817:45–58. [PubMed: 9889315]
  43. Hackett T.A, De La Mothe L.A, Ulbert I, Karmos G, Smiley J, Schroeder C.E. Multisensory convergence in auditory cortex: II. Thalamocortical connections of the caudal superior temporal plane. Journal of Comparative Neurology. 2007a;502:924–52. [PubMed: 17444488]
  44. Hackett T.A, Smiley J.F, Ulbert I, editors. et al. Sources of somatosensory input to the caudal belt areas of auditory cortex. Perception. 2007b;36:1419–30. [PubMed: 18265825]
  45. Hall A.J, Lomber S.G. Auditory cortex projections target the peripheral field representation of primary visual cortex. Experimental Brain Research. 2008;190:413–30. [PubMed: 18641978]
  46. Hall D.A, Plack C.J. Pitch processing sites in the human auditory brain. Cerebral Cortex. 2009;19:576–85. [PMC free article: PMC2638814] [PubMed: 18603609]
  47. Harrington I.A, Stecker G.C, Macpherson E.A, Middlebrooks J.C. Spatial sensitivity of neurons in the anterior, posterior, and primary fields of cat auditory cortex. Hearing Research. 2008;240:22–41. [PMC free article: PMC2515616] [PubMed: 18359176]
  48. Hartline P.H, Vimal R.L, King A.J, Kurylo D.D, Northmore D.P. Effects of eye position on auditory localization and neural representation of space in superior colliculus of cats. Experimental Brain Research. 1995;104:402–8. [PubMed: 7589292]
  49. Imaizumi K, Priebe N.J, Crum P.A, Bedenbaugh P.H, Cheung S.W, Schreiner C.E. Modular functional organization of cat anterior auditory field. Journal of Neurophysiology. 2004;92:444–57. [PubMed: 15014102]
  50. Jay M.F, Sparks D.L. Sensorimotor integration in the primate superior colliculus: II. Coordinates of auditory signals. Journal of Neurophysiology. 1987;57:35–55. [PubMed: 3559680]
  51. Jenison R.L. Correlated cortical populations can enhance sound localization performance. Journal of the Acoustical Society of America. 2000;107:414–21. [PubMed: 10641650]
  52. Kacelnik O, Nodal F.R, Parsons C.H, King A.J. Training-induced plasticity of auditory localization in adult mammals. PLoS Biology. 2006;4:627–38.
  53. Kayser C, Petkov C.I, Augath M, Logothetis N.K. Functional imaging reveals visual modulation of specific fields in auditory cortex. Journal of Neuroscience. 2007;27:1824–35. [PubMed: 17314280]
  54. Kayser C, Petkov C.I, Logothetis N.K. Visual modulation of neurons in auditory cortex. Cerebral Cortex. 2008;18:1560–74. [PubMed: 18180245]
  55. Keniston L.P, Allman B.L, Meredith M.A. The rostral suprasylvian sulcus (RSSS) of the ferret: A ‘new’ multisensory area. Society for Neuroscience Abstracts. 2008;38:457–10.
  56. Keniston L.P, Allman B.L, Meredith M.A, Clemo H.R. Somatosensory and multisensory properties of the medial bank of the ferret rostral suprasylvian sulcus. Experimental Brain Research. 2009;196:239–51. [PMC free article: PMC2854486] [PubMed: 19466399]
  57. Kim R.S, Seitz A.R, Shams L. Benefits of stimulus congruency for multisensory facilitation of visual learning. PLoS ONE. 2008;3:e1532. [PMC free article: PMC2211398] [PubMed: 18231612]
  58. King A.J. Visual influences on auditory spatial learning. Philosophical Transactions of the Royal Society of London. Series B, Biological Sciences. 2009;364:331–9. [PMC free article: PMC2674475] [PubMed: 18986967]
  59. King A.J, Hutchings M.E. Spatial response properties of acoustically responsive neurons in the superior colliculus of the ferret: A map of auditory space. Journal of Neurophysiology. 1987;57:596–624. [PubMed: 3559694]
  60. King A.J, Nelken I. Unraveling the principles of auditory cortical processing: Can we learn from the visual system? Nature Neuroscience. 2009;12:698–701. [PMC free article: PMC3657701] [PubMed: 19471268]
  61. King A.J, Middlebrooks J.C. Cortical representation of auditory space. In: Winer J.A, Schreiner C.E, editors. The Auditory Cortex. New York: Springer; 2011. pp. 329–41.
  62. King A.J, Schnupp J.W, Doubell T.P. The shape of ears to come: Dynamic coding of auditory space. Trends in Cognitive Sciences. 2001;5:261–70. [PubMed: 11390297]
  63. Las L, Shapira A.H, Nelken I. Functional gradients of auditory sensitivity along the anterior ectosylvian sulcus of the cat. Journal of Neuroscience. 2008;28:3657–67. [PubMed: 18385324]
  64. Lewald J. Rapid adaptation to auditory–visual spatial disparity. Learning and Memory. 2002;9:268–78. [PMC free article: PMC187125] [PubMed: 12359836]
  65. Loftus W.C, Sutter M.L. Spectrotemporal organization of excitatory and inhibitory receptive fields of cat posterior auditory field neurons. Journal of Neurophysiology. 2001;86:475–91. [PubMed: 11431526]
  66. Lomber S.G, Malhotra S. Double dissociation of ‘what’ and ‘where’ processing in auditory cortex. Nature Neuroscience. 2008;11:609–16. [PubMed: 18408717]
  67. Maeder P.P, Meuli R.A, Adriani M, editors. et al. Distinct pathways involved in sound recognition and localization: A human fMRI study. Neuroimage. 2001;14:802–16. [PubMed: 11554799]
  68. Malhotra S, Lomber S.G. Sound localization during homotopic and heterotopic bilateral cooling deactivation of primary and nonprimary auditory cortical areas in the cat. Journal of Neurophysiology. 2007;97:26–43. [PubMed: 17035367]
  69. Manger P.R, Masiello I, Innocenti G.M. Areal organization of the posterior parietal cortex of the ferret (Mustela putorius) Cerebral Cortex. 2002;12:1280–97. [PubMed: 12427679]
  70. Manger P.R, Nakamura H, Valentiniene S, Innocenti G.M. Visual areas in the lateral temporal cortex of the ferret (Mustela putorius) Cerebral Cortex. 2004;14:676–89. [PubMed: 15054048]
  71. Manger P.R, Engler G, Moll C.K, Engel A.K. The anterior ectosylvian visual area of the ferret: A homologue for an enigmatic visual cortical area of the cat? European Journal of Neuroscience. 2005;22:706–14. [PubMed: 16101752]
  72. Manger P.R, Engler G, Moll C.K, Engel A.K. Location, architecture, and retinotopy of the anteromedial lateral suprasylvian visual area (AMLS) of the ferret (Mustela putorius) Visual Neuroscience. 2008;25:27–37. [PubMed: 18282308]
  73. McGurk H, MacDonald J. Hearing lips and seeing voices. Nature. 1976;264:746–8. [PubMed: 1012311]
  74. McLaughlin D.F, Sonty R.V, Juliano S.L. Organization of the forepaw representation in ferret somatosensory cortex. Somatosensory & Motor Research. 1998;15:253–68. [PubMed: 9875544]
  75. Meredith M.A, Keniston L.R, Dehner L.R, Clemo H.R. Crossmodal projections from somatosensory area SIV to the auditory field of the anterior ectosylvian sulcus (FAES) in cat: Further evidence for subthreshold forms of multisensory processing. Experimental Brain Research. 2006;172:472–84. [PubMed: 16501962]
  76. Merigan W.H, Maunsell J.H. How parallel are the primate visual pathways? Annual Review of Neuroscience. 1993;16:369–402. [PubMed: 8460898]
  77. Middlebrooks J.C, Knudsen E.I. A neural code for auditory space in the cat's superior colliculus. Journal of Neuroscience. 1984;4:2621–34. [PubMed: 6491727]
  78. Molholm S, Ritter W, Murray M.M, Javitt D.C, Schroeder C.E, Foxe J.J. Multisensory auditory-visual interactions during early sensory processing in humans: A high-density electrical mapping study. Brain Research Cognitive Brain Research. 2002;14:115–28. [PubMed: 12063135]
  79. Musacchia G, Schroeder C.E. Neuronal mechanisms, response dynamics and perceptual functions of multisensory interactions in auditory cortex. Hearing Research. 2009;258:72–9. [PMC free article: PMC2989528] [PubMed: 19595755]
  80. Nakamoto K.T, Jones S.J, Palmer A.R. Descending projections from auditory cortex modulate sensitivity in the midbrain to cues for spatial position. Journal of Neurophysiology. 2008;99:2347–56. [PubMed: 18385487]
  81. Nelken I, Chechik G, Mrsic-Flogel T.D, King A.J, Schnupp J.W. Encoding stimulus information by spike numbers and mean response time in primary auditory cortex. Journal of Computational Neuroscience. 2005;19:199–221. [PubMed: 16133819]
  82. Nelken I, Bizley J.K, Nodal F.R, Ahmed B, King A.J, Schnupp J.W. Responses of auditory cortex to complex stimuli: Functional organization revealed using intrinsic optical signals. Journal of Neurophysiology. 2008;99:1928–41. [PubMed: 18272880]
  83. Passamonti C, Bertini C, Ladavas E. Audio-visual stimulation improves oculomotor patterns in patients with hemianopia. Neuropsychologia. 2009;47:546–55. [PubMed: 18983860]
  84. Philipp R, Distler C, Hoffmann K.P. A motion-sensitive area in ferret extrastriate visual cortex: An analysis in pigmented and albino animals. Cerebral Cortex. 2006;16:779–90. [PubMed: 16135782]
  85. Phillips D.P, Orman S.S. Responses of single neurons in posterior field of cat auditory cortex to tonal stimulation. Journal of Neurophysiology. 1984;51:147–63. [PubMed: 6693932]
  86. Radeau M, Bertelson P. The after-effects of ventriloquism. Quarterly Journal of Experimental Psychology. 1974;26:63–71. [PubMed: 4814864]
  87. Ramsay A.M, Meredith M.A. Multiple sensory afferents to ferret pseudosylvian sulcal cortex. Neuroreport. 2004;15:461–5. [PubMed: 15094504]
  88. Rauschecker J.P, Tian B. Mechanisms and streams for processing of “what” and “where” in auditory cortex. Proceedings of the National Academy of Sciences of the United States of America. 2000;97:11800–6. [PMC free article: PMC34352] [PubMed: 11050212]
  89. Recanzone G.H. Rapidly induced auditory plasticity: The ventriloquism aftereffect. Proceedings of the National Academy of Sciences of the United States of America. 1998;95:869–75. [PMC free article: PMC33810] [PubMed: 9448253]
  90. Recanzone G.H. Spatial processing in the auditory cortex of the macaque monkey. Proceedings of the National Academy of Sciences of the United States of America. 2000;97:11829–35. [PMC free article: PMC34356] [PubMed: 11050216]
  91. Recanzone G.H. Representation of con-specific vocalizations in the core and belt areas of the auditory cortex in the alert macaque monkey. Journal of Neuroscience. 2008;28:13184–93. [PMC free article: PMC2614135] [PubMed: 19052209]
  92. Redies C, Diksic M, Riml H. Functional organization in the ferret visual cortex: A double-label 2-deoxyglucose study. Journal of Neuroscience. 1990;10:2791–803. [PubMed: 2388088]
  93. Renier L.A, Anurova I, De Volder A.G, Carlson S, Vanmeter J, Rauschecker J.P. Multisensory integration of sounds and vibrotactile stimuli in processing streams for “what” and “where.” Journal of Neuroscience. 2009;29:10950–60. [PMC free article: PMC3343457] [PubMed: 19726653]
  94. Rice F.L, Gomez C.M, Leclerc S.S, Dykes R.W, Moon J.S, Pourmoghadam K. Cytoarchitecture of the ferret suprasylvian gyrus correlated with areas containing multiunit responses elicited by stimulation of the face. Somatosensory & Motor Research. 1993;10:161–88. [PubMed: 8392241]
  95. Rockland K.S, Ojima H. Multisensory convergence in calcarine visual areas in macaque monkey. International Journal of Psychophysiology. 2003;50:19–26. [PubMed: 14511833]
  96. Romanski L.M, Tian B, Fritz J, Mishkin M, Goldman-Rakic P.S, Rauschecker J.P. Dual streams of auditory afferents target multiple domains in the primate prefrontal cortex. Nature Neuroscience. 1999;2:1131–6. [PMC free article: PMC2778291] [PubMed: 10570492]
  97. Schreiner C.E, Cynader M.S. Basic functional organization of second auditory cortical field (AII) of the cat. Journal of Neurophysiology. 1984;51:1284–305. [PubMed: 6737031]
  98. Schroeder C.E, Foxe J.J. The timing and laminar profile of converging inputs to multisensory areas of the macaque neocortex. Brain Research Cognitive Brain Research. 2002;14:187–98. [PubMed: 12063142]
  99. Schroeder C.E, Lakatos P, Kajikawa Y, Partan S, Puce A. Neuronal oscillations and visual amplification of speech. Trends in Cognitive Sciences. 2008;12:106–13. [PMC free article: PMC3987824] [PubMed: 18280772]
  100. Shelton B.R, Searle C.L. The influence of vision on the absolute identification of sound-source position. Perception & Psychophysics. 1980;28:589–96. [PubMed: 7208275]
  101. Smiley J.F, Hackett T.A, Ulbert I, editors. et al. Multisensory convergence in auditory cortex, I.Cortical connections of the caudal superior temporal plane in macaque monkeys. Journal of Comparative Neurology. 2007;502:894–923. [PubMed: 17447261]
  102. Stecker G.C, Mickey B.J, Macpherson E.A, Middlebrooks J.C. Spatial sensitivity in field PAF of cat auditory cortex. Journal of Neurophysiology. 2003;89:2889–903. [PubMed: 12611946]
  103. Stein B.E, Stanford T.R. Multisensory intergration: Current issues from the perspective of the single neuron. Nature Reviews. Neuroscience. 2008;9:1477–85. [PubMed: 18354398]
  104. Stein B.E, Huneycutt W.S, Meredith M.A. Neurons and behavior: The same rules of multisensory integration apply. Brain Research. 1988;448:355–8. [PubMed: 3378157]
  105. Stein B.E, Meredith M.A, Huneycott W.S, Mcdade L. Behavioral indices of multisensory integration: Orientation of visual cues is affected by auditory stimuli. Journal of Cognitive Neuroscience. 1989;1:12–24. [PubMed: 23968407]
  106. Sumby W.H, Pollack I. Visual contribution to speech intelligibility in noise. Journal of the Acoustical Society of America. 1954;26:212–15.
  107. Tian B, Reser D, Durham A, Kustov A, Rauschecker J.P. Functional specialization in rhesus monkey auditory cortex. Science. 2001;292:290–3. [PubMed: 11303104]
  108. Thomas H, Tillein J, Heil P, Scheich H. Functional organization of auditory cortex in the mongolian gerbil (Meriones unguiculatus). I. Electrophysiological mapping of frequency representation and distinction of fields. European Journal of Neuroscience. 1993;5:882–97. [PubMed: 8281300]
  109. Wallace M.T, Ramachandran R, Stein B.E. A revised view of sensory cortical parcellation. Proceedings of the National Academy of Sciences of the United States of America. 2004;101:2167–72. [PMC free article: PMC357070] [PubMed: 14766982]
  110. Warren J.D, Griffiths T.D. Distinct mechanisms for processing spatial sequences and pitch sequences in the human auditory brain. Journal of Neuroscience. 2003;23:5799–804. [PubMed: 12843284]
  111. Werner-Reiss U, Kelly K.A, Trause A.S, Underhill A.M, Groh J.M. Eye position affects activity in primary auditory cortex of primates. Current Biology. 2003;13:554–62. [PubMed: 12676085]
  112. Woods T.M, Recanzone G.H. Visually induced plasticity of auditory spatial perception in macaques. Current Biology. 2004;14:1559–64. [PubMed: 15341742]
  113. Woods T.M, Lopez S.E, Long J.H, Rahman J.E, Recanzone G.H. Effects of stimulus azimuth and intensity on the single-neuron activity in the auditory cortex of the alert macaque monkey. Journal of Neurophysiology. 2006;96:3323–37. [PubMed: 16943318]
  114. Wright B.A, Zhang Y. A review of learning with normal and altered sound-localization cues in human adults. International Journal of Audiology. 2006;45 Suppl 1:S92–8. [PubMed: 16938780]
  115. Zvyagintsev M, Nikolaev A.R, Thonnessen H, Sachs O, Dammers J, Mathiak K. Spatially congruent visual motion modulates activity of the primary auditory cortex. Experimental Brain Research. 2009;198:391–402. [PubMed: 19449155]
Copyright © 2012 by Taylor & Francis Group, LLC.
Bookshelf ID: NBK92868PMID: 22593889


  • PubReader
  • Print View
  • Cite this Page

Other titles in this collection

Related information

  • PMC
    PubMed Central citations
  • PubMed
    Links to PubMed

Similar articles in PubMed

See reviews...See all...

Recent Activity

Your browsing activity is empty.

Activity recording is turned off.

Turn recording back on

See more...