NCBI Bookshelf. A service of the National Library of Medicine, National Institutes of Health.

Murray MM, Wallace MT, editors. The Neural Bases of Multisensory Processes. Boca Raton (FL): CRC Press/Taylor & Francis; 2012.

Cover of The Neural Bases of Multisensory Processes

The Neural Bases of Multisensory Processes.

Show details

Chapter 27The Colavita Visual Dominance Effect

, , and .

27.1. INTRODUCTION

Visually dominant behavior has been observed in many different species, including birds, cows, dogs, and humans (e.g., Partan and Marler 1999; Posner et al. 1976; Uetake and Kudo 1994; Wilcoxin et al. 1971). This has led researchers to suggest that visual stimuli may constitute “prepotent” stimuli for certain classes of behavioral responses (see Colavita 1974; Foree and LoLordo 1973; LoLordo 1979; Meltzer and Masaki 1973; Shapiro et al. 1980). One particularly impressive example of vision’s dominance over audition (and more recently, touch) has come from research on the Colavita visual dominance effect (Colavita 1974). In the basic experimental paradigm, participants have to make speeded responses to a random series of auditory (or tactile), visual, and audiovisual (or visuotactile) targets, all presented at a clearly suprathreshold level. Participants are instructed to make one response whenever an auditory (or tactile) target is presented, another response whenever a visual target is presented, and to make both responses whenever the auditory (or tactile) and visual targets are presented at the same time (i.e., on the bimodal target trials). Typically, the unimodal targets are presented more frequently than the bimodal targets (the ratio of 40% auditory—or tactile—targets, 40% visual targets, and 20% bimodal targets has often been used; e.g., Koppen and Spence 2007a, 2007b, 2007c). The striking result to have emerged from a number of studies on the Colavita effect is that although participants have no problem in responding rapidly and accurately to the unimodal targets, they often fail to respond to the auditory (or tactile) targets on the bimodal target trials (see Figure 27.1a and b). It is almost as if the simultaneous presentation of the visual target leads to the “extinction” of the participants’ perception of, and/or response to, the nonvisual target on a proportion of the bimodal trials (see Egeth and Sager 1977; Hartcher-O’Brien et al. 2008; Koppen et al. 2009; Koppen and Spence 2007c).

FIGURE 27.1. Results of experiments conducted by Elcock and Spence (2009) highlighting a significant Colavita visual dominance effect over both audition (a) and touch (b).

FIGURE 27.1

Results of experiments conducted by Elcock and Spence (2009) highlighting a significant Colavita visual dominance effect over both audition (a) and touch (b). Values reported in the graphs refer to the percentage of bimodal target trials in which participants (more...)

Although the majority of research on the Colavita effect has focused on the pattern of errors made by participants in the bimodal target trials, it is worth noting that visual dominance can also show up in reaction time (RT) data. For example, Egeth and Sager (1977) reported that although participants responded more rapidly to unimodal auditory targets than to unimodal visual targets, this pattern of results was reversed on the bimodal target trials—that is, participants responded more rapidly to the visual targets than to the auditory targets. Note that Egeth and Sager made sure that their participants always responded to both the auditory and visual targets on the bimodal trials by presenting each target until the participant had made the relevant behavioral response. * A similar pattern of results in the RT data has also been reported in a number of other studies (e.g., Colavita 1974, 1982; Colavita and Weisberg 1979; Cooper 1998; Koppen and Spence 2007a; Sinnett et al. 2007; Zahn et al. 1994).

In this article, we will focus mainly (although not exclusively) on the Colavita effect present in the error data (in line with the majority of published research on this phenomenon). We start by summarizing the basic findings to have emerged from studies of the Colavita visual dominance effect conducted over the past 35 years or so. By now, many different factors have been investigated in order to determine whether they influence the Colavita effect: Here, they are grouped into stimulus-related factors (such as stimulus intensity, stimulus modality, stimulus type, stimulus position, and bimodal stimulus probability) and task/participant-related factors (such as attention, arousal, task/ response demands, and practice). A range of potential explanations for the Colavita effect are evaluated, and all are shown to be lacking. A new account of the Colavita visual dominance effect is therefore proposed, one that is based on the “biased competition” model put forward by Desimone and Duncan (1995; see also Duncan 1996; Peers et al. 2005). Although this model was initially developed in order to provide an explanation for the intramodal competition taking place between multiple visual object representations in both normal participants and clinical patients (suffering from extinction), here we propose that it can be extended to provide a helpful framework in which to understand what may be going on the Colavita visual dominance effect. In particular, we argue that a form of cross-modal biased competition can help to explain why participants respond to the visual stimulus while sometimes failing to respond to the nonvisual stimulus on the bimodal target trials in the Colavita paradigm. More generally, it is our hope that explaining the Colavita visual dominance effect may provide an important step toward understanding the mechanisms underlying multisensory interactions. First, though, we review the various factors that have been hypothesized to influence the Colavita visual dominance effect.

27.2. BASIC FINDINGS ON COLAVITA VISUAL DOMINANCE EFFECT

27.2.1. Stimulus Intensity

The Colavita visual dominance effect occurs regardless of whether the auditory and visual stimuli are presented at the same (subjectively matched) intensity (e.g., Colavita 1974; Koppen et al. 2009; Zahn et al. 1994) or the auditory stimulus is presented at an intensity that is rated subjectively as being twice that of the visual stimulus (see Colavita 1974, Experiment 2). Hartcher-O’Brien et al. (2008; Experiment 4) have also shown that vision dominates over touch under conditions in which the intensity of the tactile stimulus is matched to that of the visual stimulus (presented at the 75% detection threshold). Taken together, these results suggest that the dominance of vision over both audition and touch in the Colavita paradigm cannot simply be attributed to any systematic differences in the relative intensity of the stimuli that have been presented to participants in previous studies (but see also Gregg and Brogden 1952; O’Connor and Hermelin 1963; Smith 1933).

27.2.2. Stimulus Modality

Although the majority of the research on the Colavita visual dominance effect has investigated the dominance of vision over audition, researchers have recently shown that vision also dominates over touch in normal participants (Hartcher-O’Brien et al. 2008, 2010; Hecht and Reiner 2009; see also Gallace et al. 2007). Costantini et al. (2007) have even reported that vision dominates over touch in extinction patients (regardless of whether the two stimuli were presented from the same position, or from different sides; see also Bender 1952). Interestingly, however, no clear pattern of sensory dominance has, as yet, been observed when participants respond to simultaneously presented auditory and tactile stimuli (see Hecht and Reiner 2009; Occelli et al. 2010; but see Bonneh et al. 2008, for a case study of an autistic child who exhibited auditory dominance over both touch and vision).

Intriguingly, Hecht and Reiner (2009) have recently reported that vision no longer dominates when targets are presented in all three modalities (i.e., audition, vision, and touch) at the same time. In their study, the participants were given a separate button with which to respond to the targets in each modality, and had to press one, two, or three response keys depending on the combination of target modalities that happened to be presented on each trial. Whereas vision dominated over both audition and touch in the bimodal target trials, no clear pattern of dominance was shown on the trimodal target trials (see also Shapiro et al. 1984, Experiment 3). As yet, there is no obvious explanation for this result.

27.2.3. Stimulus Type

The Colavita visual dominance effect has been reported for both onset and offset targets (Colavita and Weisberg 1979; see also Osborn et al. 1963). The effect occurs both with simple stimuli (i.e., tones, flashes of light, and brief taps on the skin) and also with more complex stimuli, including pictures of objects and realistic object sounds, and with auditory and visual speech stimuli (see Koppen et al. 2008; Sinnett et al. 2007, 2008). The Colavita effect not only occurs when the target stimuli are presented in isolation (i.e., in an otherwise dark and silent room), but also when they are embedded within a rapidly presented stream of auditory and visual distractors (Sinnett et al. 2007). Interestingly, however, the magnitude of the Colavita visual dominance effect does not seem to be affected by whether or not the auditory and visual targets on the bimodal trials are semantically congruent (see Koppen et al. 2008).

27.2.4. Stimulus Position

Researchers have also considered what effect, if any, varying either the absolute and/or relative location from which the stimuli are presented might have on performance in the Colavita task. The Colavita visual dominance effect occurs both when the auditory stimuli are presented over headphones and when they are presented from an external loudspeaker placed in front of the participant (Colavita 1974, 1982). Researchers have demonstrated that it does not much matter whether the participants look in the direction of the visual or auditory stimulus or else fixate on some other intermediate location (see Colavita et al. 1976). Vision’s dominance over both audition and touch has also been shown to occur regardless of whether the stimuli are presented from the same spatial location or from different positions (one on either side of fixation), although the Colavita effect is somewhat larger in the former case (see Hartcher-O’Brien et al. 2008, 2010; Koppen and Spence 2007c). Taken together, these results therefore show that varying either the absolute position (e.g., presenting the stimuli from the center vs. in the periphery) or relative position (i.e., presenting the various stimuli from the same or different positions) from which the target stimuli are presented has, at most, a relatively modest impact on the magnitude of the Colavita visual dominance effect (see also Johnson and Shapiro 1989).

27.2.5. Bimodal Stimulus Probability

As already noted, studies on the Colavita visual dominance effect usually present far fewer bimodal targets than unimodal targets. Nevertheless, researchers have shown that a robust Colavita visual dominance effect can still be obtained if the probability of each type of target is equalized (i.e., when 33.3% auditory, 33.3% visual, and 33.3% bimodal targets are presented; see Koppen and Spence 2007a). Koppen and Spence (2007d) investigated the effect of varying the probability of bimodal target trials on the Colavita visual dominance effect (while keeping the relative proportion of unimodal auditory and visual target trials matched). * They found that although a significant Colavita effect was demonstrated whenever the bimodal targets were presented on 60% or less of the trials, vision no longer dominated when the bimodal targets were presented on 90% of the trials (see also Egeth and Sager 1974; Manly et al. 1999; Quinlan 2000). This result suggests that the Colavita effect is not caused by stimulus-related (i.e., sensory) factors, since these should not have been affected by any change in the probability of occurrence of bimodal targets (cf. Odgaard et al. 2003, 2004, on this point). Instead, the fact that the Colavita effect disappears if the bimodal targets are presented too frequently (i.e., on too high a proportion of the trials) would appear to suggest that response-related factors (linked to the probability of participants making bimodal target responses) are likely to play an important role in helping to explain the Colavita effect (see also Gorea and Sagi 2000).

27.2.6. Response Demands

The majority of studies on the Colavita visual dominance effect have been conducted under conditions in which participants were given a separate response key with which to respond to the targets presented in each sensory modality. Normally, participants are instructed to respond to the (relatively infrequent) bimodal targets by pressing both response keys. Similar results have, however, now also been obtained under conditions in which the participants are given a separate response key with which to respond to the bimodal targets (Koppen and Spence 2007a; Sinnett et al. 2007). This result rules out the possibility that the Colavita effect is simply caused by participants having to make two responses at more or less the same time. Surprisingly, Colavita (1974; Experiment 4) showed that participants still made a majority of visual responses after having been explicitly instructed to respond to the bimodal targets by pressing the auditory response key instead.

Koppen et al. (2008) have also reported that the Colavita effect occurs when participants are instructed to press one button whenever they either see or hear a dog, another button whenever they see or hear a cat, and to make both responses whenever a cat and a dog are presented at the same time. Under such conditions, the visual presentation of the picture of one of these animals resulted in participants failing to respond to the sound of the other animal (be it the woofing of the dog or the meowing of the cat) on 10% more of the trials than they failed to respond to the identity of the visually presented animal. Taken together, these results therefore confirm the fact that the Colavita visual dominance effect occurs under a variety of different task demands/response requirements (i.e., it occurs no matter whether participants respond to the sensory modality or semantic identity of the target stimuli).

27.2.7. Attention

Originally, researchers thought that the Colavita visual dominance effect might simply reflect a predisposition by participants to direct their attention preferentially toward the visual modality (Colavita 1974; Posner et al. 1976). Posner et al.’s idea was that people endogenously (or voluntarily) directed their attention toward the visual modality in order to make up for the fact that visual stimuli are generally less alerting than stimuli presented in the other modalities (but see Spence et al. 2001b, footnote 5). Contrary to this suggestion, however, a number of more recent studies have actually shown that although the manipulation of a person’s endogenous attention can certainly modulate the extent to which vision dominates over audition, it cannot in and of itself be used to reverse the Colavita effect. That is, even when a participant’s attention is directed toward the auditory modality (i.e., by verbally instructing them to attend to audition or by presenting unimodal auditory targets much more frequently than unimodal visual targets), people still exhibit either visually dominant behavior or else their behavior shows no clear pattern of dominance (see Koppen and Spence 2007a, 2007d; Sinnett et al. 2007). These results therefore demonstrate that any predisposition that participants might have to direct their attention voluntarily (or endogenously) toward the visual modality cannot explain why vision always seems to dominate in the Colavita visual dominance effect.

De Reuck and Spence (2009) recently investigated whether varying the modality of a secondary task would have any effect on the magnitude of the Colavita visual dominance effect. To this end, a video game (“Food boy” by T3Software) and a concurrent auditory speech stream (consisting of pairs of auditory words delivered via a central loudspeaker) were presented in the background while participants performed the two-response version of the Colavita task (i.e., pressing one key in response to auditory targets, another key in response to visual targets, and both response keys on the bimodal target trials; the auditory targets in this study consisted of a 4000-Hz pure tone presented from a loudspeaker cone placed in front of the computer screen, whereas the visual target consisted of the illumination of a red light-emitting device (LED), also mounted in front of the computer screen). In the condition involving the secondary visual task, the participants performed the Colavita task with their right hand while playing the video game with their left hand (note that the auditory distracting speech streams were presented in the background, although they were irrelevant in this condition and so could be ignored). The participants played the video game using a computer mouse to control a character moving across the bottom of the computer screen. The participants had to “swallow” as much of the food dropping from the top of the screen as possible, while avoiding any bombs that happened to fall. In the part of the study involving an auditory secondary task, the video game was run in the demonstration mode to provide equivalent background visual stimulation to the participants who now had to respond by pressing a button with their left hand whenever they heard an animal name in the auditory stream.

The results showed that the modality of the secondary task (auditory or visual) did not modulate the magnitude of the Colavita visual dominance effect significantly, that is, the participants failed to respond to a similar number of the auditory stimuli regardless of whether they were performing a secondary task that primarily involved participants having to attend to the auditory or visual modality. De Reuck and Spence’s (2009) results therefore suggest that the Colavita visual dominance effect may be insensitive to manipulations of participants’ attention toward either the auditory or visual modality that are achieved by varying the requirements of a simultaneously performed secondary task (see Spence and Soto-Faraco 2009).

Finally, Koppen and Spence (2007a) have shown that exogenously directing a participant’s attention toward either the auditory or visual modality via the presentation of a task-irrelevant nonpredictive auditory or visual cue 200 ms before the onset of the target (see Rodway 2005; Spence et al. 2001a; Turatto et al. 2002) has only a marginal effect on the magnitude of vision’s dominance over audition (see also Golob et al. 2001). Taken together, the results reported in this section therefore highlight the fact that although attentional manipulations (be they exogenous or endogenous) can sometimes be used to modulate, or even to eliminate, the Colavita visual dominance effect, they cannot be used to reverse it.

27.2.8. Arousal

Early animal research suggested that many examples of visual dominance could be reversed under conditions in which an animal was placed in a highly aroused state (i.e., when, for example, fearful of the imminent presentation of an electric shock; see Foree and LoLordo 1973; LoLordo and Furrow 1976; Randich et al. 1978). It has been reported that although visual stimuli tend to control appetitive behaviors, auditory stimuli tend to control avoidance behaviors in many species. Shapiro et al. (1984) extended the idea that changes in the level of an organism’s arousal might change the pattern of sensory dominance in the Colavita task to human participants (see also Johnson and Shapiro 1989; Shapiro and Johnson 1987). They demonstrated what looked like auditory dominance (i.e., participants making more auditory-only than visual-only responses in the Colavita task) under conditions in which their participants were aversively motivated (by the occurrence of electric shock, or to a lesser extent by the threat of electric shock, or tactile stimulation, presented after the participants’ response on a random 20% of the trials).

It should, however, be noted that no independent measure of the change in a participant’s level of arousal (i.e., such as a change in their galvanic skin response) was provided in this study. What is more, Shapiro et al.’s (1984) participants were explicitly told to respond to the stimulus that they perceived first on the bimodal target trials, that is, the participants effectively had to perform a temporal order judgment (TOJ) task. What this means in practice is that their results (and those from the study of Shapiro and Johnson (1987) and Johnson and Shapiro (1989), in which similar instructions were given) may actually reflect the effects of arousal on “prior entry” (see Spence 2010; Van Damme et al. 2009b), rather than, as the authors argued, the effects of arousal on the Colavita visual dominance effect.

Indeed, the latest research has demonstrated that increased arousal can lead to the prior entry of certain classes of stimuli over others (when assessed by means of a participant’s responses on a TOJ task; Van Damme et al. 2009b). In Van Damme et al.’s study, auditory and tactile stimuli delivered from close to one of the participant’s hands were prioritized when an arousing picture showing physical threat to a person’s bodily tissues was briefly flashed beforehand from the same (rather than opposite) location. Meanwhile, Van Damme et al. (2009a) have shown that, when participants are instructed to respond to both of the stimuli in the bimodal trials, rather than just to the stimulus that the participant happens to have perceived first, the effects of arousal on the Colavita visual dominance effect are far less clear-cut (we return later to the question of what role, if any, prior entry plays in the Colavita visual dominance effect).

Elcock and Spence (2009) recently investigated the consequences for the Colavita effect of pharmacologically modulating the participants’ level of arousal by administering caffeine. Caffeine is known to increase arousal and hence, given Shapiro et al.’s (1984) research, ingesting caffeine might be expected to modulate the magnitude of the Colavita visual dominance effect (Smith et al. 1992). * To this end, 15 healthy participants were tested in a within-participants, double-blind study, in which a 200-mg caffeine tablet (equivalent to drinking about two cups of coffee) was taken 40 min before one session of the Colavita task and a visually identical placebo pill was taken before the other session (note that the participants were instructed to refrain from consuming any caffeine in the morning before taking part in the study). The Colavita visual dominance effect was unaffected by whether the participants had ingested the caffeine tablet or the placebo (see Figure 27.1c and d). Taken together, the results reported in this section would therefore appear to suggest that, contrary to Shapiro et al.’s early claim, the magnitude of the Colavita visual dominance effect is not affected by changes in a participant’s level of arousal.

27.2.9. Practice Effects

The largest Colavita visual dominance effects have been reported in studies in which only a small number of bimodal target trials were presented. In fact, by far the largest effects on record were reported by Frank B. Colavita himself in his early research (see Koppen and Spence 2007a, Table 1, for a review). In these studies, each participant was only ever presented with a maximum of five or six bimodal targets (see Colavita 1974, 1982; Colavita et al. 1976; Colavita and Weisberg 1979). Contrast this with the smaller Colavita effects that have been reported in more recent research, where as many as 120 bimodal targets were presented to each participant (e.g., Hartcher-O’Brien et al. 2008; Koppen et al. 2008; Koppen and Spence 2007a, 2007c). This observation leads on to the suggestion that the Colavita visual dominance effect may be more pronounced early on in the experimental session (see also Kristofferson 1965). * That said, significant Colavita visual dominance effects have nevertheless still been observed in numerous studies where participants’ performance has been averaged over many hundreds of trials. Here, it may also be worth considering whether any reduction in the Colavita effect resulting from increasing the probability of (and/or practice with responding to) bimodal stimuli may also be related to the phenomenon of response coupling (see Ulrich and Miller 2008). That is, the more often two independent target stimuli happen to be presented at exactly the same time, the more likely it is that the participant will start to couple (i.e., program) their responses to the two stimuli together.

In the only study (as far as we are aware) to have provided evidence relevant to the question of the consequence of practice on the Colavita visual dominance effect, the vigilance performance of a group of participants was assessed over a 3-h period (Osborn et al. 1963). The participants in this study had to monitor a light and sound source continuously for the occasional (once every 2½ min) brief (i.e., lasting only 41 ms) offset of either or both of the stimuli. The participants were instructed to press one button whenever the light was extinguished and another button whenever the sound was interrupted. The results showed that although participants failed to respond to more of the auditory than visual targets during the first 30-min session (thus showing a typical Colavita visual dominance effect), this pattern of results reversed in the final four 30-min sessions (i.e., participants made more auditory-only than visual-only responses on the bimodal target trials; see Osborn et al. 1963; Figure 27.2). It is, however, unclear whether these results necessarily reflect the effects of practice on the Colavita visual dominance effect, or whether instead they may simply highlight the effects of fatigue or boredom after the participants had spent several hours on the task (given that auditory events are more likely to be responded to than visual events should the participants temporarily look away or else close their eyes).

FIGURE 27.2. Graph highlighting the results of Koppen and Spence’s (2007b) study of Colavita effect in which auditory and visual targets on bimodal target trials could be presented at any one of 10 SOAs.

FIGURE 27.2

Graph highlighting the results of Koppen and Spence’s (2007b) study of Colavita effect in which auditory and visual targets on bimodal target trials could be presented at any one of 10 SOAs. Although a significant visual dominance effect was observed (more...)

27.3. INTERIM SUMMARY

To summarize, the latest research has confirmed the fact that the Colavita visual dominance effect is a robust empirical phenomenon. The basic Colavita effect—defined here in terms of participants failing to respond to the nonvisual stimulus more often than they fail to respond to the visual stimulus on the bimodal audiovisual or visuotactile target trials—has now been replicated in many different studies, and by a number of different research groups (although it is worth noting that the magnitude of the effect has fluctuated markedly from one study to the next). That said, the Colavita effect appears to be robust to a variety of different experimental manipulations (e.g., of stimulus intensity, stimulus type, stimulus position, response demands, attention, arousal, etc.). Interestingly, though, while many experimental manipulations have been shown to modulate the size of the Colavita visual dominance effect, and a few studies have even been able to eliminate it entirely, only two of the studies discussed thus far have provided suggestive evidence regarding a reversal of the Colavita effect in humans (i.e., evidence that is consistent with, although not necessarily providing strong support for, auditory dominance; see Osborn et al. 1963; Shapiro et al. 1984).

Having reviewed the majority of the published research on the Colavita visual dominance effect, and having ruled out accounts of the effect in terms of people having a predisposition to attend endogenously to the visual modality (see Posner et al. 1976), differences in stimulus intensity (Colavita 1974), and/or difficulties associated with participants having to make two responses at the same time on the bimodal target trials (Koppen and Spence 2007a), how should the effect be explained? Well, researchers have recently been investigating whether the Colavita effect can be accounted for, at least in part, by the prior entry of the visual stimulus to participants’ awareness (see Spence 2010; Spence et al. 2001; Titchener 1908). It is to this research that we now turn.

27.4. PRIOR ENTRY AND COLAVITA VISUAL DOMINANCE EFFECT

Koppen and Spence (2007b) investigated whether the Colavita effect might result from the prior entry of the visual stimulus into participants’ awareness on some proportion of the bimodal target trials. That is, even though the auditory and visual stimuli were presented simultaneously in the majority of published studies of the Colavita effect, research elsewhere has shown that a visual stimulus may be perceived first under such conditions (see Rutschmann and Link 1964). In order to evaluate the prior entry account of the Colavita visual dominance effect, Koppen and Spence assessed participants’ perception of the temporal order of pairs of auditory and visual stimuli that had been used in another part of the study to demonstrate the typical Colavita visual dominance effect. * Psychophysical analysis of participants’ TOJ performance showed that when the auditory and visual stimuli were presented simultaneously, participants actually judged the auditory stimulus to have been presented slightly, although not significantly, ahead of the visual stimulus (i.e., contrary to what would have been predicted according to the prior entry account; but see Exner 1875 and Hirsh and Sherrick 1961, for similar results; see also Jaśkowski 1996, 1999; Jaśkowski et al. 1990).

It is, however, important to note that there is a potential concern here regarding the interpretation of Koppen and Spence’s (2007b) findings. Remember that the Colavita visual dominance effect is eliminated when bimodal audiovisual targets are presented too frequently (e.g., see Section 27.2.5). Crucially, Koppen and Spence looked for any evidence of the prior entry of visual stimuli into awareness in their TOJ study under conditions in which a pair of auditory and visual stimuli were presented on each and every trial. The possibility therefore remains that visual stimuli may only be perceived before simultaneously presented auditory stimuli under those conditions in which the occurrence of bimodal stimuli is relatively rare (cf. Miller et al. 2009). Thus, in retrospect, Koppen and Spence’s results cannot be taken as providing unequivocal evidence against the possibility that visual stimuli have prior entry into participants’ awareness on the bimodal trials in the Colavita paradigm. Ideally, future research will need to look for any evidence of visual prior entry under conditions in which the bimodal targets (in the TOJ task) are actually presented as infrequently as when the Colavita effect is demonstrated behaviorally (i.e., when the bimodal targets requiring a detection/discrimination response are presented on only 20% or so of the trials).

Given these concerns over the design (and hence interpretation) of Koppen and Spence’s (2007b) TOJ study, it is interesting to note that Lucey and Spence (2009) were recently able to eliminate the Colavita visual dominance effect by delaying the onset of the visual stimulus by a fixed 50 ms with respect to the auditory stimuli on the bimodal target trials. Lucey and Spence used a between-participants experimental design in which one group of participants completed the Colavita task with synchronous auditory and visual targets on the bimodal trials (as in the majority of previous studies), whereas for the other group of participants, the onset of the visual target was always delayed by 50 ms with respect to that of the auditory target. The apparatus and materials were identical to those used by Elcock and Spence (2009; described earlier) although the participants in Lucey and Spence’s study performed the three-button version of the audiovisual Colavita task (i.e., in which participants had separate response keys for auditory, visual, and bimodal targets). The results revealed that although participants made significantly more vision-only than auditory-only responses in the synchronous bimodal condition (10.3% vs. 2.4%, respectively), no significant Colavita visual dominance effect was reported when the onset of the visual target was delayed (4.6% vs. 2.9%, respectively; n.s.). These results therefore demonstrate that the Colavita visual dominance effect can be eliminated by presenting the auditory stimulus slightly ahead of the visual stimulus. The critical question here, following on from Lucey and Spence’s results, is whether auditory dominance would have been elicited had the auditory stimulus led the visual stimulus by a greater interval.

Koppen and Spence (2007b) have provided an answer to this question. In their study of the Colavita effect, the auditory and visual stimuli on the bimodal target trials were presented at one of 10 stimulus onset asynchronies (SOAs; from auditory leading by 600 ms through to vision leading by 600 ms). Koppen and Spence found that the auditory lead needed in order to eliminate the Colavita visual dominance effect on the bimodal target trials was correlated with the SOA at which participants reliably started to perceive the auditory stimulus as having been presented before the visual stimulus (defined as the SOA at which participants make 75% audition first responses; see Koppen and Spence 2007b; Figure 27.3). This result therefore suggests that the prior entry of the visual stimulus to awareness plays some role in its dominance over audition in the Colavita effect. That said, however, Koppen and Spence also found that auditory targets had to be presented 600 ms before visual targets in order for participants to make significantly more auditory-only than visual only responses on the bimodal target trials (although a similar nonsignificant trend toward auditory dominance was also reported at an auditory lead of 300 ms; see Figure 27.2).

FIGURE 27.3. (a) Schematic illustration of the results of Sinnett et al.

FIGURE 27.3

(a) Schematic illustration of the results of Sinnett et al.’s (2008; Experiment 2) speeded target detection study. The figure shows how the presentation of an accessory sound facilitates visual RTs (RTV(A)), whereas the presentation of an accessory (more...)

It is rather unclear, however, what exactly caused the auditorily dominant behavior observed at the 600 ms SOA in Koppen and Spence’s (2007b) study. This (physical) asynchrony between the auditory and visual stimuli is far greater than any shift in the perceived timing of visual relative to auditory stimuli that might reasonably be expected due to the prior entry of the visual stimulus to awareness when the targets were actually presented simultaneously (see Spence 2010). In fact, this SOA is also longer than the mean RT of participants’ responses to the unimodal auditory (440 ms) targets. Given that the mean RT for auditory only responses on the bimodal target trials was only 470 ms (i.e., 30 ms longer, on average, than the correct responses on the bimodal trials; see Koppen and Spence 2007b, Figure 1 and Table 1), one can also rule out the possibility that this failure to report the visual stimulus occurred on trials in which the participants made auditory responses that were particularly slow. Therefore, given that the visual target on the bimodal trials (in the 600 ms SOA vision-lagging condition) was likely being extinguished by an already-responded-to auditory target, one might think that this form of auditory dominance reflects some sort of refractory period effect (i.e., resulting from the execution of the participants’ response to the first target; see Pashler 1994; Spence 2008), rather than the Colavita effect proper.

In summary, although Koppen and Spence’s (2007b) results certainly do provide an example of auditory dominance, the mechanism behind this effect is most probably different from the one causing the visual dominance effect that has been reported in the majority of studies (of the Colavita effect), where the auditory and visual stimuli were presented simultaneously (see also Miyake et al. 1986). Thus, although recent research has shown that delaying the presentation of the visual stimulus can be used to eliminate the Colavita visual dominance effect (see Koppen and Spence 2007b; Lucey and Spence 2009), and although the SOA at which participants reliably start to perceive the auditory target as having been presented first correlates with the SOA at which the Colavita visual dominance effect no longer occurs (Koppen and Spence 2007b), we do not, as yet, have any convincing evidence that auditory dominance can be observed in the Colavita paradigm by presenting the auditory stimulus slightly before the visual stimulus on the bimodal target trials (i.e., at SOAs where the visual target is presented before the participants have initiated/executed their response to the already-presented auditory target). That is, to date, no simple relationship has been demonstrated between the SOA on the audiovisual target trials in the Colavita paradigm and modality dominance. Hence, we need to look elsewhere for an explanation of vision’s advantage in the Colavita visual dominance effect. Recent progress in understanding what may be going on here has come from studies looking at the effect of accessory stimuli presented in one modality on participants’ speeded responding to targets presented in another modality (Sinnett et al. 2008), and from studies looking at the sensitivity and criterion of participants’ responses in the Colavita task (Koppen et al. 2009).

27.5. EXPLAINING THE COLAVITA VISUAL DOMINANCE EFFECT

27.5.1. Accessory Stimulus Effects and Colavita Effect

One of the most interesting recent developments in the study of the Colavita effect comes from an experiment reported by Sinnett et al. (2008; Experiment 2). The participants in this study had to make speeded target detection responses to either auditory or visual targets. An auditory stimulus was presented on 40% of the trials, a visual stimulus was presented on a further 40% of the trials, and both stimuli were presented simultaneously on the remaining 20% of trials (i.e., just as in a typical study of the Colavita effect; note, however, that this task can also be thought of as a kind of go/ no-go task; see Egeth and Sager 1977; Miller 1986; Quinlan 2000). The participants responded significantly more rapidly to the visual targets when they were accompanied by an accessory auditory stimulus than when they were presented by themselves (see Figure 27.3a). By contrast, participants’ responses to the auditory targets were actually slowed by the simultaneous presentation of an accessory visual stimulus (cf. Egeth and Sager 1977).

How might the fact that the presentation of an auditory accessory stimulus speeds participants’ visual detection/discrimination responses, whereas the presentation of a visual stimulus slows their responses to auditory stimuli be used to help explain the Colavita visual dominance effect? Well, let us imagine that participants set one criterion for initiating their responses to the relatively common unimodal visual targets and another criterion for initiating their responses to the equally common unimodal auditory targets. Note that the argument here is phrased in terms of changes in the criterion for responding set by participants, rather than in terms of changes in the perceptual threshold, given the evidence cited below that behavioral responses can sometimes be elicited under conditions in which participants remain unaware (i.e., they have no conscious access to the inducing stimulus). According to Sinnett et al.’s (2008) results, the criterion for initiating a speeded response to the visual targets should be reached sooner on the relatively infrequent bimodal trials than on the unimodal visual trials, whereas it should be reached more slowly (on the bimodal than on the unimodal trials) for auditory targets.

There are at least two conceptually simple means by which such a pattern of behavioral results could be achieved. First, the participants could lower their criterion for responding to the visual targets on the bimodal trials while simultaneously raising their criterion for responding to the auditory target (see Figure 27.3b). Alternatively, however, the criterion for initiating a response might not change but the presentation of the accessory stimulus in one modality might instead have a cross-modal effect on the rate of information accrual (R) within the other modality (see Figure 27.3c). The fact that the process of information accrual (like any other internal process) is likely to be a noisy one might then help to explain why the Colavita effect is only observed on a proportion of the bimodal target trials. Evidence that is seemingly consistent with both of these simple accounts can be found in the literature.

In particular, evidence consistent with the claim that bimodal (as compared to unimodal) stimulation can result in a change in the rate of information accrual comes from an older go/no-go study reported by Miller (1986). Unimodal auditory and unimodal visual target stimuli were presented randomly in this experiment together with trials in which both stimuli were presented at one of a range of different SOAs (0–167 ms). The participants had to make a simple speeded detection response whenever a target was presented (regardless of whether it was unimodal or bimodal). Catch trials, in which no stimulus was presented (and no response was required), were also included. Analysis of the results provided tentative evidence that visual stimuli needed less time to reach the criterion for initiating a behavioral response (measured from the putative onset of response-related activity) compared to the auditory stimuli on the redundant bimodal target trials—this despite the fact that the initiation of response-related activation after the presentation of an auditory stimulus started earlier in time than following the presentation of a visual stimulus (see Miller 1986, pp. 340–341). Taken together, these results therefore suggest that stimulus-related information accrues more slowly for auditory targets in the presence (vs. absence) of concurrent visual stimuli than vice versa, just as highlighted in Figure 27.3c. Similarly, Romei et al.’s (2009) recent results showing that looming auditory signals enhance visual excitability in a preperceptual manner can also be seen as being consistent with the information accrual account. However, results arguing for the inclusion of some component of criterion shifting into one’s model of the Colavita visual dominance effect (although note that the results are inconsistent with the simple criterion-shifting model put forward in Figure 27.3b) comes from a more recent study reported by Koppen et al. (2009).

27.5.2. Perceptual and Decisional Contributions to Colavita Visual Dominance Effect

Koppen et al. (2009) recently explicitly assessed the contributions of perceptual (i.e., threshold) and decisional (i.e., criterion-related) factors to the Colavita visual dominance effect in a study in which the intensities of the auditory and visual stimuli were initially adjusted until participants were only able to detect them on 75% of the trials. Next, a version of the Colavita task was conducted using these near-threshold stimuli (i.e., rather than the clearly suprathreshold stimuli that have been utilized in the majority of previous studies). A unimodal visual target was presented on 25% of the trials, a unimodal auditory target on 25% of trials, a bimodal audiovisual target on 25% of trials (and no target was presented on the remaining 25% of trials). The task of reporting which target modalities (if any) had been presented in each trial was unspeeded and the participants were instructed to refrain from responding on those trials in which no target was presented.

Analysis of Koppen et al.’s (2009) results using signal detection theory (see Green and Swets 1966) revealed that although the presentation of an auditory target had no effect on visual sensitivity, the presentation of a visual target resulted in a significant drop in participants’ auditory sensitivity (see Figure 27.4a; see also Golob et al. 2001; Gregg and Brogden 1952; Marks et al. 2003; Odgaard et al. 2003; Stein et al. 1996; Thompson et al. 1958). These results therefore show that the presentation of a visual stimulus can lead to a small, but significant, lowering of sensitivity to a simultaneously presented auditory stimulus, at least when the participants’ task involves trying to detect which target modalities (if any) have been presented. * Koppen et al.’s results suggest that only a relatively small component of the Colavita visual dominance effect may be attributable to the asymmetrical cross-modal effect on auditory sensitivity (i.e., on the auditory perceptual threshold) that results from the simultaneous presentation of a visual stimulus. That is, the magnitude of the sensitivity drop hardly seems large enough to account for the behavioral effects observed in the normal speeded version of the Colavita task.

FIGURE 27.4. Summary of mean sensitivity (d’) values (panel a) and criterion (c) (panel b) for unimodal auditory, unimodal visual, bimodal auditory, and bimodal visual targets in Koppen et al.

FIGURE 27.4

Summary of mean sensitivity (d’) values (panel a) and criterion (c) (panel b) for unimodal auditory, unimodal visual, bimodal auditory, and bimodal visual targets in Koppen et al.’s (2009) signal detection study of the Colavita visual (more...)

The more important result to have emerged from Koppen et al.’s (2009) study in terms of the argument being developed here was the significant drop in participants’ criterion for responding on the bimodal (as compared to the unimodal) target trials. Importantly, this drop was significantly larger for visual than for auditory targets (see Figure 27.4b). The fact that the criterion dropped for both auditory and visual targets is inconsistent with the simple criterion shifting account of the asymmetrical cross-modal effects highlighted by Sinnett et al. (2008) that were put forward in Figure 27.3b. In fact, when the various results now available are taken together, the most plausible model of the Colavita visual dominance effect would appear to be one in which an asymmetrical lowering of the criterion for responding to auditory and visual targets (Koppen et al. 2009), is paired with an asymmetrical cross-modal effect on the rate of information accrual (Miller 1986; see Figure 27.3d).

However, although the account outlined in Figure 27.3d may help to explain why it is that a participant will typically respond to the visual stimulus first on the bimodal target trials (despite the fact that the auditory and visual stimuli are actually presented simultaneously), it does not explain why participants do not quickly recognize the error of their ways (after making a vision-only response, say), and then quickly initiate an additional auditory response. The participants certainly had sufficient time in which to make a response before the next trial started in many of the studies where the Colavita effect has been reported. For example, in Koppen and Spence’s (2007a, 2007b, 2007c) studies, the intertarget interval was in the region of 1500–1800 ms, whereas mean vision-only response latencies fell in the 500–700 ms range.

27.5.3. Stimulus, (Perception), and Response?

We believe that in order to answer the question of why participants fail to make any response to the auditory (or tactile) targets on some proportion of the bimodal target trials in the Colavita paradigm, one has to break with the intuitively appealing notion that there is a causal link between (conscious) perception and action. Instead, it needs to be realized that our responses do not always rely on our first becoming aware of the stimuli that have elicited those responses. In fact, according to Neumann (1990), the only causal link that exists is the one between a stimulus and its associated response. Neumann has argued that conscious perception should not always be conceptualized as a necessary stage in the chain of human information processing. Rather, he suggests that conscious perception can, on occasion, be bypassed altogether. Support for Neumann’s view that stimuli can elicit responses in the absence of awareness comes from research showing, for example, that participants can execute rapid and accurate discrimination responses to masked target stimuli that they are subjectively unaware of (e.g., Taylor and McCloskey 1996). The phenomenon of blindsight is also pertinent here (e.g., see Cowey and Stoerig 1991). Furthermore, researchers have shown that people sometimes lose their memory for the second of two stimuli as a result of their having executed a response to the first stimulus (Crowder 1968; Müsseler and Hommel 1997a, 1997b; see also Bridgeman 1990; Ricci and Chatterjee 2004; Rizzolatti and Berti 1990). On the basis of such results, then, our suggestion is that a participant’s awareness (of the target stimuli) in the speeded version of the Colavita paradigm may actually be modulated by the responses that they happen to make (select or initiate) on some proportion of the trials, rather than necessarily always being driven by their conscious perception of the stimuli themselves (see also Hefferline and Perera 1963).

To summarize, when participants try to respond rapidly in the Colavita visual dominance task, they may sometimes end up initiating their response before becoming aware of the stimulus (or stimuli) that have elicited that response. Their awareness of which stimuli have, in fact, been presented is then constrained by the response(s) that they actually happen to make. In other words, if (as a participant) I realize that I have made (or am about to make) a vision-only response, it would seem unsurprising that I only then become aware of the visual target, even if an auditory target had also been presented at the same time (although it perhaps reached the threshold for initiating a response more slowly than the visual stimulus; see above). Here, one might even consider the possibility that participants simply stop processing (or stop responding to) the target stimulus (or stimuli) after they have selected/triggered a response (to the visual target; i.e., perhaps target processing reflects a kind of self-terminating processing). Sinnett et al.’s (2008) research is crucial here in showing that, as a result of the asymmetrical cross-modal effects of auditory and visual stimuli on each other, the first response that a participant makes on a bimodal target trial is likely to be to a visual (rather than an auditory) stimulus.

If this hypothesis regarding people’s failure to respond to some proportion of the auditory (or tactile) stimuli on the bimodal trials in the Colavita paradigm were to be correct, one would expect the fastest visual responses to occur on those bimodal trials in which participants make a visual-only response. Koppen and Spence’s (2007a; Experiment 3) results show just such a result in their three-response study of the Colavita effect (i.e., where participants made one response to auditory targets, one to visual targets, and a third to the bimodal targets; note, however, that the participants did not have the opportunity to respond to the visual and auditory stimuli sequentially in this study). In Koppen and Spence’s study, the visual-only responses on the bimodal target trials were actually significantly faster, on average (mean RT = 563 ms), than the visual-only responses on unimodal visual trials (mean RT = 582 ms; see Figure 27.5). This result therefore demonstrates that even though participants failed to respond to the auditory target, its presence nevertheless still facilitated their behavioral performance. Finally, the vision-only responses (on the bimodal trials) were also found, on average, to be significantly faster than the participants’ correct bimodal responses on the bimodal target trails (mean = 641 ms).

FIGURE 27.5. Schematic timeline showing the mean latency of participants’ responses (both correct and incorrect responses) in Koppen et al.

FIGURE 27.5

Schematic timeline showing the mean latency of participants’ responses (both correct and incorrect responses) in Koppen et al.’s (2007a) three-button version of the Colavita effect. Significant differences between particular conditions (more...)

Interestingly, however, participants’ auditory-only responses on the bimodal target trials in Koppen and Spence’s (2007a) study were significantly slower, on average, than on the unimodal auditory target trials (mean RTs of 577 and 539 ms, respectively). This is the opposite pattern of results to that seen for the visual target detection data (i.e., a bimodal slowing of responding for auditory targets paired with a bimodal speeding of responding to the visual targets). This result provides additional evidence for the existence of an asymmetrical cross-modal effect on the rate of information accrual). Indeed, taken together, these results mirror those reported by Sinnett et al. (2008) in their speeded target detection task, but note here that the data come from a version of the Colavita task instead. Thus, it really does seem as though the more frequent occurrence of vision-only as compared to auditory-only responses on the bimodal audiovisual target trials in the Colavita visual dominance paradigm is tightly linked to the speed with which a participant initiates his/her response. When participants respond rapidly, they are much more likely to make an erroneous visual-only response than to make an erroneous auditory-only response. *

27.6. BIASED (OR INTEGRATED) COMPETITION AND COLAVITA VISUAL DOMINANCE EFFECT

How can the asymmetric cross-modal effects of simultaneously presented auditory and visual targets on each other (that were highlighted in the previous section) be explained? We believe that a fruitful approach may well come from considering them in the light of the biased (or integrated) competition hypothesis (see Desimone and Duncan 1995; Duncan 1996). According to Desimone and Duncan, brain systems (both sensory and motor) are fundamentally competitive in nature. What is more, within each system, a gain in the activation of one object/event representation always occurs at a cost to others. That is, the neural representation of different objects/events is normally mutually inhibitory. An important aspect of Desimone and Duncan’s biased competition model relates to the claim that the dominant neural representation suppresses the neural activity associated with the representation of the weaker stimulus (see Duncan 1996). In light of the discussion in the preceding section (see Section 27.5.2), one might think of biased competition as affecting the rate of information accrual, changing the criterion for responding, and/or changing perceptual sensitivity (but see Gorea and Sagi 2000, 2002). An extreme form of this probabilistic winner-takes-all principle might therefore help to explain why it is that the presentation of a visual stimulus can sometimes have such a profound effect on people’s awareness of the stimuli coded by a different brain area (i.e., modality; see also Hahnloser et al. 1999).

Modality-based biased competition can perhaps also provide a mechanistic explanation for the findings of a number of other studies of multisensory information processing. For example, over the years, many researchers have argued that people’s attention is preferentially directed toward the visual modality when pairs of auditory and visual stimuli are presented simultaneously (e.g., see Falkenstein et al. 1991; Golob et al. 2001; Hohnsbein and Falkenstein 1991; Hohnsbein et al. 1991; Oray et al. 2002). As Driver and Vuilleumier (2001, p. 75) describe the biased (or integrated) competition hypothesis: “ . . . multiple concurrent stimuli always compete to drive neurons and dominate the networks (and ultimately to dominate awareness and behavior).” They continue: “various phenomena of ‘attention’ are cast as emergent properties of whichever stimuli happen to win the competition.” In other words, particularly salient stimuli will have a competitive advantage and may thus tend to “attract attention” on purely bottom-up grounds. Visual stimuli might then, for whatever reason (see below), constitute a particularly salient class of stimuli. Such stimulus-driven competition between the neural activation elicited by the auditory (or tactile) and visual targets on bimodal target trials might also help to explain why the attentional manipulations that have been utilized previously have proved so ineffective in terms of reversing the Colavita visual dominance effect (see Koppen and Spence 2007d; Sinnett et al. 2007). That is, although the biasing of a participant’s attention toward one sensory modality (in particular, the nonvisual modality) before stimulus onset may be sufficient to override the competitive advantage resulting from any stimulus-driven biased competition (see McDonald et al. 2005; Spence 2010; Vibell et al. 2007), it cannot reverse it.

27.6.1. Putative Neural Underpinnings of Modality-Based Biased Competition

Of course, accounting for the Colavita visual dominance effect in terms of biased competition does not itself explain why it is the visual stimulus that always wins the competition more frequently than the nonvisual stimulus. Although a satisfactory neurally inspired answer to this question will need to await future research, it is worth noting here that recent research has highlighted the importance of feedback activity from higher order to early sensory areas in certain aspects of visual awareness (e.g., Lamme 2001; Lamme et al. 2000; Pascual-Leone and Walsh 2001; but see also Macknik 2009; Macknik and Martinez-Conde 2007, in press). It is also pertinent to note that far more of the brain is given over to the processing of visual stimuli than to the processing of stimuli from the other sensory modalities. For example, Sereno et al. (1995) suggest that nearly half of the cortex is involved in the processing of visual information. Meanwhile, Felleman and van Essen (1991) point out that in the macaque there are less than half the number of brain areas involved in the processing of tactile information as involved in the processing of visual information. In fact, in their authoritative literature review, they estimate that 55% of neocortex (by volume), is visual, as compared to 12% somatosensory, 8% motor, 3% auditory, and 0.5% gustatory. Given such statistics, it would seem probable that the visual system might have a better chance of setting-up such feedback activity following the presentation of a visual stimulus than would the auditory or tactile systems following the simultaneous presentation of either an auditory or tactile stimulus. Note that this account suggests that visual dominance is natural, at least for humans, in that it may have a hardwired physiological basis (this idea was originally captured by Colavita et al.’s (1976) suggestion that visual stimuli might be “prepotent”). It is interesting to note in this context that the amount of cortex given over to the processing of auditory and tactile information processing is far more evenly matched than for the competition between audition and vision, hence perhaps explaining the lack of a clear pattern of dominance when stimuli are presented in these two modalities at the same time (see Hecht and Reiner 2009; Occelli et al. 2010).

It is also important to note here that progress in terms of explaining the Colavita effect at a neural level might also come from a more fine-grained study of the temporal dynamics of multisensory integration in various brain regions. In humans, the first wave of activity in primary auditory cortex in response to the presentation of suprathreshold stimuli is usually seen at a latency of about 10–15 ms (e.g., Liegeois-Chauvel et al. 1994; Howard et al. 2000; Godey et al. 2001; Brugge et al. 2003). Activity in primary visual cortex starts about 40–50 ms after stimulus presentation (e.g., Foxe et al. 2008; see also Schroeder et al. 1998), whereas for primary somatosensory cortex the figure is about 8–12 ms (e.g., Inui et al., 2004; see also Schroeder et al. 2001). Meanwhile, Schroeder and Foxe (2002, 2004) have documented the asymmetrical time course of the interactions taking place between auditory and visual cortex. Their research has shown that the visual modulation of activity in auditory cortex occurs several tens of milliseconds after the feedforward sweep of activation associated with the processing of auditory stimuli, under conditions where auditory and visual stimuli happen to be presented simultaneously from a location within peripersonal space (i.e., within arm’s reach; see Rizzolatti et al. 1997). This delay is caused by the fact that much of the visual input to auditory cortex is routed through superior temporal polysensory areas (e.g., Foxe and Schroeder 2002; see also Ghazanfar et al. 2005; Kayser et al. 2008; Smiley et al. 2007), and possibly also through prefrontal cortex. It therefore seems plausible to suggest that such delayed visual (inhibitory) input to auditory cortex might play some role in disrupting the setting-up of the feedback activity from higher (auditory) areas. * That said, Falchier et al. (2010) recently reported evidence suggesting the existence of a more direct routing of information from visual to auditory cortex (i.e., from V2 to caudal auditory cortex), hence potentially confusing the story somewhat.

By contrast, audition’s influence on visual information processing occurs more rapidly, and involves direct projections from early auditory cortical areas to early visual areas. That is, direct projections have now been documented from the primary auditory cortex A1 to the primary visual cortex V1 (e.g., see Wang et al. 2008; note, however, that these direct connections tend to target peripheral, rather than central, locations in the visual field; that said, other projections may well be more foveally targeted). Interestingly, however, until very recently no direct connections had as yet been observed in the opposite direction (see Falchier et al. 2010). These direct projections from auditory to visual cortex may help to account for the increased visual cortical excitability seen when an auditory stimulus is presented together with a visual stimulus (e.g., Martuzzi et al. 2007; Noesselt et al. 2007; Rockland and Ojima 2003; Romei et al. 2007, 2009; see also Besle et al. 2009; Clavagnier et al. 2004; Falchier et al. 2003). Indeed, Bolognini et al. (2010) have recently shown that transcranic magnetic stimulation (TMS)-elicited phosphenes (presented near threshold) are more visible when a white noise burst is presented approximately 40 ms before the TMS pulse (see also Romei et al. 2009).

It is also interesting to note here that when auditory and tactile stimuli are presented simultaneously from a distance of less than 1 m (i.e., in peripersonal space), the response in multisensory convergence regions of auditory association cortex is both rapid and approximately simultaneous for these two input modalities (see Schroeder and Foxe 2002, p. 193; see also Foxe et al. 2000, 2002; Murray et al. 2005; Schroeder et al. 2001). Such neurophysiological timing properties may then also help to explain why no clear Colavita dominance effect has as yet been reported between these two modalities (see also Sperdin et al. 2009). * That said, any neurally inspired account of the Colavita effect will likely also have to incorporate the recent discovery of feedforward multisensory interactions to early cortical areas taking place in the thalamus (i.e., via the thalamocortical loop; Cappe et al. 2009).

Although any attempt to link human behavior to single-cell neurophysiological data in either awake and anesthetized primates is clearly speculative at this stage, we are nevertheless convinced that this kind of interdisciplinary approach will be needed if we are to develop a fuller understanding of the Colavita effect in the coming years. It may also prove fruitful, when trying to explain why it is that participants fail to make an auditory (or tactile) response once they have made a visual one to consider the neuroscience research on the duration (and decay) of sensory memory in the different modalities (e.g., Lu et al. 1992; Harris et al. 2002; Uusitalo et al. 1996; Zylberberg et al. 2009). Here, it would be particularly interesting to know whether there are any systematic modality-specific differences in the decay rate of visual, auditory, and tactile sensory memory.

27.6.2. Clinical Extinction and Colavita Visual Dominance Effect

It will most likely also be revealing in future research to explore the relationship between the Colavita visual dominance effect and the clinical phenomenon of extinction that is sometimes seen in clinical patients following lateralized (typically right parietal) brain damage (e.g., Baylis et al. 1993; Bender 1952; Brozzoli et al. 2006; Driver and Vuilleumier 2001; Farnè et al. 2007; Rapp and Hendel 2003; Ricci and Chatterjee 2004). The two phenomena share a number of similarities: Both are sensitive to the relative spatial position from which the stimuli are presented (Costantini et al. 2007; Hartcher-O’Brien et al. 2008, 2010; Koppen and Spence 2007c); both are influenced by the relative timing of the two stimuli (Baylis et al. 2002; Costantini et al. 2007; Koppen and Spence 2007b; Lucey and Spence 2009; Rorden et al. 1997); both affect perceptual sensitivity as well as being influenced by response-related factors (Koppen et al. 2009; Ricci and Chatterjee 2004; Sinnett et al. 2008; see also Gorea and Sagi 2002). The proportion of experimental trials on which each phenomenon occurs in the laboratory has also been shown to vary greatly between studies.

In terms of the biased (or integrated) competition hypothesis (Desimone and Duncan 1995; Duncan 1996), extinction (in patients) is thought to reflect biased competition against stimuli from one side (Driver and Vuilleumier 2001; Rapp and Hendel 2003), whereas here we have argued that the Colavita effect reflects biased competition that favors the processing of visual stimuli. Although extinction has typically been characterized as a spatial phenomenon (i.e., it is the contralesional stimulus that normally extinguishes a simultaneously presented ipsilesional stimulus), it is worth noting that nonspatial extinction effects have also been reported (Costantini et al. 2007; Humphreys et al. 1995; see also Battelli et al. 2007). Future neuroimaging research will hopefully help to determine the extent to which the neural substrates underlying the Colavita visual dominance effect in healthy individuals and the phenomenon of extinction in clinical patients are similar (Sarri et al. 2006). Intriguing data here come from a neuroimaging study of a single patient with visual–tactile extinction reported by Sarri et al. In this patient, awareness of touch on the bimodal visuotactile trials was associated with increased activity in right parietal and frontal regions. Sarri et al. argued that the cross-modal extinction of the tactile stimulus in this patient resulted from increased competition arising from the functional coupling of visual and somatosensory cortex with multisensory parietal cortex.

The literature on unimodal and cross-modal extinction suggests that the normal process of biased competition can be interrupted by the kinds of parietal damage that lead to neglect and/or extinction. It would therefore be fascinating to see whether one could elicit the same kinds of biases in neural competition (usually seen in extinction patients) in normal participants, simply by administering TMS over posterior parietal areas (see Driver and Vuilleumier 2001; Duncan 1996; Sarri et al. 2006). Furthermore, following on from the single-cell neurophysiological work conducted by Schroeder and his colleagues (e.g., see Schroeder and Foxe 2002, 2004; Schroeder et al. 2004), it might also be interesting to target superior temporal polysensory areas, and/or the prefrontal cortex in order to try and disrupt the modality-based biased competition seen in the Colavita effect (i.e., rather than the spatial or temporal competition that is more typically reported in extinction patients; see Battelli et al. 2007). There are two principle outcomes that could emerge from such a study, and both seem plausible: (1) TMS over one or more such cortical sites might serve to magnify the Colavita visual dominance effect observed in normal participants, based on the consequences of pathological damage to these areas observed in extinction patients; (2) TMS over these cortical sites might also reduce the magnitude of the Colavita effect, by interfering with the normal processes of biased competition, and/or by interfering with the late-arriving cross-modal feedback activity from visual to auditory cortex (see Section 27.6.1). It would, of course, also be very interesting in future research to investigate whether extinction patients exhibit a larger Colavita effect than normal participants in the traditional version of the Colavita task (cf. Costantini et al. 2007).

27.7. CONCLUSIONS AND QUESTIONS FOR FUTURE RESEARCH

Research conducted over the past 35 years or so has shown the Colavita visual dominance effect to be a robust empirical phenomenon. However, traditional explanations of the effect simply cannot account for the range of experimental data that is currently available. In this article, we argue that the Colavita visual dominance effect may be accounted for in terms of Desimone and Duncan’s (1995; see also Duncan 1996) model of biased (or integrated) competition. According to the explanation outlined here, the Colavita visual dominance effect can be understood in terms of the cross-modal competition between the neural representations of simultaneously presented visual and auditory (or tactile) stimuli. Cognitive neuroscience studies would certainly help to further our understanding of the mechanisms underlying the Colavita effect. It would be particularly interesting, for example, to compare the pattern of brain activation on those trials in which participants fail to respond correctly to the nonvisual stimulus to the activation seen on those trials in which they respond appropriately (cf. Fink et al. 2000; Golob et al. 2001; Sarri et al. 2006; Schubert et al. 2006). Event-related potential studies could also help to determine just how early (or late, see Falkenstein et al. 1991; Quinlan 2000; Zahn et al. 1994) the processing of ignored and reported auditory (or tactile) stimuli differs (see Hohnsbein et al. 1991).

27.7.1. Modeling the Colavita Visual Dominance Effect

There is also a considerable amount of interesting work to be done in terms of modeling the Colavita visual dominance effect. Cooper (1998) made a start on this more than a decade ago. He developed a computational model that was capable of simulating the pattern of participants’ RTs in the Colavita task. Cooper’s model consisted of separate modality-specific input channels feeding into a single “object representation network” (whose function involved activating specific response schemas—presumably equivalent to a target stimuli reaching the criterion for responding, as discussed earlier) in which the speed of each channel was dependent on the strength (i.e., weight) of the channel itself. By assuming that the visual channel was stronger than the auditory channel, the model was able to successfully account for the fact that although responses to auditory stimuli are faster than responses to visual stimuli in unimodal trials, the reverse pattern is typically found on bimodal target trials.

The challenge for researchers in this area will be to try and develop models that are also capable of accounting for participants’ failure to respond to the nonvisual stimulus (i.e., the effect that has constituted the focus for the research discussed in this article; cf. Peers et al. 2005); such models might presumably include the assignment of different weights to visual and auditory cues, biases to preferentially respond to either visual or auditory stimuli, different gain/loss functions associated with responding, or failing to respond, to auditory and visual target stimuli, etc. It will be especially interesting here to examine whether the recent models of Bayesian multisensory integration (see Ernst 2005) that have proved so successful in accounting for many aspects of cross-modal perception, sensory dominance, and multisensory information processing, can also be used to account for the Colavita visual dominance effect.

27.7.2. Multisensory Facilitation versus Interference

Finally, in closing, it is perhaps worth pausing to consider the Colavita effect in the context of so many other recent studies that have demonstrated the benefits of multisensory over unisensory stimulus presentation (e.g., in terms of speeding simple speeded detection responses; Nickerson 1973; Sinnett et al. 2008, Experiment 1; see also Calvert et al. 2004). To some, the existence of the Colavita effect constitutes a puzzling example of a situation in which multisensory stimulation appears to impair (rather than to facilitate) human performance. It is interesting to note here though that whether one observes benefits or costs after multisensory (as compared to unisensory) stimulation seems to depend largely on the specific requirements of the task faced by participants. For example, Sinnett et al. (2008; Experiment 2) reported the facilitation of simple speeded detection latencies on bimodal audiovisual trials (i.e., they observed a violation of the race model; Miller 1982, 1991) when their participants had to make the same simple speeded detection responses to auditory, visual, and audiovisual targets. By contrast, they observed an inhibitory effect when their participants had to respond to the targets in each modality by pressing a separate response key (i.e., the typical Colavita paradigm). However, this latter result is not really so surprising if one stops to consider the fact that in the Colavita task participants can really be thought of as performing two tasks at once: that is, in the traditional two-response version of the Colavita task, the participants perform both a speeded auditory target detection task as well as a speeded visual target detection task. Although on the majority of (unimodal) trials the participants only have to perform one task, on a minority of (bimodal) trials they have to perform both tasks at the same time (and it is on these trials that the Colavita effect occurs when the nonvisual stimulus is seemingly ignored). * By contrast, in the redundant target effect paradigm (see earlier), both stimuli are relevant to the same task (i.e., to making a simple speeded target detection response).

Researchers have known for more than half a century that people find it difficult to perform two tasks at the same time (regardless of whether the target stimuli relevant to performing those tasks are presented in the same versus different sensory modalities (e.g., Pashler 1994; Spence 2008). One can therefore think of the Colavita paradigm in terms of a form of dual-task interference (resulting from modality-based biased competition at the response-selection level)—interference that appears to be intimately linked to the making of speeded responses to the target stimuli (however, see Koppen et al. 2009). More generally, it is important to stress that although multisensory integration may, under the appropriate conditions, give rise to improved perception/performance, the benefits may necessarily come at the cost of some loss of access to the component unimodal signals (cf. Soto-Faraco and Alsius 2007, 2009). In closing, it is perhaps worth highlighting the fact that the task-dependent nature of the consequences of multisensory integration that show up in studies related to the Colavita effect have now also been demonstrated in a number of different behavioral paradigms, in both humans (see Cappe et al. in press; Gondan and Fischer 2009; Sinnett et al. 2008; Spence et al. 2003) and monkeys (see Besle et al. 2009; Wang et al. 2008).

REFERENCES

  1. Battelli L, Pascual-Leone A, Cavanagh P. The ‘when’ pathway of the right parietal lobe. Trends in Cognitive Sciences. 2007;11:204–210. [PMC free article: PMC3613278] [PubMed: 17379569]
  2. Baylis G.C, Driver J, Rafal R. D. Visual extinction and stimulus repetition. Journal of Cognitive Neuroscience. 1993;5:453–466. [PubMed: 23964918]
  3. Baylis G.C, Simon S. L, Baylis L. L, Rorden C. Visual extinction with double simultaneous stimulation: What is simultaneous? Neuropsychologia. 2002;40:1027–1034. [PubMed: 11900754]
  4. Bender M.B. Disorders in perception. Springfield, IL: Charles Thomas; 1952.
  5. Besle J, Bertrand O, Giard M. H. Electrophysiological (EEG, sEEG, MEG) evidence for multiple audiovisual interactions in the human auditory cortex. Hearing Research. 2009;258(1-2):143–151. [PubMed: 19573583]
  6. Bolognini N, Senna I, Maravita A, Pasqual-Leone A, Merabeth L. B. Auditory enhancement of visual phosphene perception: The effect of temporal and spatial factors and of stimulus intensity. Neuroscience Letters. 2010;477:109–114. [PMC free article: PMC3538364] [PubMed: 20430065]
  7. Bonneh Y.S, Belmonte M. K, Pei F, Iversen P. E, Kenet T, Akshoomoff N, Adini Y, Simon H. J, Moore C. I, Houde J. F, Merzenich M. M. Cross-modal extinction in a boy with severely autistic behavior and high verbal intelligence. Cognitive Neuropsychology. 2008;25:635–652. [PubMed: 18651259]
  8. Bridgeman B. The physiological basis of the act of perceiving. In: Relationships between perception and action: Current approaches. Berlin: Springer; 1990. pp. 21–42.
  9. Brozzoli C, Demattè M. L, Pavani F, Frassinetti F, Farnè A. Neglect and extinction: Within and between sensory modalities. Restorative Neurology and Neuroscience. 2006;24:217–232. [PubMed: 17119300]
  10. Brugge J.F, Volkov I. O, Garell P. C, Reale R. A, Howard M. A. Functional connections between auditory cortex on Heschl’s gyrus and on the lateral superior temporal gyrus in humans. Journal of Neurophysiology. (third) 2003;90:3750–3763. [PubMed: 12968011]
  11. Calvert G.A, Spence C, Stein B. E, editors. The handbook of multisensory processes. Cambridge, MA: MIT Press; 2004.
  12. Cappe C, Barone P. Heteromodal connections supporting multisensory integration at low levels of cortical processing in the monkey. European Journal of Neuroscience. 2005;22:2886–2902. [PubMed: 16324124]
  13. Cappe C, Morel A, Barone P, Rouiller E. M. The thalamocortical projection systems in primates: An anatomical support for multisensory and sensorimotor interplay. Cerebral Cortex. 2009;19:2025–2037. [PMC free article: PMC2722423] [PubMed: 19150924]
  14. Clavagnier S, Falchier A, Kennedy H. Long-distance feedback projections to area V1: Implications for multisensory integration, spatial awareness, and visual consciousness. Cognitive, Affective, and Behavioural Neuroscience. 2004;4:117–126. [PubMed: 15460918]
  15. Colavita F.B. Human sensory dominance. Perception and Psychophysics. 1974;16:409–412.
  16. Colavita F.B. Visual dominance and attention in space. Bulletin of the Psychonomic Society. 1982;19:261–262.
  17. Colavita F.B, Tomko R, Weisberg D. Visual prepotency and eye orientation. Bulletin of the Psychonomic Society. 1976;8:25–26.
  18. Colavita F.B, Weisberg D. A further investigation of visual dominance. Perception and Psychophysics. 1979;25:345–347. [PubMed: 461094]
  19. Cooper R. Visual dominance and the control of action. In: Gernsbacher M. A, Derry S. J, editors. Proceedings of the 20th Annual Conference of the Cognitive Science Society. Mahwah, NJ: Lawrence Erlbaum Associates; 1998. pp. 250–255.
  20. Costantini M, Bueti D, Pazzaglia M, Aglioti S. M. Temporal dynamics of visuo-tactile extinction within and between hemispaces. Neuropsychology. 2007;21:242–250. [PubMed: 17402824]
  21. Cowey A, Stoerig P. The neurobiology of blindsight. Trends in the Neurosciences. 1991;14:140–145. [PubMed: 1710851]
  22. Critchley M. Tactile thought, with special reference to the blind. Brain. 1953;76:19–35. [PubMed: 13041920]
  23. Crowder R.G. Repetition effects in immediate memory when there are no repeated elements in the stimuli. Journal of Experimental Psychology. 1968;78:605–609.
  24. De Reuck T, Spence C. 2009. Attention and visual dominance. Unpublished manuscript.
  25. Desimone R, Duncan J. Neural mechanisms of selective visual attention. Annual Review of Neuroscience. 1995;18:193–222. [PubMed: 7605061]
  26. Driver J, Vuilleumier P. Perceptual awareness and its loss in unilateral neglect and extinction. Cognition. 2001;79:39–88. [PubMed: 11164023]
  27. Duncan J. Cooperating brain systems in selective perception and action. In: Inui T, McClelland J. L, editors. Attention and performance XVI: Information integration in perception and communication. Cambridge, MA: MIT Press; 1996. pp. 549–578.
  28. Egeth H.E, Sager L. C. On the locus of visual dominance. Perception and Psychophysics. 1977;22:77–86.
  29. Elcock S, Spence C. 2009. Caffeine and the Colavita visual dominance effect. Unpublished manuscript.
  30. Ernst M. A Bayesian view on multimodal cue integration. In: Knoblich G, Thornton I, Grosejan M, Shiffrar M, editors. Perception of the human body from the inside out. New York: Oxford Univ. Press; 2005. pp. 105–131.
  31. Exner S. Experimentelle Untersuchung der einfachsten psychischen Processe (Experimental study of the most simple psychological processes). Archiv für die gesammte Physiologie des menschens und der Thiere (Pflüger’s Archive). 1875;11:403–432.
  32. Falchier A, Clavagnier S, Barone P, Kennedy H. Anatomical evidence of multimodal integration in primate striate cortex. Journal of Neuroscience. 2003;22:5749–5759. [PubMed: 12097528]
  33. Falchier A, Schroeder C. E, Hackett T. A, Lakatos P, Nascimento-Silva S, Ulbert I, Karmos G, Smiley J. F. Projection from visual areas V2 and prostriata to caudal auditory cortex in the monkey. Cerebral Cortex. 2010;20:1529–1538. [PMC free article: PMC2882821] [PubMed: 19875677]
  34. Falkenstein M, Hohnsbein J, Hoormann J, Blanke L. Effects of crossmodal divided attention on late ERP components: II. Error processing in choice reaction tasks. Electroencephalography and Clinical Neurophysiology. 1991;78:447–455. [PubMed: 1712280]
  35. Farnè A, Brozzoli C, Làdavas E, Ro T. Investigating multisensory spatial cognition through the phenomenon of extinction. In: Haggard P, Rossetti Y, Kawato M, editors. Attention and performance XXII: Sensorimotor foundations of higher cognition. Oxford: Oxford Univ. Press; 2007. pp. 183–206.
  36. Felleman D.J, Van Essen D. C. Distributed hierarchical processing in primate cerebral cortex. Cerebral Cortex. 1991;1:1–47. [PubMed: 1822724]
  37. Fink G.R, Driver J, Rorden C, Baldeweg T, Dolan R. J. Neural consequences of competing stimuli in both visual hemifields: A physiological basis for visual extinction. Annals of Neurology. 2000;47:440–446. [PubMed: 10762154]
  38. Foree D.D, LoLordo V. M. J. Attention in the pigeon: Differential effects of food-getting versus shock-avoidance procedures. Journal of Comparative and Physiological Psychology. 1973;85:551–558. [PubMed: 4762141]
  39. Foxe J.J, Morocz I. A, Murray M. M, Higgins B. A, Javitt D. C, Schroeder C. E. Multisensory auditory–somatosensory interactions in early cortical processing revealed by high-density electrical mapping. Cognitive Brain Research. 2000;10:77–83. [PubMed: 10978694]
  40. Foxe J.J, Strugstad E. C, Sehatpour P, Molholm S, Pasieka W, Schroeder C. E, McCourt M. E. Parvocellular and magnocellular contributions to the initial generators of the visual evoked potential: High-density electrical mapping of the “C1” component. Brain Topography. 2008;21:11–21. [PubMed: 18784997]
  41. Foxe J.J, Wylie G. R, Martinez A, Schroeder C. E, Javitt D. C, Guilfoyle D, Ritter W, Murray M. M. Auditory–somatosensory multisensory processing in auditory association cortex: An fMRI study. Journal of Neurophysiology. 2002;88:540–543. [PubMed: 12091578]
  42. Fu K.-M. G, Johnston T. A, Shah A. S, Arnold L, Smiley J, Hackett T. A, Garraghty P. E, Schroeder C. E. Auditory cortical neurons respond to somatosensory stimulation. Journal of Neuroscience. 2003;23:7510–7515. [PMC free article: PMC6740753] [PubMed: 12930789]
  43. Gallace A, Tan H. Z, Spence C. Multisensory numerosity judgments for visual and tactile stimuli. Perception and Psychophysics. 2007;69:487–501. [PubMed: 17727102]
  44. Ghazanfar A.A, Maier J. X, Hoffman K. L, Logothetis N. K. Multisensory integration of dynamic faces and voices in Rhesus monkey auditory cortex. Journal of Neuroscience. 2005;25:5004–5012. [PMC free article: PMC6724848] [PubMed: 15901781]
  45. Godey B, Schwartz D, de Graaf J. B, Chauvel P, Liegeois-Chauvel C. Neuromagnetic source localization of auditory evoked fields and intracerebral evoked potentials: A comparison of data in the same patients. Clinical Neurophysiology. 2001;112:1850–1859. [PubMed: 11595143]
  46. Golob E.J, Miranda G. G, Johnson J. K, Starr A. Sensory cortical interactions in aging, mild cognitive impairment, and Alzheimer’s disease. Neurobiology of Aging. 2001;22:755–763. [PubMed: 11705635]
  47. Gondan M, Fischer V. Serial, parallel, and coactive processing of double stimuli presented with onset asynchrony. Perception. 2009;38 Suppl:16.
  48. Gorea A, Sagi D. Failure to handle more than one internal representation in visual detection tasks. Proceedings of the National Academy of Sciences of the United States of America. 2000;97:12380–12384. [PMC free article: PMC17350] [PubMed: 11050253]
  49. Gorea A, Sagi D. Natural extinction: A criterion shift phenomenon. Visual Cognition. 2002;9:913–936.
  50. Green D.M, Swets J. A. Signal detection theory and psychophysics. New York: Wiley; 1966.
  51. Gregg L.W, Brogden W. J. The effect of simultaneous visual stimulation on absolute auditory sensitivity. Journal of Experimental Psychology. 1952;43:179–186. [PubMed: 14927820]
  52. Hackett T.A, De La Mothe L, Ulbert I, Karmos G, Smiley J, Schroeder C. E. Multisensory convergence in auditory cortex: II. Thalamocortical connections of the caudal superior temporal plane. Journal of Comparative Neurology. 2007;502:924–952. [PubMed: 17444488]
  53. Hahnloser R, Douglas R. J, Mahowald M, Hepp K. Feedback interactions between neuronal pointers and maps for attentional processing. Nature Neuroscience. 1999;2:746–752. [PubMed: 10412065]
  54. Harris J.A, Miniussi C, Harris I. M, Diamond M. E. Transient storage of a tactile memory trace in primary somatosensory cortex. Journal of Neuroscience. 2002;22:8720–8725. [PubMed: 12351747]
  55. Hartcher-O’Brien J, Gallace A, Krings B, Koppen C, Spence C. When vision ‘extinguishes’ touch in neurologically-normal people: Extending the Colavita visual dominance effect. Experimental Brain Research. 2008;186:643–658. [PubMed: 18301885]
  56. Hartcher-O’Brien J, Levitan C, Spence C. Out-of-touch: Does vision dominate over touch when it occurs off the body? Brain Research. 2010;1362:48–55. [PubMed: 20850418]
  57. Hecht D, Reiner M. Sensory dominance in combinations of audio, visual and haptic stimuli. Experimental Brain Research. 2009;193:307–314. [PubMed: 18985327]
  58. Hefferline R.F, Perera T. B. Proprioceptive discrimination of a covert operant without its observation by the subject. Science. 1963;139:834–835. [PubMed: 13963703]
  59. Hirsh I.J, Sherrick C. E Jr. Perceived order in different sense modalities. Journal of Experimental Psychology. 1961;62:423–432. [PubMed: 13907740]
  60. Hohnsbein J, Falkenstein M. Visual dominance: Asymmetries in the involuntary processing of visual and auditory distractors. In: Blum B, editor. Channels in the visual nervous system: Neurophysiology, psychophysics and models. London: Freund Publishing House; 1991. pp. 301–313.
  61. Hohnsbein J, Falkenstein M, Hoormann J. Visual dominance is reflected in reaction times and event-related potentials (ERPs). In: Blum B, editor. Channels in the visual nervous system: Neurophysiology, psychophysics and models. London: Freund Publishing House; 1991. pp. 315–333.
  62. Howard M.A, Volkov I. O, Mirsky R, Garell P. C, Noh M. D, Granner M, Damasio H, Steinschneider M, Reale R. A, Hind J. E, Brugge J. F. Auditory cortex on the human posterior superior temporal gyrus. Journal of Comparative Neurology. 2000;416:79–92. [PubMed: 10578103]
  63. Humphreys G.W, Romani C, Olson A, Riddoch M. J, Duncan J. Nonspatial extinction following lesions of the parietal lobe in man. Nature. 1995;372:357–359. [PubMed: 7969493]
  64. Inui K, Wang X, Tamura Y, Kaneoke Y, Kakigi R. Serial processing in the human somatosensory system. Cerebral Cortex. 2004;14:851–857. [PubMed: 15054058]
  65. Jaśkowski P. Simple reaction time and perception of temporal order: Dissociations and hypotheses. Perceptual and Motor Skills. 1996;82:707–730. [PubMed: 8774006]
  66. Jaśkowski P. Reaction time and temporal-order judgment as measures of perceptual latency: The problem of dissociations. In: Aschersleben G, Bachmann T, Műsseler J, editors. Cognitive contributions to the perception of spatial and temporal events. North-Holland: Elsevier Science; 1999. pp. 265–282.
  67. Jaśkowski P, Jaroszyk F, Hojan-Jesierska D. Temporal-order judgments and reaction time for stimuli of different modalities. Psychological Research. 1990;52:35–38. [PubMed: 2377723]
  68. Johnson T.L, Shapiro K. L. Attention to auditory and peripheral visual stimuli: Effects of arousal and predictability. Acta Psychologica. 1989;72:233–245. [PubMed: 2618791]
  69. Kayser C, Petkov C. I, Logothetis N. K. Visual modulation of neurons in auditory cortex. Cerebral Cortex. 2008;18:1560–1574. [PubMed: 18180245]
  70. Koppen C, Alsius A, Spence C. Semantic congruency and the Colavita visual dominance effect. Experimental Brain Research. 2008;184:533–546. [PubMed: 17885751]
  71. Koppen C, Levitan C, Spence C. A signal detection study of the Colavita effect. Experimental Brain Research. 2009;196:353–360. [PubMed: 19488743]
  72. Koppen C, Spence C. Seeing the light: Exploring the Colavita visual dominance effect. Experimental Brain Research. 2007a;180:737–754. [PubMed: 17333012]
  73. Koppen C, Spence C. Audiovisual asynchrony modulates the Colavita visual dominance effect. Brain Research. 2007b;1186:224–232. [PubMed: 18005944]
  74. Koppen C, Spence C. Spatial coincidence modulates the Colavita visual dominance effect. Neuroscience Letters. 2007c;417:107–111. [PubMed: 17408855]
  75. Koppen C, Spence C. Assessing the role of stimulus probability on the Colavita visual dominance effect. Neuroscience Letters. 2007d;418:266–271. [PubMed: 17398003]
  76. Kristofferson A.B. Attention in time discrimination and reaction time. Washington D.C: Office of Technical Services, U.S. Department of Commerce; 1965. NASA Contractors Report 194.
  77. Lamme V. A. F. Blindsight: The role of feedforward and feedback corticocortical connections. Acta Psychologica. 2001;107:209–228. [PubMed: 11388136]
  78. Lamme V.A. F, Supèr H, Landman R, Roelfsema P. R, Spekreijse H. The role of primary visual cortex (V1) in visual awareness. Vision Research. 2000;40:1507–1521. [PubMed: 10788655]
  79. Liegeois-Chauvel C, Musolino A, Badier J. M, Marquis P, Chauvel P. Evoked potentials recorded from the auditory cortex in man: Evaluation and topography of the middle latency components. Electroencephalography and Clinical Neuroscience. 1994;92:204–214. [PubMed: 7514990]
  80. LoLordo V. M. Selective associations. In: Dickinson A, Boakes R. A, editors. Mechanisms of learning and motivation: A memorial to Jerzy Konorsky. Hillsdale, NJ: Erlbaum; 1979. pp. 367–399.
  81. LoLordo V. M, Furrow D. R. Control by the auditory or the visual element of a compound discriminative stimulus: Effects of feedback. Journal of the Experimental Analysis of Behavior. 1976;25:251–256. [PMC free article: PMC1333457] [PubMed: 932617]
  82. Lu Z.-L, Williamson S. J, Kaufman L. Behavioral lifetime of human auditory sensory memory predicted by physiological measures. Science. 1992;258:1669–1670. [PubMed: 1455246]
  83. Lucey T, Spence C. 2009. Visual dominance. Unpublished manuscript.
  84. Macaluso E, Driver J. Multisensory spatial interactions: A window onto functional integration in the human brain. Trends in Neurosciences. 2005;28:264–271. [PubMed: 15866201]
  85. Macknik S.L. The role of feedback in visual attention and awareness. Perception. 2009;38 Suppl:162.
  86. Macknik S, Martinez-Conde S. The role of feedback in visual masking and visual processing. Advances in Cognitive Psychology. 2007;3:125–152. [PMC free article: PMC2864985] [PubMed: 20517504]
  87. Macknik S, Martinez-Conde S. The role of feedback in visual attention and awareness. In: Gazzaniga M. S. A, editor. The new cognitive neurosciences. Cambridge, MA: MIT Press; pp. 1163–1177.
  88. Manly T, Robertson I. H, Galloway M, Hawkins K. The absent mind: Further investigations of sustained attention to response. Neuropsychologia. 1999;37:661–670. [PubMed: 10390027]
  89. Marks L.E, Ben-Artzi E, Lakatos S. Cross-modal interactions in auditory and visual discrimination. International Journal of Psychophysiology. 2003;50:125–145. [PubMed: 14511841]
  90. Martuzzi R, Murray M. M, Michel C. M, Thiran J. P, Maeder P. P, Clarke S, Meuli R. A. Multisensory interactions within human primary cortices revealed by BOLD dynamics. Cerebral Cortex. 2007;17:1672–1679. [PubMed: 16968869]
  91. McDonald J. J, Teder-Sälejärvi W. A, Di Russo F, Hillyard S. A. Neural basis of auditory-induced shifts in visual time-order perception. Nature Neuroscience. 2005;8:1197–1202. [PubMed: 16056224]
  92. Meltzer D, Masaki M. A. Measures of stimulus control and stimulus dominance. Bulletin of the Psychonomic Society. 1973;1:28–30.
  93. Miller J.O. Divided attention: Evidence for coactivation with redundant signals. Cognitive Psychology. 1982;14:247–279. [PubMed: 7083803]
  94. Miller J.O. Time course of coactivation in bimodal divided attention. Perception and Psychophysics. 1986;40:331–343. [PubMed: 3786102]
  95. Miller J.O. Channel interaction and the redundant targets effect in bimodal divided attention. Journal of Experimental Psychology: Human Perception and Performance. 1991;17:160–169. [PubMed: 1826309]
  96. Miller J, Ulrich R, Rolke B. On the optimality of serial and parallel processing in the psychological refractory period paradigm: Effects of the distribution of stimulus onset asynchronies. Cognitive Psychology. 2009;58:273–310. [PubMed: 19281972]
  97. Miyake S, Taniguchi S, Tsuji K. Effects of light stimulus upon simple reaction time and EP latency to the click presented with different SOA. Japanese Psychological Research. 1986;28:1–10.
  98. Murray M.M, Molholm S, Michel C. M, Heslenfeld D. J, Ritter W, Javitt D. C, Schroeder C. E, Foxe J. J. Grabbing your ear: Auditory–somatosensory multisensory interactions in early sensory cortices are not constrained by stimulus alignment. Cerebral Cortex. 2005;15:963–974. [PubMed: 15537674]
  99. Müsseler J, Hommel B. Blindness to response-compatible stimuli. Journal of Experimental Psychology: Human Perception and Performance. 1997a;23:861–872. [PubMed: 9180047]
  100. Müsseler J, Hommel B. Detecting and identifying response-compatible stimuli. Psychonomic Bulletin and Review. 1997b;4:125–129.
  101. Neumann O. Direct parameter specification and the concept of perception. Psychological Research. 1990;52:207–215. [PubMed: 2281129]
  102. Nickerson R. Intersensory facilitation of reaction time: Energy summation or preparation enhancement? Psychological Review. 1973;80:489–509. [PubMed: 4757060]
  103. Noesselt T, Rieger J. W, Schoenfeld M. A, Kanowski M, Hinrichs H, Heinze H.-J, Driver J. Audiovisual temporal correspondence modulates human multisensory temporal sulcus plus primary sensory cortices. Journal of Neuroscience. 2007;27:11431–11441. [PMC free article: PMC2957075] [PubMed: 17942738]
  104. Occelli V, Hartcher O’Brien C, Spence C, Zampini M. Assessing the audiotactile Colavita effect in near and rear space. Experimental Brain Research. 2010;203:517–532. [PubMed: 20431874]
  105. O’Connor N, Hermelin B. Sensory dominance in autistic children and subnormal controls. Perceptual and Motor Skills. 1963;16:920.
  106. Odgaard E.C, Arieh Y, Marks L. E. Cross-modal enhancement of perceived brightness: Sensory interaction versus response bias. Perception and Psychophysics. 2003;65:123–132. [PubMed: 12699315]
  107. Odgaard E.C, Arieh Y, Marks L. E. Brighter noise: Sensory enhancement of perceived loudness by concurrent visual stimulation. Cognitive, Affective, and Behavioral Neuroscience. 2004;4:127–132. [PubMed: 15460919]
  108. Oray S, Lu Z. L, Dawson M. E. Modification of sudden onset auditory ERP by involuntary attention to visual stimuli. International Journal of Psychophysiology. 2002;43:213–224. [PubMed: 11850087]
  109. Osborn W.C, Sheldon R. W, Baker R. A. Vigilance performance under conditions of redundant and nonredundant signal presentation. Journal of Applied Psychology. 1963;47:130–134.
  110. Partan S, Marler P. Communication goes multimodal. Science. 1999;283:1272–1273. [PubMed: 10084931]
  111. Pascual-Leone A, Walsh V. Fast backprojections from the motion to the primary visual area necessary for visual awareness. Science. 2001;292:510–512. [PubMed: 11313497]
  112. Pashler H. Dual-task interference in simple tasks: Data and theory. Psychological Bulletin. 1994;116:220–244. [PubMed: 7972591]
  113. Peers P.V, Ludwig C. J. H, Rorden C, Cusack R, Bonfiglioli C, Bundesen C, Driver J, Antoun N, Duncan J. Attentional functions of parietal and frontal cortex. Cerebral Cortex. 2005;15:1469–1484. [PubMed: 15689522]
  114. Posner M.I, Nissen M. J, Klein R. M. Visual dominance: An information-processing account of its origins and significance. Psychological Review. 1976;83:157–171. [PubMed: 769017]
  115. Quinlan P. The “late” locus of visual dominance. Abstracts of the Psychonomic Society. 2000;5:64.
  116. Randich A, Klein R. M, LoLordo V. M. Visual dominance in the pigeon. Journal of the Experimental Analysis of Behavior. 1978;30:129–137. [PMC free article: PMC1332708] [PubMed: 16812092]
  117. Rapp B, Hendel S. K. Principles of cross-modal competition: Evidence from deficits of attention. Psychonomic Bulletin and Review. 2003;10:210–219. [PubMed: 12747510]
  118. Ricci R, Chatterjee A. Sensory and response contributions to visual awareness in extinction. Experimental Brain Research. 2004;157:85–93. [PubMed: 14968280]
  119. Rizzolatti G, Berti A. Neglect as a neural representation deficit. Revue Neurologique (Paris). 1990;146:626–634. [PubMed: 2124720]
  120. Rizzolatti G, Fadiga L, Fogassi L, Gallese V. The space around us. Science. 1997;277:190–191. [PubMed: 9235632]
  121. Rockland K.S, Ojima H. Multisensory convergence in calcarine visual areas in macaque monkey. International Journal of Psychophysiology. 2003;50:19–26. [PubMed: 14511833]
  122. Rodway P. The modality shift effect and the effectiveness of warning signals in different modalities. Acta Psychologica. 2005;120:199–226. [PubMed: 15993828]
  123. Romei V, Murray M. M, Cappe C, Thut G. Preperceptual and stimulus-selective enhancement of low-level human visual cortex excitability by sounds. Current Biology. 2009;19:1799–1805. [PubMed: 19836243]
  124. Romei V, Murray M. M, Merabet L. B, Thut G. Occipital transcranial magnetic stimulation has opposing effects on visual and auditory stimulus detection: Implications for multisensory interactions. Journal of Neuroscience. 2007;27:11465–11472. [PMC free article: PMC6673223] [PubMed: 17959789]
  125. Rorden C, Mattingley J. B, Karnath H.-O, Driver J. Visual extinction and prior entry: Impaired perception of temporal order with intact motion perception after unilateral parietal damage. Neuropsychologia. 1997;35:421–433. [PubMed: 9106271]
  126. Rutschmann J, Link R. Perception of temporal order of stimuli differing in sense mode and simple reaction time. Perceptual and Motor Skills. 1964;18:345–352. [PubMed: 14166017]
  127. Sarri M, Blankenburg F, Driver J. Neural correlates of crossmodal visual–tactile extinction and of tactile awareness revealed by fMRI in a right-hemisphere stroke patient. Neuropsychologia. 2006;44:2398–2410. [PubMed: 16765998]
  128. Schroeder C.E, Foxe J. J. The timing and laminar profile of converging inputs to multisensory areas of the macaque neocortex. Brain Research: Cognitive Brain Research. 2002;14:187–198. [PubMed: 12063142]
  129. Schroeder C.E, Foxe J. J. Multisensory convergence in early cortical processing. In: Calvert G. A, Spence C, Stein B. E, editors. The handbook of multisensory processes. Cambridge, MA: MIT Press; 2004. pp. 295–309.
  130. Schroeder C.E, Lindsley R. W, Specht C, Marcovici A, Smiley J. F, Javitt D. C. Somatosensory input to auditory association cortex in the macaque monkey. Journal of Neurophysiology. 2001;85:1322–1327. [PubMed: 11248001]
  131. Schroeder C.E, Mehta A. D, Givre S. J. A spatiotemporal profile of visual system activation revealed by current source density analysis in the awake macaque. Cerebral Cortex. 1998;8:575–592. [PubMed: 9823479]
  132. Schroeder C.E, Molholm S, Lakatos P, Ritter W, Foxe J. J. Human simian correspondence in the early cortical processing of multisensory cues. Cognitive Processing. 2004;5:140–151.
  133. Schubert R, Blankenberg F, Lemm S, Villringer A, Curio G. Now you feel it, now you don’t: ERP correlates of somatosensory awareness. Psychophysiology. 2006;43:31–40. [PubMed: 16629683]
  134. Sereno M.I, Dale A. M, Reppas J. B, Kwong K. K, Belliveau J. W, Brady T. J, Rosen B. R, Tootell R. B. H. Borders of multiple visual areas in humans revealed by functional magnetic resonance imaging. Science. 1995;268:889–893. [PubMed: 7754376]
  135. Shapiro K.L, Egerman B, Klein R. M. Effects of arousal on human visual dominance. Perception and Psychophysics. 1984;35:547–552. [PubMed: 6483557]
  136. Shapiro K.L, Jacobs W. J, LoLordo V. M. Stimulus–reinforcer interactions in Pavlovian conditioning of pigeons: Implications for selective associations. Animal Learning and Behavior. 1980;8:586–594.
  137. Shapiro K.L, Johnson T. L. Effects of arousal on attention to central and peripheral visual stimuli. Acta Psychologica. 1987;66:157–172. [PubMed: 3687484]
  138. Sinnett S, Soto-Faraco S, Spence C. The co-occurrence of multisensory competition and facilitation. Acta Psychologica. 2008;128:153–161. [PubMed: 18207117]
  139. Sinnett S, Spence C, Soto-Faraco S. Visual dominance and attention: The Colavita effect revisited. Perception and Psychophysics. 2007;69:673–686. [PubMed: 17929691]
  140. Smiley J, Hackett T. A, Ulbert I, Karmos G, Lakatos P, Javitt D. C, Schroeder C. E. Multisensory convergence in auditory cortex: I. Cortical connections of the caudal superior temporal plane in Macaque monkey. Journal of Comparative Neurology. 2007;502:894–923. [PubMed: 17447261]
  141. Smith A. Effects of caffeine on human behavior. Food Chemistry and Toxicology. 2002;40:1243–1255. [PubMed: 12204388]
  142. Smith A.P, Kendrick A. M, Maben A. L. Effects of breakfast and caffeine on performance and mood in the late morning and after lunch. Neuropsychobiology. 1992;26:198–204. [PubMed: 1299795]
  143. Smith W.F. The relative quickness of visual and auditory perception. Journal of Experimental Psychology. 1933;16:239–257.
  144. Soto-Faraco S, Alsius A. Conscious access to the unisensory components of a cross-modal illusion. Neuroreport. 2007;18:347–350. [PubMed: 17435600]
  145. Soto-Faraco S, Alsius A. Deconstructing the McGurk–MacDonald illusion. Journal of Experimental Psychology: Human Perception and Performance. 2009;35:580–587. [PubMed: 19331510]
  146. Spence C. Cognitive neuroscience: Searching for the bottleneck in the brain. Current Biology. 2008;18:R965–R968. [PubMed: 18957255]
  147. Spence C. Prior entry: Attention and temporal perception. In: Nobre A. C, Coull J. T, editors. Attention and time. Oxford: Oxford Univ. Press; 2010. pp. 89–104.
  148. Spence C, Baddeley R, Zampini M, James R, Shore D. I. Crossmodal temporal order judgments: When two locations are better than one. Perception and Psychophysics. 2003;65:318–328. [PubMed: 12713247]
  149. Spence C, Nicholls M. E. R, Driver J. The cost of expecting events in the wrong sensory modality. Perception and Psychophysics. 2001a;63:330–336. [PubMed: 11281107]
  150. Spence C, Shore D. I, Klein R. M. Multisensory prior entry. Journal of Experimental Psychology: General. 2001b;130:799–832. [PubMed: 11757881]
  151. Spence C, Soto-Faraco S. Auditory perception: Interactions with vision. In: Plack C, editor. Auditory perception. Oxford: Oxford Univ. Press; 2009. pp. 271–296.
  152. Sperdin H.F, Cappe C, Foxe J. J, Murray M. M. Early, low-level auditory–somatosensory multisensory interactions impact reaction time speed. Frontiers in Integrative Neuroscience. 2009;3(2):1–10. [PMC free article: PMC2659167] [PubMed: 19404410]
  153. Stein B.E, London N, Wilkinson L. K, Price D. P. Enhancement of perceived visual intensity by auditory stimuli: A psychophysical analysis. Journal of Cognitive Neuroscience. 1996;8:497–506. [PubMed: 23961981]
  154. Stone J.V, Hunkin N. M, Porrill J, Wood R, Keeler V, Beanland M, Port M, Porter N. R. When is now? Perception of simultaneity. Proceedings of the Royal Society (B). 2001;268:31–38. [PMC free article: PMC1087597] [PubMed: 12123295]
  155. Tajadura-Jiménez A, Kitagawa N, Väljamäe A, Zampini M, Murray M. M, Spence C. Auditory–somatosensory multisensory interactions are spatially modulated by stimulated body surface and acoustic spectra. Neuropsychologia. 2009;47:195–203. [PubMed: 18760293]
  156. Taylor J.L, McCloskey D. I. Selection of motor responses on the basis of unperceived stimuli. Experimental Brain Research. 1996;110:62–66. [PubMed: 8817257]
  157. Thompson R.F, Voss J. F, Brogden W. J. Effect of brightness of simultaneous visual stimulation on absolute auditory sensitivity. Journal of Experimental Psychology. 1958;55:45–50. [PubMed: 13502524]
  158. Titchener E.B. Lectures on the elementary psychology of feeling and attention. New York: Macmillan; 1908.
  159. Turatto M, Benso F, Galfano G, Gamberini L, Umilta C. Non-spatial attentional shifts between audition and vision. Journal of Experimental Psychology: Human Perception and Performance. 2002;28:628–639. [PubMed: 12075893]
  160. Uetake K, Kudo Y. Visual dominance over hearing in feed acquisition procedure of cattle. Applied Animal Behaviour Science. 1994;42:1–9.
  161. Ulrich R, Miller J. Response grouping in the psychological refractory period (PRP) paradigm: Models and contamination effects. Cognitive Psychology. 2008;57:75–121. [PubMed: 18262510]
  162. Uusitalo M.A, Williamson S. J, Seppä M. T. Dynamical organisation of the human visual system revealed by lifetimes of activation traces. Neuroscience Letters. 1996;213:149–152. [PubMed: 8873136]
  163. Van Damme S, Crombez G, Spence C. Is the visual dominance effect modulated by the threat value of visual and auditory stimuli? Experimental Brain Research. 2009a;193:197–204. [PubMed: 18953530]
  164. Van Damme S, Gallace A, Spence C, Moseley G. L. Does the sight of physical threat induce a tactile processing bias? Modality-specific attentional facilitation induced by viewing threatening pictures. Brain Research. 2009b;1253:100–106. [PubMed: 19094970]
  165. Vibell J, Klinge C, Zampini M, Spence C, Nobre A. C. Temporal order is coded temporally in the brain: Early ERP latency shifts underlying prior entry in a crossmodal temporal order judgment task. Journal of Cognitive Neuroscience. 2007;19:109–120. [PubMed: 17214568]
  166. Wang Y, Celebrini S, Trotter Y, Barone P. Visuo-auditory interactions in the primary visual cortex of the behaving monkey: Electrophysiological evidence. BMC Neuroscience. 2008;9:79. [PMC free article: PMC2527609] [PubMed: 18699988]
  167. Wilcoxin H.C, Dragoin W. B, Kral P. A. Illness-induced aversions in rat and quail: Relative salience of visual and gustatory cues. Science. 1971;171:826–828. [PubMed: 5541167]
  168. Zahn T.P, Pickar D, Haier R. J. Effects of clozapine, fluphenazine, and placebo on reaction time measures of attention and sensory dominance in schizophrenia. Schizophrenia Research. 1994;13:133–144. [PubMed: 7986770]
  169. Zwyghuizen-Doorenbos A, Roehrs T. A, Lipschutz L, Timms V, Roth T. Effects of caffeine on alertness. Psychopharmacology. 1990;100:36–39. [PubMed: 2296626]
  170. Zylberberg A, Dehaene S, Mindlin G. B, Sigman M. Neurophysiological bases of exponential sensory decay and top-down memory retrieval: A model. Frontiers in Computational Neuroscience. 2009;3(4):1–16. [PMC free article: PMC2659975] [PubMed: 19325713]

Footnotes

*

That is, the visual target was only turned off once the participants made a visual response, and the auditory target was only turned off when the participants made an auditory response. This contrasts with Colavita’s (1974) studies, in which a participant’s first response turned off all the stimuli, and with other more recent studies in which the targets were only presented briefly (i.e., for 50 ms; e.g., Koppen and Spence 2007a, 2007b, 2007c, 2007d).

*

Note that researchers have also manipulated the relative probability of unimodal auditory and visual targets (see Egeth and Sager 1977; Quinlan 2000; Sinnett et al. 2007). However, since such probability manipulations have typically been introduced in the context of trying to shift the focus of a participant’s attention between the auditory and visual modalities, they will be discussed later (see Section 27.2.7).

*

Caffeine is a stimulant that accelerates physiological activity, and results in the release of adrenaline and the increased production of the neurotransmitter dopamine. Caffeine also interferes with the operation of another neurotransmitter: adenosine (Smith 2002; Zwyghuizen-Doorenbos et al. 1990).

*

Note that if practice were found to reduce the magnitude of the Colavita visual dominance effect, then this might provide an explanation for why increasing the probability of occurrence of bimodal target trials up to 90% in Koppen and Spence’s (2007d) study has been shown to eliminate the Colavita effect (see Section 27.2.5). Alternatively, however, increasing the prevalence (or number) of bimodal targets might also lead to the increased coupling of a participants’ responses on the bimodal trials (see main text for further details; Ulrich and Miller 2008).

*

Note the importance of using the same stimuli within the same pool of participants, given the large individual differences in the perception of audiovisual simultaneity that have been reported previously (Smith 1933; Spence 2010; Stone et al. 2001).

*

Note here that a very different result (i.e., the enhancement of perceived auditory intensity by a simultaneously-presented visual stimulus) has been reported in other studies in which the participants simply had to detect the presence of an auditory target (see Odgaard et al. 2004). This discrepancy highlights the fact that the precise nature of a participant’s task constitutes a critical determinant of the way in which the stimuli presented in different modalities interact to influence human information processing (cf. Gondan and Fisher 2009; Sinnett et al. 2008; Wang et al. 2008, on this point).

Note here that we are talking about the traditional two-response version of the Colavita task. Remember that in the three-response version, the participant’s first response terminates the trial, and hence there is no opportunity to make a second response.

*

One final point to note here concerns the fact that when participants made an erroneous response on the bimodal target trials, the erroneous auditory-only responses were somewhat slower than the erroneous vision-only responses, although this difference failed to reach statistical significance.

*

Note here also the fact that visual influences on primary and secondary auditory cortex are greatest when the visual stimulus leads the auditory stimulus by 20–80 ms (see Kayser et al. 2008), the same magnitude of visual leads that have also been shown to give rise to the largest Colavita effect (see Figure 2; Koppen and Spence 2007b).

*

It would be interesting here to determine whether the feedforward projections between primary auditory and tactile cortices are any more symmetrical than those between auditory and visual cortices (see Cappe and Barone 2005; Cappe et al. 2009; Hackett et al. 2007; Schroeder et al. 2001; Smiley et al. 2007, on this topic), since this could provide a neural explanation for why no Colavita effect has, as yet, been reported between the auditory and tactile modalities (Hecht and Reiner 2009; Occelli et al. 2010). That said, it should also be borne in mind that the nature of auditory-somatosensory interactions have recently been shown to differ quite dramatically as a function of the body surface stimulated (e.g., different audio–tactile interactions have been observed for stimuli presented close to the hands in frontal space vs. close to the back of the neck in rear space; see Fu et al. 2003; Tajadura-Jiminez et al. 2009; cf. Critchley 1953, p. 19). The same may, of course, also turn out to be true for the auditory–tactile Colavita effect.

*

One slight complication here though relates to the fact that people typically start to couple multiple responses to different stimuli into response couplets under the appropriate experimental conditions (see Ulrich and Miller 2008). Thus, one could argue about whether participants’ responses on the bimodal target trials actually counts as a third single (rather than dual) task, but one that, in the two-response version of the Colavita task involves a bi-finger, rather than a unifingered response. When considered in this light, the interference of performance seen in the Colavita task does not seem quite so surprising.

Copyright © 2012 by Taylor & Francis Group, LLC.
Bookshelf ID: NBK92851PMID: 22593876

Views

  • PubReader
  • Print View
  • Cite this Page

Other titles in this collection

Related information

  • PMC
    PubMed Central citations
  • PubMed
    Links to PubMed

Similar articles in PubMed

See reviews...See all...

Recent Activity

Your browsing activity is empty.

Activity recording is turned off.

Turn recording back on

See more...