Chapter 24Multisensory Perception and Bodily Self-Consciousness

24.3.1. Body Part Studies: Rubber Hand Illusion

Probably the most commonly used body illusion is the so-called “rubber hand illusion” (Botvinick and Cohen 1998), in which a subject watches a rubber hand on a table being stroked in synchrony with his corresponding (left or right) hidden hand. After a few seconds this simple manipulation causes the rubber hand (Ehrsson et al. 2004; Lloyd 2007) to “feel like my own hand,” that is, to be self-attributed. This does not happen when the stroking is applied asynchronously, suggesting that an intermodal correlation of different senses is crucial for self-attribution (Botvinick and Cohen 1998). The phenomenological experience of self-attribution is accompanied by a change in where subjects localize their real stroked hand (“proprioceptive drift”; Botvinick and Cohen 1998; Tsakiris and Haggard 2005; Kammers et al. 2009; Longo et al. 2008; Schütz-Bosbach et al. 2009). It has been argued that this latter finding demonstrates that the changes in bodily self-consciousness induced by the rubber hand illusion are due to changes in low-level, multisensory body representations. Recent studies of the illusion revealed a number of further behavioral changes related to the rubber hand such as increased cortical (Ehrsson et al. 2007), physiological (skin conductance response; Ehrsson 2007; Hägni et al. 2008), and fear responses to a threat to the rubber hand. Moreover, there are also rubber hand illusion–related changes in the real stimulated hand (e.g., body part–specific decrease in skin temperature; Moseley et al. 2008). However, the relation between these different measurements is still unclear (Schütz-Bosbach et al. 2009), and recent studies discuss the possibility of the existence of multiple (parallel and serial) body representations and dimensions of bodily self-consciousness (Longo et al. 2008; Kammers et al. 2009) that are differentially affected by the rubber hand illusion.

The rubber hand illusion has been explained as an effect of visual capture—the dominance of vision over other modalities in representations of the spatial location of events (Botvinick and Cohen 1998)—and has been related to properties of bimodal neurons in the parietal and premotor cortices (Ehrsson et al. 2004; Graziano et al. 2000; Iriki et al. 1996, 2001; Rizzolatti et al. 1981; Tsakiris et al. 2007). A recent article on the rubber hand illusion (Makin et al. 2008) proposed an explanatory model for the rubber hand illusion that implicates the role of multisensory integration within peri-hand space. The relative weighting (compared to that of proprioception) of visual information about hand position is greater when the hand is not moving; thus, in this situation visual information can bias proprioception. Furthermore, because vision can dominate over touch in the representation of spatial location, the brushstrokes that are seen to occur on the rubber hand may be processed as though they are occurring nearer to or on the real hand. Thus, the central representation of the location of the real hand may shift toward the rubber hand (Lloyd 2007). Given the temporal congruence of the seen and felt stroking, these inputs are integrated together as a coherent multisensory event in spatial coordinates that are shifted toward those of the rubber hand (Graziano et al. 2000). Makin and colleagues (2008) propose that this may result in the sensation of touch and ownership being referred to the rubber hand. It should be noted that these mechanisms and this direction of causality have yet to be verified experimentally. It is worth noting that the size of the drift is generally quite small (a few centimeters) compared to the actual distance between the fake and the real hand, and that the induced changes in illusory touch and (even more so) ownership during the rubber hand illusion are most often relatively weak changes in conscious bodily experience (even after 30 min of stroking).

Several studies have investigated the brain mechanisms involved in the rubber hand illusion, for example, using functional MRI (Ehrsson et al. 2004) and positron emission tomoraphy (Tsakiris et al. 2007). A systematic review of the studies using the rubber hand illusion would be beyond the scope of the present review, as this chapter focuses on scientific experimentation with full body illusions and global aspects of bodily self-consciousness. The interested reader is referred to the recent review on body part–specific aspects of bodily self-consciousness by Makin and colleagues (2008). We only note here that comparison of neuroimaging studies of the rubber hand illusion is hampered by the fact that the studies employed different methods to induce the rubber hand illusion, used different control conditions, different behavioral proxies to quantify illusory touch and ownership, and employed different brain imaging techniques. Not surprisingly, though, these studies implicated several key brain areas that have previously been shown to be important in multisensory integration, such as the premotor and intraparietal cortices as well as the TPJ, insula, extrastriate cortex, and the cerebellum.

24.3.2. Full Body Studies

Although illusory ownership in the rubber hand illusion exemplifies a deviant form of bodily self-consciousness, the illusion only affects partial ownership, or the attribution and localization of a hand with respect to the global bodily self, that is, it is characterized by a change in part-to-whole relationships. As we have seen, the situation is different in neurological patients who have illusory perceptions of their full bodies such as in OBEs and heautoscopy. These states are characterized by abnormal experience with respect to the global bodily self, for example, a mislocalization and a misidentification of the entire body and self (Blanke et al. 2004; Blanke and Mohr 2005; Brugger et al. 1997). Recent studies in healthy subjects (Ehrsson et al. 2007; Lenggenhager et al. 2007, 2009; Mizumoto and Ishikawa 2005) have therefore sought to investigate these global aspects of self-consciousness (self-location and self-identification) by the systematic manipulation of the multisensory cues that the brain uses to create a representation of self-location and identity. As we shall see, these experimental setups have allowed us to gain insight into the biological mechanisms that are important for humans’ everyday “inside-body experience.” They show that this experience—which is often taken for granted (“where else should I be localized than in my body?”)—is made possible by active multisensory brain processes.

Two groups (Ehrsson 2007; Lenggenhager et al. 2007) have separately developed novel techniques to dissociate (1) the location of the physical body, (2) the location of the self (self-location), (3) the location of the origin of the first-person visuospatial perspective, and (4) self-identification. Both groups utilized congruent and incongruent visual–tactile stimulation to alter these four aspects of bodily self-consciousness by extending a protocol similar to that used in the rubber hand illusion (Botvinick and Cohen 1998)—to the full body (see Figure 24.1; see also Altschuler and Ramachandran 2007; Mizumoto and Ishikawa 2005; Stratton 1899). The general idea in these full body studies is to mislead subjects about where they experience their body and/or self to be, and/or with what location and which body they self-identify with. To achieve this, a visual (real-time video) image of their body was presented via a head-mounted display (HMD) that was linked to a video camera that filmed their back from behind (Figure 24.1). They were thus able to see themselves from an “outside” or third-person visuospatial perspective, as though they were viewing their own body from the visuospatial perspective of the camera (note that this is related to changes in perspective during OBEs). In one study (Lenggenhager et al. 2007), subjects viewed the video image of their body (the “virtual body”) while they were stroked on their real back with a stick. This stroking was felt on their back and also seen in front on the virtual body either simultaneously (in real time) or not (when delayed by a video delay). The stroking manipulation thus generated either congruent (synchronous) or incongruent (asynchronous) visuo-tactile stimulation (as had been shown to affect the perception of hand ownership and hand location in the rubber hand illusion; Botvinick and Cohen 1998). It was found that the illusion of self-identification with the virtual body (i.e., global ownership, the feeling that “the virtual body is my body”) and the referral of touch (“feeling the touch of the stick where I saw it touching my virtual body”) were both stronger when subjects were stroked synchronously than when they were stroked asynchronously (Lenggenhager et al. 2007). Self-location was also measured by passively displacing blindfolded subjects after the stroking period and then asking them to walk back to the original position. Note that, as predicted, self-location was experienced at a position that was closer to the virtual body, as if the subject was located “in front” of the position where (s)he had been standing during the experiment. This ensemble of measures has been termed the full body illusion (FBI).

In a related study (Ehrsson 2007), subjects were stroked on their chest (Figure 24.1). They were seated while they viewed themselves (via an HMD) from behind, and they could see a stick moving (synchronous or asynchronous with the touch) just below the camera’s lens. In this case, subjects felt that the stick they saw was touching their real chest, they self-identified with the camera’s location and they felt that looking at the virtual body was like viewing the body of someone else (i.e., decreased self-identification with the virtual body). Self-location was not quantified in this study by using the drift measure as in Lenggenhager et al.’s (2007) study; instead, a threatening stimulus was presented to the apparent location of the origin of the visuospatial perspective (just below the camera). The skin conductance response to a swinging hammer (approaching the camera) was found to be higher during synchronous stroking than during asynchronous stroking, providing implicit physiological evidence that subjects self-identified with a spatial position that was displaced toward the position of the camera.

There were several differences in bodily experiences in these two similar setups, and it is worth considering what may account for these. Meyer (2008) proposed (in a response to these studies) that in both setups the brain may use at least four different sources of information to generate the conscious experience of self-location and self-identification: (1) where the body is seen, (2) where the world is seen from (the origin of the visuospatial perspective), (3) where the touch is seen to occur, and (4) where the touch is felt to occur. These four “cues” do not correspond in both experimental setups (but in everyday life, they usually do). Meyer argued that the most important of these cues for the conscious experience of self-location might be where the touch is seen to occur (i.e., where the stroking stick is seen). He concluded this because, first, in neither setup did self-location (measured via drift by Lenggenhager et al. 2007 or assessed via a questionnaire score by Ehrsson 2007) exactly coincide with the location where the touch was felt (i.e., where the physical body was located). Second, the seen location of the virtual body biased self-location in one study (Lenggenhager et al. 2007) but not in the other (Ehrsson 2007), and third, the location of the visuospatial perspective corresponded to self-location in Ehrsson’s (2007) study but not in Lenggenhager et al.’s (2007) study. However, in both cases (during synchronous stroking), self-location coincided with (or more accurately, was biased toward) the location where the touch was seen to occur (i.e., the seen location of the stroking stick).

It is not very surprising that the tactile sense appears to have the weakest role in determining self-location. Touch, after all, cannot give any reliable information regarding the location of the body in external space, except via tactile contact with external surfaces. There is, however, an additional important point to consider regarding the four cues: self-location was biased toward the virtual body more when the seen stroking was synchronous with the felt stroking than when it was asynchronous (Blanke et al. 2008). Thus, the congruence between tactile and visual input is an additional important factor in determining self-location in this context. It seems that when vision and touch are incongruent, the influence of the “visual information about stroking” is weaker and not preeminent as Meyer implies. Thus, in the asynchronous condition, subjects’ self-location is closer to where the touch is felt (i.e., where their physical body is actually located) than it is in the synchronous condition.

It should be noted that different methods (different experimental conditions and dependent variables to quantify changes in bodily self-consciousness) were used in these studies (Ehrsson 2007; Lenggenhager et al. 2007). It is therefore difficult to make meaningful, direct comparisons between the results of these studies. A more recent study (Lenggenhager et al. 2009) therefore sought to directly compare the approaches presented in these previous studies by using identical body positions and measures in order to quantify the conscious experience of self-identification, first-person visuospatial perspective, and self-location. In addition, the authors investigated these aspects of bodily self-consciousness while subjects were tested in the supine position (as OBEs usually occur in this position; Bünning and Blanke 2005; Green 1968).

Subjects were again fitted with an HMD that displayed a video image of their body. Their virtual body thus appeared to be located below their physical body (see Figure 24.2). The dependent behavioral measure for the quantification of self-location was a new one: a “mental ball dropping” (MBD) task in which subjects had to imagine that a ball fell from their hand, and they had to press one button when they imagined that it left their grasp, and then another button when they imagined that it hit the floor. The authors hypothesized that MBD estimation would be greater (i.e., the time that subjects imagined it would take for the ball to reach the ground would be longer) when subjects’ self-location (where they perceived their self to be) was higher from the ground than when it was closer to the ground. The prediction in this study was that, compared to asynchronous stroking, synchronous back stroking would lead to a “downward” shift in self-location (toward the virtual body, seen as though below subjects) and an increased self-identification with the virtual body. Synchronous chest stroking, conversely, would lead to an “upward” shift in self-location (“away” from the virtual body seen below), and a decreased self-identification with the virtual body. As predicted, self-identification with the virtual body and referral of touch to the virtual body were found to be greater during synchronous than during asynchronous back stroking. In contrast, during synchronous chest stroking, there was decreased self-identification with the virtual and decreased illusory touch. The MBD time estimates (quantifying self-location) were lower for synchronous back stroking than synchronous chest stroking, suggesting that, as predicted, self-location was more biased toward the virtual body in the synchronous back stroking condition and relatively more toward the location of the visuospatial perspective (a third-person perspective) in the synchronous chest stroking condition. This study confirmed the earlier suggestion that self-location and self-identification are strongly influenced by where the stroking is seen to occur. Thus, self-location was biased toward the virtual body located as though below (or in front) when subjects were stroked on the back, and biased toward the location of the visuospatial perspective (behind/above the virtual body) when subjects were stroked on their chests. These studies revealed that humans’ “inside-body” self-location and “inside-body” first-person perspective can be transferred to an extracorporeal self-location and a third-person perspective.

It is notable that the subjective upward drift in self-location during synchronous chest stroking was correlated with sensations of elevation and floating (as assessed by questionnaires). This suggests that when subjects adopt a relaxed prone position—synchronous visual–tactile events may interfere with vestibular processing. The importance of vestibular (otolith) input in abnormal self-location has already been demonstrated (Blanke et al. 2002, 2004). Furthermore, there is evidence that vestibular cues may interfere with body and self-representation (Le Chapelain et al. 2001; Lenggenhager et al. 2008; Lopez et al. 2008; Yen Pik Sang et al. 2006). The relatively motionless prone body position of the subjects in this study would have minimized vestibular sensory updating and thus may have further contributed to the occurrence of such vestibular sensations, highlighting their potential relevance for bodily self-consciousness, OBEs, and related experiences (see also Lopez et al. 2008; Schwabe and Blanke 2008).

Can the mechanisms (explained above) for the rubber hand illusion also explain the changes in self-location, first-person perspective, and self-identification during the FBI? It is probable that some mechanisms are shared but there are likely to be several important conceptual, behavioral, and neurobiological differences. The finding that in the FBI there appears to be referral of touch to a virtual body viewed as though at a distance of 2 m away is in contrast to the finding that the rubber hand illusion is greatly weakened or abolished by changing the posture of the rubber hand to an implausible one (Tsakiris and Haggard 2005) or by placing the rubber hand at more distant positions (Lloyd 2007). Viewing one’s body from an external perspective at 2 m distance is even less “anatomically plausible” than a rubber hand with a misaligned posture; therefore, it is perhaps surprising that the FBI occurs at all under such conditions. However, it has been shown that the visual receptive field size of parietal bimodal neurons with tactile receptive fields centered on the shoulder or the back can be very large—extending sometimes for more than a meter in extrapersonal space (Duhamel et al. 1998; Maravita and Iriki 2004). Shifts in the spatial characteristics of such trunk-centered bimodal neurons may thus account for the observed changes during the FBI (Blanke and Metzinger 2009). What these differences illustrate is that the constraints operating in the FBI are in certain ways markedly different to those operating in the rubber hand illusion. They appear similar in that the strength of both illusions depends on the temporal congruence between seen and felt stroking. However, the constraints regarding the spatial relations between the location of the origin of the first-person visuospatial perspective and the rubber hand are different to those between the location of the origin of the first-person visuospatial perspective and the location of the seen virtual body (see also Blanke and Metzinger 2009). Moreover, in the RHI it is the hand with respect to the body that is mislocalized: a “body part–body” interaction. In the FBI the entire body (the bodily self) is mislocalized within external space: a “body-world” interaction. It may be that the “whole body drift” entails that (during the synchronous condition) the “volume” of peripersonal space is relocated (toward the virtual body) within a stable external space (compatible with subjective reports during OBEs). Alternatively, it may be that peripersonal and extrapersonal space are modified. The dimensions of the external room—for example, the proximity of walls to the subjects—are likely to affect the FBI more than the RHI, but this has not been systematically tested yet. Given the differences between the illusions, it is to be expected that there should be differences in both the spatial constraints and neural bases (at the level of bimodal visuo-tactile neurons and of brain regions encoding multisensory bodily signals) between these illusions.

24.3.3. Mislocalization of Touch during FBIs

The studies discussed above (Ehrsson 2007; Lenggenhager et al. 2007, 2009) suggest that during the FBI changes in self-location and self-identification are accompanied by a mislocalization of touch, that is, the feeling of touch is biased toward where the touch is seen on one’s own body in extrapersonal space. However, the evidence for this that was presented in these studies (Ehrsson 2007; Lenggenhager et al. 2007, 2009) came only from questionnaire ratings, specifically the statements: “It seemed as if I was feeling the touch in the location where I saw the virtual body touched” (Lenggenhager et al. 2007, 2009) and “I experienced that the hand I was seeing approaching the cameras was directly touching my chest (with the rod)” (Ehrsson 2007). Questionnaire ratings, being explicit judgments, are susceptible to various biases, for example, experimenter expectancy effects. Also, the questions were asked only after the period of stroking, not during, and so were not “online” measures of bodily self-consciousness. Furthermore, as recently pointed out (Ehrsson and Petkova 2008), such questions are somewhat ambiguous in a VR setup: they are, arguably, unable to distinguish between self-identification with a virtual body and self-recognition in a VR/video system. A more recent study (Aspell et al. 2009) therefore developed an online measure for the mislocalization of touch that would be less susceptible to response biases and that would test more directly whether tactile mapping is altered during the FBI. This study investigated whether modifications in bodily self-consciousness are associated with changes in tactile spatial representations.

To investigate this, the authors (Aspell et al. 2009) adapted the cross-modal congruency task (Spence et al. 2004) for the full body. This task was used because the cross-modal congruency effect (CCE) measured in the task can function as a behavioral index of the perceived proximity of visual and tactile stimuli. In previous studies of the CCE (Igarashi et al. 2008; Pavani and Castiello 2004; Pavani et al. 2000; Shore et al. 2006; Spence et al. 2004), the visual and tactile stimuli were presented on foam cubes held in the hands: single vibrotactile devices paired with small lights [light emitting diodes (LEDs)] were positioned next to the thumb and index finger of each hand. Subjects made speeded elevation discriminations (“up”/index or “down”/thumb) of the tactile stimuli while attempting to ignore the visual distractors. It was found that subjects performed worse when a distracting visual stimulus occurred at an incongruent elevation with respect to the tactile (target) stimulus. Importantly, the CCE (difference between reaction times during incongruent and congruent conditions) was larger when the visual and tactile stimuli occurred closer to each other in space (Spence et al. 2004).

The CC task was adapted for the full body (from the typical setup for the hands; Spence et al. 2004) by placing the vibrotactile and LEDs on the subject’s torso (back). Subjects were able to view their body and the LEDs via an HMD (see Figure 24.3) as the setup was similar to that used in the previous FBI study (Lenggenhager et al. 2007). To investigate whether “full body CCEs” would be associated in a predictable way with changes in bodily self-consciousness, subjects’ self-identification with the virtual body and self-location were manipulated across different blocks by employing either synchronous or asynchronous stroking of the subjects’ backs. CCEs were measured during the stroking period and, as predicted, were found to be larger during synchronous than asynchronous blocks, indicating that, as predicted, there was a greater mislocalization of touch during synchronous stroking compared to during asynchronous stroking. [Note that although a number of components—attention, response bias, and multisensory integration—are all thought to contribute to the CCE to varying degrees (e.g., depending on the stimulus-onset asynchrony between the visual and tactile stimuli)—the finding of a difference in the CCE between same and different side stimuli during the synchronous condition, but not during the asynchronous condition, indicates that the visual and tactile stimuli were represented as being closer to each other in the former case.] In the synchronous condition, there was also a greater bias in self-location toward the virtual body and a greater self-identification with the virtual body compared to in asynchronous blocks (as in Lenggenhager 2007). Control conditions revealed that the modulating effect of spatial remapping of touch was body-specific.

Interestingly, this study also found that the size of the CCE, the degree of self-identification with, and the bias in self-location toward the virtual body were all modulated by the stimulus onset synchrony between the visual and vibrotactile stimuli used in the CCE task. These data thus suggest that certain key components of bodily self-consciousness—that is, “what I experience as my body” (self-identification) and “where I experience my body to be” (self-location)—are associated with changes in the spatial representation of tactile stimuli. They imply that a greater degree of visual capture of tactile location occurs when there is a greater degree of self-identification for the seen body. This change in the tactile spatial representation of stimuli is not a remapping on the body, but is, we suggest, a change in tactile mapping with respect to extrapersonal space: the tactile sensations are perceived at a spatial location biased toward the virtual body.

24.3.4. Multisensory First-Person Perspective

Less work has been carried out on the question of whether the experienced spatial position of the first-person perspective can be dissociated from that of self-location (Blanke and Metzinger 2009; Schwabe and Blanke 2008). The aforementioned FBI studies suggest that the first-person visuospatial perspective can (at least with a video setup) be dissociated from self-location in healthy subjects. This has rarely been reported in patients with own body illusions such as OBEs and related experiences. As seen above, in a typical OBE the self is experienced as “colocalized” with the first-person visuospatial perspective. However, a recent neurological study (De Ridder et al. 2007) showed that intracranial electrical stimulation at the right TPJ may lead to the experience of dissociation of self-location from the first-person visuospatial perspective. Thus, the patient experienced extracorporeal self-location and disembodiment to a position behind his body, but perceived the environment from his normal, body-centered, first-person visuospatial perspective (and not from the disembodied perspective as is classically reported by people with OBEs). Furthermore, some patients suffering from heautoscopy may experience two rapidly alternating first-person visuospatial perspectives and self-locations (Blanke et al. 2004; Brugger et al. 1994). In such patients, the first-person visuospatial perspective may sometimes even be experienced at two positions at the same time and this is often associated with feelings of bilocation: the experience of a duplicated or split self, that is, not just a split between body and self as in OBEs, but between two experienced self-locations (see also Lopez et al. 2008).

The first-person visuospatial perspective is perhaps the only perspective that usually comes to mind, and yet vision is not the only modality with an inherent “perspectivalness” (Metzinger et al. 2003; Metzinger 2007)—there is certainly also an auditory first-person perspective and possibly also “perspectives” based primarily on proprioceptive and motor signals (Schwabe and Blanke 2008). Again, in healthy subjects the auditory perspective and visual perspective are spatially congruent, and yet patients with heautoscopy may describe spatial incongruence between these perspectives (for further examples and discussion, see Blanke et al. 2004; Blanke and Metzinger 2009).