Chapter 37Assessing the Role of Visual and Auditory Cues in Multisensory Perception of Flavor

Zampini M, Spence C.

Publication Details

37.1. INTRODUCTION

Our perception of the objects and events that fill the world in which we live depends on the integration of the sensory inputs that simultaneously reach our various sensory systems (e.g., vision, audition, touch, taste, and smell). Perhaps the best-known examples of genuinely multisensory experiences come from our perception and evaluation of food and drink. The average person would say that the flavor of food derives primarily from its taste in the mouth. They are often surprised to discover that there is a strong “nasal” role in the perception of flavor. In fact, it has been argued that the majority of the flavor of food actually comes from its smell (e.g., Cain 1977; Murphy and Cain 1980; Rozin 1982). * Our perception of food and drink, however, is not simply a matter of combining gustatory and olfactory food cues (although this is undoubtedly very important; Dalton et al. 2000). For instance, our evaluation of the pleasantness of a particular foodstuff can be influenced not only by what it looks, smells, and tastes like, but also what it sounds like in the mouth (think, for example, of the auditory sensations associated with biting into a potato chip or a stick of celery; see Spence and Zampini 2006, for a review). The feel of a foodstuff (i.e., its oral–somatosensory attributes) is also very important; the texture, temperature, viscosity, and even the painful sensations we experience when eating hot foods (e.g., chilli peppers) all contribute to our overall multisensory experience of foodstuffs (e.g., Bourne 1982; Lawless et al. 1985; Tyle 1993). Flavor perception is also influenced by the interactions taking place between oral texture and both olfactory and gustatory cues (see also Bult et al. 2007; Christensen 1980a, 1980b; Hollowood et al. 2002). Given the multisensory nature of our perception of food, it should come as little surprise that many studies have been conducted in order to try and understand the relative contribution of each sense to our overall evaluation of food (e.g., see Delwiche 2004; Spence 2002; Stevenson 2009; Stillman 2002). In this chapter, we review the contribution of visual and auditory cues to the multisensory perception of food. Moreover, any possible influence of visual and auditory aspects of foods and drinks might take place at different stages of the food experience. Visual cues are perceived when foodstuffs are outside of the mouth. Auditory cues are typically primarily perceived when we are actually consuming food.

37.2. MULTISENSORY INTERACTIONS BETWEEN VISUAL AND FLAVOR PERCEPTION

37.2.1. Role of Color Cues on Multisensory Flavor Perception

Over the past 80 years or so, many researchers have been interested in the role of visual information in the perception of foodstuffs (Moir 1936). It seems that the visual appearance of food and drink can have a profound impact on our perception and evaluation of flavor. The role of color cues on people’s flavor perception has been investigated in many different studies, although the majority of the research has been published in food science journals rather than psychology or neuroscience (for reviews, see Clydesdale 1993; Delwiche 2004; Spence et al. 2010; Stevenson 2009). The majority of these studies have shown that people’s perception of a variety of different foods and drinks can be dramatically modified by changing the color of food or drink items (e.g., DuBose et al. 1980; Duncker 1939; Garber et al. 2000; Johnson and Clydesdale 1982; Morrot et al. 2001; Philipsen et al. 1995; Roth et al. 1988; Stillman 1993; Wheatley 1973; Zampini et al. 2007; Zellner and Durlach 2003).

One of the most dramatic early empirical demonstrations of the strong link between color and the pleasure we derive from food (and/or our appetitive responses to food) was reported by Wheatley (1973). He described a situation in which a group of people ate a meal of steak, French fries, and peas under color-masking lighting conditions. Halfway through the meal, normal lighting was restored revealing that the steak was colored blue, the French fries had been colored green, and the peas were red. According to Wheatley’s description, the mere sight of the food was sufficient to induce nausea in many of his dinner guests. Such results, although anecdotal, do at least hint at the powerful influence that visual cues can have over our appetitive responses.

Color has also been shown to exert a significant effect on our ability to recognize specific foodstuffs. For example, in one oft-cited study, DuBose et al. (1980) presented participants with drinks incorporating a variety of different color–flavor combinations (the flavored solutions were colored either appropriately, inappropriately, or else were presented as colorless solutions). DuBose et al. found that participants’ identification of the flavors of many of the drinks was significantly influenced by their color. In particular, the participants were less accurate in identifying the flavor of fruit-flavored beverages when they were unaware of the appropriate color. For instance, 40% of the participants reported that a cherry-flavored beverage actually tasted of orange when it had been inappropriately colored orange (compared to 0% orange-flavor responses when the drink was appropriately colored red; a similar effect was reported for the lime-flavored beverage). Many other researchers have reported a similar visual modulation of participants’ odor discrimination/identification responses (e.g., Blackwell 1995; Davis 1981; Koza et al. 2005; Morrot et al. 2001; Stevenson and Oaten 2008; Zellner et al. 1991; Zellner and Kautz 1990; Zellner and Whitten 1999).

Although the potential influence of color cues on people’s flavor identification responses is by now well documented, the evidence regarding the impact of changes in color intensity on perceived flavor intensity is rather less clear. For example, ambiguous results have been reported in studies in which the participants had to rate the intensity of the flavor of solutions that varied in the intensity of the color that had been added to the solutions (e.g., DuBose et al. 1980; Johnson and Clydesdale 1982; Johnson et al. 1983; see Clydesdale 1993, for a review). For example, DuBose et al. found that overall flavor intensity was affected by color intensity, with more intense coloring resulting in stronger flavor evaluation responses by participants for the orange-flavored, but not for the cherry-flavored beverages, tested in their study. However, in other studies, the concentration of coloring in the solutions did not influence participants’ ratings, regardless of whether the solutions were appropriately or inappropriately colored (e.g., Alley and Alley 1998; Frank et al. 1989; Zampini et al. 2007).

Researchers have also investigated the effect of varying the intensity of the color on the perceived intensity of tastes and odors separately. For instance, the addition of a red coloring to cherry-and strawberry-flavored sucrose solutions has been found to increase the perceived sweetness of these solutions in certain studies (Johnson and Clydesdale 1982; Johnson et al. 1983). Maga (1974) hypothesized that the influence of colors on sweetness perception in humans might be particularly strong for colors that are typically associated with the natural ripening of fruits (e.g., yellow, red; see also Lavin and Lawless 1998; Strugnell 1997). By contrast, researchers have reported that the addition of color has no effect on the perceived saltiness of foods such as soups (Gifford and Clydesdale 1986; Gifford et al. 1987; Maga 1974), perhaps because (in contrast to sweet foods) there are no particular colors associated with the salt content of a food (i.e., salt is ubiquitous to many different kinds, and hence colors, of food; see Maga 1974 and Lavin and Lawless 1998, on this point).

In one of the earliest studies to have been published in this area, Pangborn (1960) reported that people reported green-colored pear nectar as being less sweet than colorless pear nectar. However, Pangborn and Hansen (1963) failed to replicate these results. Although they found that green coloring had no effect on the perceived sweetness of pear nectar, its addition did give rise to an overall increase in sensitivity to sweetness when color was added to the solutions. Similarly, for the pairing of color with odor, Zellner and Kautz (1990) reported that solutions were rated as having a more intense odor when color had been added to the solutions than when it was absent, regardless of the appropriateness of the color–odor match. In fact, Zellner and Kautz noted that the participants in their study simply refused to believe that colored and uncolored solutions of equal odor intensity were actually equally strong.

The explanation for these contradictory results regarding the influence of variations in color intensity on the perception of taste, odor, and flavor (i.e., odor + taste) intensity is far from obvious (see Shankar et al. 2010). For example, Chan and Kane-Martinelli (1997) reported that the perceived flavor intensity for certain foods (such as chicken bouillon) was higher with the commercially available color sample than when the samples were given in a higher-intensity color (see also Clydesdale 1993, on this point). Note also that if the discrepancy between the intensity of the color and the intensity of the flavor is too great, participants may experience a disconfirmation of expectation (or some form of dissonance between the visually and gustatorily determined flavor intensities) and the color and taste cues may no longer be linked (e.g., Clydesdale 1993; cf. Ernst and Banks 2002; Yeomans et al. 2008).

Another potentially important issue in terms of assessing interactions between color and flavor is the role of people’s awareness of the congruency of the color–flavor pairings used (Zampini et al. 2007). In fact, in most of the research that has been published to date on the effects of color cues on human flavor perception, the participants were not explicitly informed that the flavors of the solutions they were evaluating might not be paired with the appropriately colored solutions (e.g., see DuBose et al. 1980; Johnson and Clydesdale 1982; Morrot et al. 2001; Oram et al. 1995; Philipsen et al. 1995; Roth et al. 1988; Stillman 1993; Zellner and Durlach 2003). One might therefore argue that the visual modulation of flavor perception reported in many of these previous studies simply reflects a decisional bias introduced by the obvious variation in the color cues (cf. the literature on the effectiveness of the color of medications on the placebo effect; e.g., de Craen et al. 1996; see also Engen 1972), rather than a genuine perceptual effect (i.e., whereby the color cues actually modulate the perception of flavor itself; although see also Garber et al. 2001, 2008, for an alternative perspective from the field of marketing). For example, if participants found it difficult to correctly identify the flavor of the food or drink on the basis of gustatory and olfactory cues in flavor discrimination tasks, then they may simply have based their responses on the more easily discriminable color cues instead. Therefore, it might be argued that participants’ judgments in these previous studies may simply have been influenced by decisional processes instead.

In their study, Zampini et al. (2007) tried to reduce any possible influence of response biases that might emerge when studying color–flavor interactions by explicitly informing their participants that the color–flavor link would often be misleading (i.e., that the solutions would frequently be presented in an inappropriate color; cf. Bertelson and Aschersleben 1998). This experimental manipulation was introduced in order to investigate whether the visual cues would still influence human flavor perception when the participants were aware of the lack of any meaningful correspondence between the color and the flavor of the solutions that they were tasting. The participants in Zampini et al.’s study were presented with strawberry, lime, orange fruit–flavored solutions or flavorless solutions, and requested to identify the flavor of each solution. Each of the different flavors was associated equiprobably with each of the different colors (red, green, orange, and colorless). This meant that, for example, the strawberry-flavored solutions were just as likely to be colored red, green, orange, as to be presented as a colorless solution. Therefore, each of the solutions might have been colored either “appropriately” or “inappropriately” (consisting of incongruently colored or colorless solutions). The participants were informed that they would often be tricked by the color of the solutions that would often not correspond to the flavor typically associated with that color.

The most important finding to emerge from Zampini et al.’s (2007) study was that color information has a strong impact on flavor identification even when participants were informed that the colors of the drinks that they were testing were often misleading. In particular, flavors associated with appropriate colors (e.g., lime flavor–green color; orange flavor–orange color) or colorless were recognized far more accurately than when they were presented with an inappropriate coloring (i.e., lime-flavored drinks that were colored either red or orange; orange-flavored drinks that were colored either green or red). These results therefore show that inappropriate coloring tends to lead to impaired flavor discrimination responses, whereas appropriate coloring does not necessarily improve the accuracy of participants’ flavor discrimination responses (at least when compared to the flavor discrimination accuracy for the colorless solutions). Interestingly, however, no significant effect of color was shown for the strawberry-flavored solutions. That is, the inappropriate coloring of the strawberry-flavored solutions (i.e., when those solutions were colored green or orange) did not result in a significant reduction in the participants’ ability to recognize the actual strawberry flavor. One possible explanation for this result is that those flavors that are more strongly associated with a particular color are more difficult to identify when presented in inappropriately colored solutions (see Shankar et al. 2009). In fact, Zampini et al. (2007, Experiment 1; see Table 37.1) has shown that the link between color and a specific flavor was stronger for the orange- and green-colored solutions than for the red-colored solutions. That is, the participants in their study more often matched the orange color with the flavor of orange and the green color with the flavor of lime. By contrast, the red color was associated with strawberry, raspberry, and cherry flavors.

TABLE 37.1. Flavors Most Frequently Associated with Each Colored Solution in Zampini et al.′s (2007, Experiment 1) Study.

TABLE 37.1

Flavors Most Frequently Associated with Each Colored Solution in Zampini et al.′s (2007, Experiment 1) Study.

Whatever the reason for the difference in the effect of the various colors on participants’ flavor discrimination responses, it is important to note that Zampini et al.’s (2007) results nevertheless show that people can still be misled by the inappropriate coloring of a solution even if they know that the color does not provide a reliable guide to the flavor of the solution. By contrast, the participants in the majority of previous studies in this area (e.g., DuBose et al. 1980; Johnson and Clydesdale 1982; Morrot et al. 2001; Oram et al. 1995; Philipsen et al. 1995; Roth et al. 1988; Stillman 1993; Zellner and Durlach 2003) were not explicitly informed that the flavors of the solutions might not be paired with the appropriately colored solutions. Zampini et al.’s results therefore suggest that the modulatory role of visual information on multisensory flavor perception is robust enough to override any awareness that participants might have (e.g., as informed by the experimenter) concerning the lack of congruency between the color and the flavor of the solutions that they taste. However, it would be interesting in future research to investigate whether knowing that there is no meaningful relationship between the color of the solutions and their flavor would modulate (i.e., reduce vs. enhance) the influence of colors on flavors perception, as compared to the situation in which the participants are not given any prior information about whether the colors are meaningfully related to the flavors.

37.2.2. Color–Flavor Interactions: Possible Role of Taster Status

Given recent interest in the consequences of individual differences in taster status on flavor perception (see Drewnowski 2003, for a review), Zampini and his colleagues (2008) wanted to investigate whether any possible multisensory effects of visual (i.e., the colors of the solutions) and/or gustatory (i.e., the presence vs. absence of fruit acids) cues on flavor perception might be affected by the taster status of their participants. Previous research has demonstrated the existence of three subgroups of tasters (nontasters, medium tasters, and supertasters), varying in their sensitivity to 6-n-propylthiouracil (PROP; e.g., Bartoshuk et al. 1992) as well as to a variety of other tastants (e.g., Prescott et al. 2001; Reed 2008). * Surprisingly, however, none of the previous studies that has investigated individual differences in taste perception has as yet looked at the possible influence of taster status on the visual modulation of (or dominance over) flavor perception.

In Zampini et al.’s (2008) study, the taster status of each participant was initially assessed using suprathreshold PROP filter paper strips (see Bartoshuk et al. 1994). The participants had to place the PROP filter paper strips on their tongue and then rate the intensity of the sensation of bitterness that they experienced on a Labelled Magnitude Scale (e.g., Green et al. 1993). The participants were then classified into one of three taster groups: nontasters, medium tasters, and supertasters based on the cutoff values (non-tasters <10.90; 10.91 < medium tasters < 61.48; supertasters > 61.49; see also Essick et al. 2003, for a similar criterion). Zampini et al.’s findings revealed that the modulatory cross-modal effect of visual cues on people’s flavor identification responses were significantly more pronounced in the nontasters than in the medium tasters, who, in turn, were influenced to a greater extent by visual cues on their flavor identification responses than were the supertasters (see Figure 37.1). In particular, the nontasters (and, to a lesser extent, medium tasters) identified the flavors of the solutions significantly more accurately when they were colored appropriately than when they were colored inappropriately (or else were presented as colorless solutions). By contrast, the supertasters identified the flavors of the solutions more accurately overall, and their performance was not affected by the colors of the solutions.

FIGURE 37.1. Mean flavor intensity ratings for three groups of participants (nontasters, medium tasters, and supertasters) for blackcurrant, orange, and flavorless solutions presented in Zampini et al.

FIGURE 37.1

Mean flavor intensity ratings for three groups of participants (nontasters, medium tasters, and supertasters) for blackcurrant, orange, and flavorless solutions presented in Zampini et al.’s (2008) study of effects of color cues on multisensory (more...)

Zampini et al.’s (2008) results are consistent with recent accounts of sensory dominance derived from studies of cross-modal interactions between tactile, visual, and auditory stimuli (see, e.g., Alais and Burr 2004; Ernst and Banks 2002). Ernst and Banks used the maximum likelihood estimation (MLE) approach to argue that the contribution of a given sensory input to multisensory perception is determined by weighting the sensor estimates in each sensory modality by the noise (or variance) present in that modality. It could be argued that in Zampini et al.’s study, the estimates of the flavors of the fruit-flavored solutions by the nontasters were simply more variable (i.e., their judgments were less sensitive) than those of either the medium tasters or the supertasters. As a consequence, given the presumably uniform levels of visual discriminability across these three groups of participants, the MLE account would predict that nontasters should weigh the visual cues more highly when making their responses than the medium tasters, who in turn should weigh the gustatory cues less highly than the supertasters, just as we observed. It will be an interesting question for future research to determine whether flavor discrimination responses can be modeled using the MLE approach. It is important to note here that such an analysis may also be able to reveal whether there are any underlying attentional biases (to weight information from one sensory modality more highly than information from another modality) that may be present in the different taster groups (cf. Battaglia et al. 2003). Moreover, it is interesting to consider at this point that although more than 100 studies examining visual contributions to flavor perception have been published over the past 80 years, Zampini et al.’s study represents the first attempt to take the taster status of participants into consideration when analyzing their results. The results of Zampini et al.’s study clearly demonstrate that taster status plays an important role in modulating the cross-modal contribution of visual cues to flavor perception in fruit-flavored beverages. *

37.2.3. Color–Flavor Interactions: Possible Role of Learned Associations between Colors and Flavors

The influence of color on flavor perception may (and, some might say, must) be due to learned associations between specific colors and particular flavors. Some of these associations are fairly universal. For example, the flavor and color association of ripe fruits (Maga 1974; see also Morrot et al. 2001). By contrast, other color–flavor association might be context dependent and so might be different in different parts of the world (see Duncker 1939; Lucchelli et al. 1978; Shankar et al. 2010; Spence 2002; Wheatley 1973). For instance, lemons are typically yellow in Europe, whereas in Colombia they are mostly dark green. Therefore, a particular color–flavor pairing that seems congruent to people in a certain part of the world may seem incongruent to those who live elsewhere (cf. Demattè et al. 2006, 2009).

Seventy years ago, Duncker (1939) considered the role of individual differences in learning such associations. The participants in his early study were presented with milk chocolate that had been colored brown or white (a new color for chocolate at the time the study was conducted) both with the same flavor. Participants who had never seen white chocolate before reported that the white chocolate had a different flavor to the brown-colored chocolate. The only participant who had come across white chocolate before taking part in the study reported that the different colored chocolates all tasted the same. Although it should be noted that this early study had a number of methodological limitations (i.e., only a small number of participants were tested, not to mention the fact that no statistical analysis of the data was reported), the results nevertheless highlight the possible importance of prior experience and knowledge in modulation color–flavor interactions.

A follow-up of Duncker’s (1939) seminal study has been conducted recently by Levitan et al. (2008; see also Shankar et al. 2009). The researchers in this study investigated whether people’s prior beliefs concerning specific color–flavor associations might not affect their ability to discriminate the flavor of colored sugar-coated chocolate sweets, Smarties (Nestlé). Smarties are readily available in eight different colors but only two different flavors, as test stimuli: the orange Smarties that are produced for the UK market contain orange-flavored chocolate, whereas all of the other colors contain unadulterated milk chocolate. By contrast, Smarties that have been produced for other markets all contain unadulterated milk chocolate, regardless of their color. Crucially, the participants were sometimes presented with pairs of stimuli that differed in their color but not in their flavor, or with pairs of Smarties that differed in both their color and flavor, or else with Smarties pairs that differed in their flavor but not their color.

In a preliminary questionnaire, a number of the participants in Levitan et al.’s (2008) study stated their belief that a certain non-orange (i.e., red and green) Smartie had a distinctive flavor (which is incorrect), whereas other participants believed (correctly) that all the non-orange Smarties tasted the same. In the first experiment, the participants were presented with all possible pairings of orange, red, and green Smarties and were asked to judge whether a given pair of Smarties differed in flavor by tasting them while either sighted or blindfolded. The results showed that people’s beliefs concerning specific color–flavor associations for Smarties exerted a significant modulatory effect on their flavor responses. In the sighted condition, those participants who believed that non-orange Smarties all taste the same were more likely to judge correctly that a red–green pairing of Smarties tasted identical in comparison to the first group, who performed at a level that was significantly below chance (i.e., they reported that the red and green Smarties tasted different on the majority of trials). In other words, those participants who thought that there was a difference between the flavors of the red and green Smarties did in fact judge the two Smarties as tasting different far more frequently when compared with participants who did not hold such a belief in the sighted condition. The results of Levitan et al.’s study are consistent with the results of the other studies presented in this section in showing that food color can have a powerful cross-modal influence on people’s perception of the flavor of food. However, Levitan et al.’s findings show that people’s beliefs about the cross-modal color–flavor associations of specific foods can modulate this influence, and that such cognitive influences can be robust and long-lasting despite extensive experience with the particular food item concerned. *

In another recent study, Shankar et al. (2009) found that another variety of sugar-coated chocolate candies (multicolored M&Ms, which are all physically identical in taste) were rated as having a stronger chocolate flavor when they were labeled as “dark chocolate” than when they were labeled as “milk chocolate.” Many other studies have found a similar effect of expectations produced by labeling a stimulus before sampling on flavor perception (see Cardello 1994; Deliza and MacFie 1996; Lee et al. 2006; Yeomans et al. 2008; Zellner et al. 2004, for reviews). Shankar et al. have also investigated whether the influence of expectations on flavor perception might be driven by color information (see Levitan et al. 2008). In their study, participants were asked to evaluate how “chocolatey” they found green- or brown-colored M&Ms. Participants rated the brown M&Ms as being more “chocolatey” than the green ones. This result suggests that the color brown generates stronger expectations of “chocolate” than the green color (cf. Duncker 1939). Finally, Shankar et al. studied whether there was an interaction between the expectation generated by either color or label on multisensory flavor perception. The participants were again presented with brown-or green-colored M&Ms and informed about the “chocolate category” (i.e., either “milk chocolate” or “dark chocolate”) with each color–label combination (green–milk, brown–milk, green–dark, brown–dark) presented in a randomized order. Brown-colored M&Ms were given a higher chocolatey rating than green-colored M&Ms. Similarly, those labeled as “dark chocolate” were given higher ratings than those labeled “milk chocolate.” However, no interaction between these colors and labels was found, thus suggesting that these two factors exerted independent effects, implying that two distinct associations were being retrieved from memory and then utilized (e.g., the color–flavor association and the label–flavor association). Shankar et al.’s findings therefore provide the first evidence that color can influence the flavor of a product whose flavor identity cannot be predicted by its color. In other words, the colors of the coatings of the M&Ms are independent of their taste (which is always chocolate).

One final issue that remains unresolved here concerns the extent to which the influence of color on flavor discrimination reflects a perceptual versus a more decisional effect, or whether instead both perceptual and decisional factors may contribute to participants’ performance (see Spence et al., submitted; and Zampini et al. 2007, on this point). If it is a purely perceptual effect, the participant’s gustatory experience should be changed by viewing the color, that is, knowledge of the color might improve the sensitivity of participants’ flavor discrimination responses by reducing the variability of the multisensory flavor signal. Alternatively, however, according to the decisional account, people should always have given the same gustatory response for a given color–flavor pairing regardless of whether sighted or blindfolded. In fact, what may have changed is their decisional criteria. In Levitan et al.’s (2008) study, the participants who were uncertain of their responses for a given pair of Smarties might have biased their choice toward making different responses because they could see that they had a different color. By contrast, those participants who already knew that red and green Smarties were normally identical in taste might have been biased toward making a same response. In the case of olfaction, Engen (1972) has already shown results consistent with the claim that color can influence odor perception as a result of its effect on decisional mechanisms, but this does not, of course, necessarily rule out a role for perceptual interactions as well, at least when tested under the appropriate experimental conditions (see Zellner and Kautz 1990).

However, it is possible to hypothesize that a person’s beliefs about particular foods tasting different if they have a different color may paradoxically result in them actually tasting different. Analogously, de Craen et al. (1996) discussed a number of findings showing that color cues modulate the effectiveness of medicines as well as placebo pills. Although the mechanism behind placebo effects such as these is not as yet well understood, the effects themselves are nevertheless robust (e.g., for a recent review, see Koshi and Short 2007). What is more, just as in Levitan et al.’s (2008) Smarties experiment, there is at least some evidence that different people may hold different beliefs about differently colored pills, and that these beliefs can carry over into the actual effects that the differently colored placebo pills are shown to have (Lucchelli et al. 1978). Therefore, if people’s beliefs about color and medication can affect their physical state (e.g., resulting in a genuine change in their tolerance for pain, say, or in their ability to sleep), it would seem conceivable that a person’s belief that a certain colored Smartie tasted distinctive (from a Smartie of a different color) might give rise to the effect of it, paradoxically, actually tasting different to that person, despite there being no physical difference in flavor.

37.2.4. Color–Flavor Interactions: Neural Correlates

The results discussed so far on the potential influences of visual cues on flavor perception are consistent with the growing body of neurophysiological and electrophysiological data demonstrating the intimate link between visual, olfactory, and gustatory flavor information at a neuronal level (Osterbauer et al. 2005; Rolls 2004; Rolls and Baylis 1994; Small 2004; Small and Prescott 2005; Verhagen and Engelen 2006). For instance, Osterbauer and his colleagues have used functional neuroimaging to investigate how activity in the human orbitofrontal cortex (OFC) can be modulated by the presentation of particular combinations of odors and colors. The participants in this study had to smell different odors including lemon, strawberry, spearmint, and caramel that were presented by means of a computer-controlled olfactometer. The odors were presented in isolation or else together with a color. The participants wore prism glasses to see full-screen colors presented onscreen outside the magnet bore. On some occasions the odor matched the color, such as when the smell of lemon was presented with the color yellow, whereas at other times the odor and color did not match, such as when spearmint odor was presented with the color brown. Osterbauer et al.’s findings revealed that the presentation of appropriate odor–color combinations (e.g., odor of strawberry matched with red color) increased the brain activity seen in the OFC when compared with the brain activation seen in the odor-alone conditions. By contrast, there was a suppression of neural activity in the same area when inappropriate color–odor combinations were presented (e.g., when the odor of strawberry was presented with a turquoise patch of color on the monitor; see also De Araujo et al. 2003). Taken together, these results would appear to suggest that presenting an appropriate color–odor association may actually lead to increased neural activity in brain areas responsible for processing olfactory stimuli, whereas presenting inappropriate color–odor associations can suppress brain activity below that observed to the odors alone. The positive correlation between the perceived congruency of color–odor pairs and the changes in the pattern of brain activation found in Osterbauer et al.’s study (see also Skrandies and Reuther 2008), therefore, provides a neurophysiological basis for the perceptual changes elicited by changing the color of food.

37.2.5. Interim Summary

Taken together, the results reviewed thus far demonstrate that visual information can have a dramatic impact on flavor perception and evaluation in humans. In particular, most of the studies have shown that it is possible to impair flavor discrimination performance by coloring fruit-flavored solutions inappropriately. The effect of color cues on human flavor perception can be explained by the fact that visual information sets up an expectation regarding the flavor that is about to be experienced. This expectation may originate from any previous experience with similar food stimuli that have contributed to build up such associations between the visual aspect and the experienced flavor (see Shankar et al. 2010; Yeomans et al. 2008). Stevenson and his colleagues (e.g., Stevenson and Boakes 2004; Stevenson et al. 1998) have suggested than any interaction taking place between gustation and olfaction might be explained in terms of associative learning processes. Their findings show that we are able to create strong links between odors and tastes that are repeatedly presented together. It is possible to hypothesize that the strong correspondences between colors and flavors may rely on a similar mechanism. The same foodstuffs are usually experienced first through their visual appearance and then through their flavor. It is possible that in our life we learn to build up a strong association between visual and flavor food properties that are systematically combined. Therefore, people who are presented first with the visual aspect of food and drinks generated a series of expectations about the flavor that those food and drinks should have. White and Prescott (2007) have put forward a similar explanation for their findings regarding the influence of odors on tastes identification, when odors were presented in advance of taste.

In the previous section, a study was discussed in which participants’ beliefs on the color–flavor association based on their previous experiences significantly modulated their responses (see Levitan et al. 2008). In particular, participants who expected a difference between food products that were colored differently were more likely to report a difference than those without any such prior belief. Therefore, flavor perception might be considered as constituting a multisensory experience with somewhat different rules that those regulating other multisensory interactions. Research suggests that spatial coincidence and temporal synchrony are two of the key factors determining whether multisensory integration will take place (at the single cell level) to give rise to the rich multisensory perceptual objects that fill our everyday lives (for reviews, see Calvert et al. 2004). Given that the cross-modal influence of visual cues on flavor perception occur long before we taste foods and occur in different regions of space (i.e., food is only ever seen outside the oral cavity but tasted within it; see Hutchings 1977), it would seem reasonable to suggest that expectancy plays a greater role than the spatial and temporal rules (see Shankar et al. 2010). It might, for example, be less likely that visual-flavor interactions would be influenced by the spatial and temporal rules of multisensory integration (which might better help to explain the integration of auditory, visual, and tactile, that is, the spatial senses; it might also explain the integration of olfactory/gustatory and oral–somatosensory cues in the basic flavor percept). Therefore, we believe that the multisensory study of flavor perception is particularly interesting for multisensory researchers precisely because the rules of integration, and cross-modal influence, are likely to be somewhat different.

In the previous sections, we also discussed how individual differences can affect the nature of the cross-modal visual–flavor interactions that are observed. In particular, visual influences on multisensory flavor perception can be significantly modulated as a function of the taster status of the participant. Visual dominance effects in multisensory flavor perception are more pronounced in those participants who are less sensitive to gustatory cues (i.e., nontasters) than in supertasters, who appear to enjoy the benefit of enhanced gustatory resolution. Therefore, taster status, although often neglected in studies investigating color–flavor interactions, should certainly be considered more carefully in any future research in this area. Finally, we have reviewed the role of expectancy resulting from visual information on the overall food perception.

37.3. ROLE OF AUDITORY CUES IN THE MULTISENSORY EXPERIENCE OF FOODSTUFFS

Most of the visual cues typically occur before our consumption of food and drink, whereas auditory cues are typically only available at the moment of consumption (or mastication). Therefore, one might expect the role of expectancy to be reduced when looking at the effect of sounds on the perception of food. Certainly, visual and auditory cues provide information at distinct stages of eating. In the second part of this chapter, we therefore briefly discuss the possible role that auditory cues may play in the multisensory perception of foodstuffs.

Several studies have demonstrated the influential role that auditory information plays in our perception of food (for a review, see Spence and Zampini 2006). For example, it has been shown that people’s ratings of the pleasantness of many foods can be strongly influenced by the sounds produced when people bite into them (e.g., Drake 1970; Vickers 1981, 1983; Vickers and Bourne 1976). Food sounds have a particularly noticeable influence on people’s perception of crispness that is closely associated with pleasantness, especially in crunchy foods (i.e., crisps; e.g., Vickers 1983). Taken together, these results therefore suggest that the perception of the crispness of (especially) crunchy foods (e.g., crisps, biscuits, cereals, vegetables) is largely characterized by tactile, mechanical, kinesthetic, and auditory properties (e.g., Vickers 1987).

Many foodstuffs produce particular sounds when we eat them. For instance, Drake (1963) reported that the sounds produced by chewing or crushing a variety of different foodstuffs varied in their amplitude, frequency, and temporal characteristics. Analysis of the auditory characteristics of different foods has shown that crispy foods are typically higher in pitch than crunchy foods (Vickers 1979). However, the role of auditory cues in the evaluation of food qualities (e.g., crispness) have been investigated by using different kinds of foodstuffs, that might have different levels of freshness (e.g., Christensen and Vickers 1981; Drake 1963; Seymour and Hamann 1988; Vickers 1984; Vickers and Bourne 1976; Vickers and Wasserman 1979). Those studies also clearly show that despite the informational richness contained in the auditory feedback provided by biting into and/or chewing food, people are typically unaware of the effect that such sounds have on their overall multisensory perception or evaluation of particular stimuli. In particular, Zampini and Spence (2004, 2005) have shown that people’s perception and evaluation of different foodstuffs (e.g., potato chips and sparkling water) can be modulated by changing the overall sound level or just the high-frequency components (see also Chen et al. 2005; Masuda et al. 2008; Varela et al. 2006).

37.3.1. Effect of Sound Manipulation on the Perception of Crisps

Zampini and Spence (2004) studied the multisensory interactions between auditory, oral, tactile, mechanical, kinesthetic, and visual information in the rating of the perception of the “crispness” and “freshness” of potato chips (or crisps), to investigate whether the evaluation of the crispness and freshness of potato chips would be affected by only modifying the sounds produced during the biting action. In fact, the Pringles potato chips used in their experiment have all the same visual (i.e., shape) and oral–tactile (i.e., texture) aspects. The participants in this study had to make a single bite with their front teeth into a large number (180) of potato chips (otherwise known as crisps in the United Kingdom) with their mouth placed directly above the microphone and then to spit the crisp out (without swallowing) into a bowl placed on their lap. They then rated the crispness and freshness of each potato chip using a computer-based visual analog scale. The participants might hear the veridical sounds they made when biting into a crisp without any auditory frequency adjustment or with frequencies in the range 2–20 of the biting sounds amplified or attenuated by 12 dB. Furthermore, for each frequency manipulation, there was an attenuation of the overall volume of 0 (i.e., no attenuation), 20, or 40 dB. The results showed that the perception of both crispness and freshness were affected by the modulation of the auditory cues produced during the biting action. In particular, the potato chips were perceived as being both crisper and fresher when either the overall sound level was increased, or when just the high frequency sounds (in the range of 2–20 kHz) were selectively amplified (see Figure 37.2).

FIGURE 37.2. (a) Schematic view of apparatus and participant in Zampini et al.

FIGURE 37.2

(a) Schematic view of apparatus and participant in Zampini et al.’s (2004) study. Door of experimental booth was closed during the experiment and response scale was viewed through the window in left-hand side wall of booth. Mean responses for (more...)

Given that the crisps in Zampini and Spence’s (2004) study were very similar to each other in terms of their visual, tactile, and flavor attributes, the only perceptual aspect that varied during the task was the sound (which, of course, also contributes to flavor). Therefore, participants may have “felt” that the crisps had a different texture only guided by the sound since the other senses always received the same information. Additional evidence highlighting the powerful effect of auditory cues on the overall perception of the crisps was that the majority of the participants (15 out of 20) stated anecdotally on debriefing after the experiment that they believed the crisps to have been selected from different packages. Additionally, the majority of the participants also reported that the auditory information had been more salient than the oral tactile information, and this may also help to account for the effects reported by Zampini and Spence. In fact, one of the fundamental laws of multisensory integration that has emerged over the past few decades states that the sense that provides the more reliable (or salient) information is the one that dominates, or modulates, perception in another sensory modality (e.g., Ernst and Banks 2002; Shimojo and Shams 2001; Welch and Warren 1980). However, the sensory dominance effect can be explained by the fact that the human brain might rely on the most attended senses (Spence and Shankar 2010). The role of attention in the multisensory influence of auditory information on food perception is consistent with the results of a study in which the participants had to try and detect weak solutions of sucrose or citric acid in a mixture (Marks and Wheeler 1998). Participants were more accurate at detecting the tastant they were attending to than for the tastant they were not attending to (see also Ashkenazi and Marks 2004). Marks and Wheeler suggested that our ability to detect a particular sensory quality (e.g., tastant or flavor) may be modulated by selective attention toward (or away from) that quality. Therefore, in a similar vein, one might suggest that the effect found in crispness perception by increasing the overall loudness of the sounds produced when biting into crisps can change a participant’s perception by making the sounds more pronounced than would have been the case if this information had been derived solely from the texture in the mouth or from normal-level auditory cues. That is, participants’ attention would be directed toward this feature of the food by externally changing the relative weighting of the sensory cues that signify this. Louder sounds are also presumably more likely to capture a person’s attention than quieter sounds. However, at present, it is unclear how many of the findings taken to support an attentional account of any sensory dominance effect can, in fact, be better accounted for in terms of sensory estimates of stimulus attributes simply being more accurate (i.e., less variable) in the dominant modality than those in the other modalities (e.g., Alais and Burr 2004; Battaglia et al. 2003; Ernst and Banks 2002). Finally, it is important to note that these explanations are not mutually exclusive. For example, Zampini and Spence’s (2004) results can be accounted for either in terms of attentional capture or in terms of multisensory integration.

37.3.2. Effect of Auditory Cues on the Perception of Sparkling Water

In a follow-up study, Zampini and Spence (2005) studied the possible influence of auditory cues in the perception and evaluation of carbonation of water. Our perception of the carbonation of a beverage often relies on the integration of a variety of multisensory cues from visual, oral–somatosensory, nociceptive, auditory, and even tactile cues that are provided by the bubbles (e.g., Chandrashekar et al. 2009; Vickers 1991; Yau and McDaniel 1992). Zampini and Spence (2005) examined the relationship between the auditory cues produced by sparkling water and its perceived level of carbonation both when carbonated water samples were assessed in a cup and when they were assessed in the mouth. The carbonation sounds were modified adopting the same experimental paradigm developed in their previous research on the perception of potato chips (Zampini and Spence 2004). The sparkling water samples held in participants’ hands were judged to be more carbonated when the overall sound level was increased and/or when the high-frequency components (2–20 kHz) of the water sound were amplified. Interestingly, however, a subsequent experiment failed to demonstrate any effect of these auditory manipulations on the perception of carbonation and oral irritation from water samples that were held in the mouth. Taken together, these results therefore show that auditory cues can modulate the perception of the carbonation of a water sample held in the hand, but cannot modulate people’s perception of a water sample held in the mouth. This might be because the perception of carbonation in the mouth is more dependent on oral–somatosensory and/or nociceptive inputs than on auditory cues, or alternatively, that it is more important that we correctly perceive stimuli once they have entered the oral cavity (see Koza et al. 2005). Once again, these findings are consistent with the hypothesis that the modality dominating multisensory perception (when the senses are put into conflict) is the most accurate and/or informative sense (e.g., see Ernst and Banks 2002).

37.4. CONCLUSIONS

The past few years have seen a rapid growth of interest in the multisensory aspects of food perception (see Auvray and Spence 2008; Delwiche 2004; Prescott 1999, 2004; Stevenson 2009; Stevenson and Tomiczek 2007; Stillman 2002; Verhagen and Engelen 2006, for reviews). The research reviewed here highlights the profound effect that visual (i.e., color of food) and auditory cues (i.e., variations in the overall sound level and variations in the spectral distribution of energy) can have on people’s perception foodstuffs (such as potato chips and beverages). When people are asked to identify the flavors of food and beverages, their responses can be influenced by the colors of those food and beverages. In particular, the identification of specific flavors has often been shown to be less accurate when they are paired with an inappropriate color (e.g., DuBose et al. 1980; Zampini et al. 2007, 2008). Our perception of the flavor and physical characteristics of food and beverages can also be modulated by auditory cues. For instance, it is possible to change the perceived crispness of crisps or the perceived fizziness of a carbonated beverage (such as sparkling water) simply by modifying the sounds produced when eating the crisps or by the bubbles of the sparkling water (Zampini et al. 2004, 2005).

It is important to note that visual and auditory information are available at different stages of eating. Typically, visual (not to mention orthonasal olfactory and, on occasion, auditory) cues are available long before our ingestion of food (and before any other sensory cues associated with the food are available). Therefore, visual cues (e.g., food colors) might be expected to create an expectancy concerning the possible flavor of the food to be eaten (Hutchings 1977; Shankar et al. 2010). By contrast, any role of expectancy might be reduced when thinking at the potential influence of auditory cues on the perception of food. In fact, the sounds produced when biting into or chewing food are available at the moment of consumption. Therefore, it is possible to hypothesize that the role of multisensory integration is somewhat different when looking at the role of visual and auditory cues on the overall food perception. Given that visual cues are typically available long before a food is consumed and outside the mouth, it is quite unlikely that visual–flavor interactions are modulated by the spatial and temporal rules (i.e., greater multisensory interaction with spatial and temporal coincidence between the stimuli; see Calvert et al. 2004, for a review). Therefore, visual influences on multisensory flavor perception are better explained by looking at the role of expectancy than at the role of the spatial and temporal rules, which might help us to understand the role of auditory cues on food perception instead. However, some sounds might produce an expectancy effect as well. For example, sound of the food package being opened will normally precede the consumption of a particular packaged food item (think only of the rattling of the crisps packet). Several researchers have demonstrated that people’s expectations regarding what they are about to consume can also have a significant effect on their perception of pleasantness of the food or drink itself (see Spence et al., in press, for a recent review). It is also important to note that the visual and auditory contribution to multisensory flavor perception typically takes place without people necessarily being consciously aware that what they are seeing or hearing is influencing their overall flavor experience (e.g., Zampini et al. 2004, 2005). In Zampini et al.’s more recent research (e.g., Zampini et al. 2007, 2008), the participants were influenced by the inappropriate colors of the beverages that they were evaluating even though they had been informed beforehand that there might be a lack of congruency between the colors that they saw and the flavors that they were tasting. This shows, therefore, that the effect was powerful enough to override participants’ awareness that color information might mislead their identification of the flavors. The potential role of the sounds made when eating food on food perception is often ignored by people. For example, most of the participants in Zampini et al.’s (2004) study thought that the crisps were actually different (i.e., sorted from different packages or with different level of freshness and, therefore, of crispness). They seem to ignore the fact that the experimenters changed only the sounds produced when biting into the crisps and the crisps were not different. Nevertheless, the study reported here are consistent with a growing number of neurophysiological and electrophysiological studies demonstrating close visual–flavor (Osterbauer et al. 2005; Small 2004; Small and Prescott 2006; Verhagen and Engelen 2006) and audiotacile (Gobbelé et al. 2003; Kitagawa and Spence 2006; Levänen et al. 1998; Schroeder et al. 2001; von Békésy 1957) * interactions at the neuronal level. Results such as these therefore help to emphasize the limitations that may be associated with relying solely on introspection and verbal report (as is often the case in commercial consumer testing settings) when trying to measure people’s perception and evaluation of foodstuffs.

REFERENCES

  1. Alais D, Burr D. The ventriloquist effect results from near-optimal bimodal integration. Current Biology. 2004;14:257–262. [PubMed: 14761661]
  2. Alley R.L, Alley T.R. The influence of physical state and color on perceived sweetness. Journal of Psychology: Interdisciplinary and Applied. 1998;132:561–568. [PubMed: 9729847]
  3. Ashkenazi A, Marks L.E. Effect of endogenous attention on detection of weak gustatory and olfactory flavors. Perception & Psychophysics. 2004;66:596–608. [PubMed: 15311659]
  4. Auvray M, Spence C. The multisensory perception of flavor. Consciousness & Cognition. 2008;17:1016–1031. [PubMed: 17689100]
  5. Bartoshuk L.M, Duffy V.B, Miller I.J. PTC/PROP tasting: Anatomy, psychophysics, and sex effects. Physiology & Behavior. 1994;56:1165–1171. [PubMed: 7878086]
  6. Bartoshuk L.M, Fast K, Karrer T.A, Marino S, Price R.A, Reed D.A. PROP supertasters and the perception of sweetness and bitterness. Chemical Senses. 1992;17::594..
  7. Battaglia P.W, Jacobs R.A, Aslin R.N. Bayesian integration of visual and auditory signals for spatial localization. Journal of the Optical Society of America A. 2003;20:1391–1397. [PubMed: 12868643]
  8. Bertelson P, Aschersleben G. Automatic visual bias of perceived auditory location. Psychonomic Bulletin & Review. 1998;5:482–489.
  9. Blackwell L. Visual clues and their effects on odour assessment. Nutrition and Food Science. 1995;5:24–28.
  10. Bourne M.C. Food texture and viscosity. New York: Academic Press; 1982.
  11. Bufe B, Breslin P.A, Kuhn C. et al. The molecular basis of individual differences in phenylthiocarbamide and propylthiouracil bitterness perception. Current Biology. 2005;15:322–327. [PMC free article: PMC1400547] [PubMed: 15723792]
  12. Bult J.H.F, de Wijk R.A, Hummel T. Investigations on multimodal sensory integration: Texture, taste, and ortho- and retronasal olfactory stimuli in concert. Neuroscience Letters. 2007;411:6–10. [PubMed: 17110032]
  13. Cain W.S. History of research on smell. In: Carterette E.C, Friedman M.P, editors. In Handbook of perception: Vol. 6a: Tasting and smelling. New York: Academic Press; 1977. pp. 197–229.
  14. Calvert G, Spence C, Stein B.E. The handbook of multisensory processing. Cambridge, MA: MIT Press; 2004.
  15. Cardello A.V. Consumer expectations and their role in food acceptance. In: MacFie H.J.H, Thomson D.M.H, editors. In Measurement of food preferences. London: Blackie Academic & Professional; 1994. pp. 253–297.
  16. Chan M.M, Kane-Martinelli C. The effect of color on perceived flavour intensity and acceptance of foods by young adults and elderly adults. Journal of the American Dietetic Association. 1997;97:657–659. [PubMed: 9183329]
  17. Chandrashekar J, Yarmolinsky D, von Buchholtz L. et al. The taste of carbonation. Science. 2009;326:443–445. [PMC free article: PMC3654389] [PubMed: 19833970]
  18. Chen H, Karlsson C, Povey M. Acoustic envelope detector for crispness assessment of biscuits. Journal of Texture Studies. 2005;36:139–156.
  19. Christensen C.M. Effects of taste quality and intensity on oral perception of viscosity. Perception & Psychophysics. 1980a;28:315–320. [PubMed: 7465314]
  20. Christensen C.M. Effects of solution viscosity on perceived saltiness and sweetness. Perception & Psychophysics. 1980b;28:347–353. [PubMed: 7465318]
  21. Christensen C.M, Vickers Z.M. Relationship of chewing sounds to judgments of food crispness. Journal of Food Science. 1981;46:574–578.
  22. Clydesdale F.M. Color as a factor in food choice. Critical Reviews in Food Science and Nutrition. 1993;33:83–101. [PubMed: 8424857]
  23. Dalton P, Doolittle N, Nagata H, Breslin P.A.S. The merging of the senses: Integration of subthreshold taste and smell. Nature Neuroscience. 2000;3:431–432. [PubMed: 10769380]
  24. Davis R. The role of nonolfactory context cues in odor identification. Perception & Psychophysics. 1981;30:83–89. [PubMed: 7290899]
  25. De Araujo I.E.T, Rolls E.T, Kringelbach M.L, GloneMc F, Phillips N. Taste-olfactory convergence, and the representation of the pleasantness of flavour, in the human brain. European Journal of Neuroscience. 2003;18:2059–2068. [PubMed: 14622239]
  26. de Craen A.J.M, Roos P.J, de Vries A.L, Kleijnen J. Effect of colour of drugs: Systematic review of perceived effect of drugs and their effectiveness. British Medical Journal. 1996;313:1624–1626. [PMC free article: PMC2359128] [PubMed: 8991013]
  27. Deliza R, MacFie H. The generation of sensory expectation by external cues and its effect on sensory perception and hedonic ratings: A review. Journal of Sensory Studies. 1996;11:103–128.
  28. Delwiche J. The impact of perceptual interactions on perceived flavour. Food Quality and Preference. 2004;15::137–146.
  29. Demattè M.L, Sanabria D, Spence C. Crossmodal associations and interactions between olfaction and vision. Chemical Senses. 2006;31:E50–E51.
  30. Demattè M.L, Sanabria D, Spence C. Olfactory identification: When vision matters? Chemical Senses. 2009;34:103–109. [PubMed: 18794200]
  31. Drake B.K. Food crunching sounds. An introductory study. Journal of Food Science. 1963;28:233–241.
  32. Drake B.K. Relationships of sounds and other vibrations to food acceptability. Proceedings of the 3rd International Congress of Food Science and Technology. 1970:437–445.
    Washington, DC: Aug 9-14,
  33. Drewnowski A. Genetics of human taste perception. In: Doty R.L, editor. In Human olfaction and gustation. 2nd. New York: Marcel Dekker, Inc; 2003. pp. 847–860.
  34. DuBose C.N, Cardello A.V, Maller O. Effects of colourants and flavourants on identification, perceived flavour intensity, and hedonic quality of fruit-flavoured beverages and cake. Journal of Food Science. 1980;45::1393–1399, 1415.
  35. Duffy V.B. Variation in oral sensation: Implications for diet and health. Current Opinion in Gastroenterology. 2007;23:171–177. [PubMed: 17268246]
  36. Duncker K. The influence of past experience upon perceptual properties. American Journal of Psychology. 1939;52::255–265.
  37. Engen T. The effect of expectation on judgments of odour. Acta Psychologica. 1972;36:450–458. [PubMed: 4679755]
  38. Ernst M.O, Banks M.S. Humans integrate visual and haptic information in a statistically optimal fashion. Nature. 2002;415:429–433. [PubMed: 11807554]
  39. Essick G.K, Chopra A, Guest S, McGlone F. Lingual tactile acuity, taste perception, and the density and diameter of fungiform papillae in female subjects. Physiology & Behavior. 2003;80:289–302. [PubMed: 14637228]
  40. Frank R.A, Ducheny K, Mize S.J.S. Strawberry odor, but not red color, enhances the sweetness of sucrose solutions. Chemical Senses. 1989;14:371–377.
  41. Garber L.L Jr., Hyatt E.M, Boya Ü.Ö. The mediating effects of the appearance of nondurable consumer goods and their packaging on consumer behavior. In: Schifferstein H.N.J, Hekkert P, editors. In Product experience. London: Elsevier; 2008. pp. 581–602.
  42. Garber L.L Jr., Hyatt E.M, Starr R.G Jr. The effects of food colour on perceived flavour. Journal of Marketing Theory and Practice. 2000;8:59–72.
  43. Garber L.L Jr., Hyatt E.M, Starr R.G Jr. Placing food color experimentation into a valid consumer context. Journal of Food Products Marketing. 2001;7:3–24.
  44. Gifford S.R, Clydesdale F.M. The psychophysical relationship between colour and sodium chloride concentrations in model systems. Journal of Food Protection. 1986;49:977–982.
  45. Gifford S.R, Clydesdale F.M, Damon R.A Jr. The psychophysical relationship between colour and salt concentration in chicken flavoured broths. Journal of Sensory Studies. 1987;2:137–147.
  46. Gobbelé R, Schürrmann M, Forss N, Juottonen K, Buchner H, Hari R. Activation of the human posterior and tempoparietal cortices during audiotactile interaction. Neuroimage. 2003;20:503–511. [PubMed: 14527610]
  47. Green B.G, Shaffer G.S, Gilmore M.M. Derivation and evaluation of a semantic scale of oral sensation with apparent ratio properties. Chemical Senses. 1993;18:683–702.
  48. Hollowood T.A, Linforth R.S.T, Taylor A.J. The effect of viscosity on the perception of flavour. Chemical Senses. 2002;27:583–591. [PubMed: 12200337]
  49. Hutchings J.B. The importance of visual appearance of foods to the food processor and the consumer. In: Birch G.G, Brennan J.G, Parker K.J, editors. In Sensory properties of foods. London: Applied Science Publishers; 1977. pp. 45–57.
  50. Johnson J.L, Clydesdale F.M. Perceived sweetness and redness in coloured sucrose solutions. Journal of Food Science. 1982;47:747–752.
  51. Johnson J.L, Dzendolet E, Clydesdale F.M. Psychophysical relationships between sweetness and redness in strawberry-drinks. Journal of Food Protection. 1983;46:21–25.
  52. Kim U.K, Breslin P.A, Reed D, Drayna D. Genetics of human taste perception. Journal of Dental Research. 2004;83:448–453. [PubMed: 15153450]
  53. Kitagawa N, Spence C. Audiotactile multisensory interactions in information processing. Japanese Psychological Research. 2006;48:158–173.
  54. Koshi E.B, Short C.A. Placebo theory and its implications for research and clinical practice: A review of the recent literature. Pain Practice. 2007;7:4–20. [PubMed: 17305673]
  55. Koza B, Cilmi A, Dolese M, Zellner D. Color enhances orthonasal olfactory intensity and reduces retronasal olfactory intensity. Chemical Senses. 2005;30:643–649. [PubMed: 16141290]
  56. Lavin J, Lawless H.T. Effects of colour and odor on judgments of sweetness among children and adults. Food Quality and Preference. 1998;9:283–289.
  57. Lawless H, Rozin P, Shenker J. Effects of oral capsaicin on gustatory, olfactory and irritant sensations and flavor identification in humans who regularly or rarely consume chili pepper. Chemical Senses. 1985;10:579–89.
  58. Lee L, Frederick S, Ariely D. Try it, you’ll like it. Psychological Science. 2006;17:1054–1058. [PubMed: 17201787]
  59. Levänen S, Jousmäki V, Hari R. Vibration-induced auditory-cortex activation in a congenitally deaf adult. Current Biology. 1998;8:869–872. [PubMed: 9705933]
  60. Levitan C.A, Zampini M, Li R, Spence C. Assessing the role of colour cues and people’s beliefs about colour-flavour associations on the discrimination of the flavour of sugar-coated chocolates. Chemical Senses. 2008;33:415–423. [PubMed: 18310060]
  61. Lucchelli P.E, Cattaneo A.D, Zattoni J. Effect of capsule colour and order of administration of hypnotic treatments. European Journal of Clinical Pharmacology. 1978;13:153–155. [PubMed: 26573]
  62. Maga J.A. Influence of colour on taste thresholds. Chemical Senses and Flavour. 1974;1:115–119.
  63. Marks L.E, Wheeler M.E. Attention and the detectability of weak-taste stimuli. Chemical Senses. 1998;23::19–29. [PubMed: 9530966]
  64. Masuda M, Yamaguchi Y, Arai K, Okajima K. Effect of auditory information on food recognition. IEICE Technical Report. 2008;108(356):123–126.
  65. Moir H.C. Some observations on the appreciation of flavour in food stuffs. Chemistry and Industry. 1936;55:145–148.
  66. Morrot G, Brochet F, Dubourdieu D. The colour of odors. Brain and Language. 2001;79:309–320. [PubMed: 11712849]
  67. Murphy C, Cain W.S. Taste and olfaction: Independence vs. interaction. Physiology and Behavior. 1980;24:601–605. [PubMed: 7375580]
  68. Oram N, Laing D.G, Hutchinson I. et al. The influence of flavour and colour on drink identification by children and adults. Developmental Psychobiology. 1995;28:239–246. [PubMed: 7621986]
  69. Osterbauer R.A, Matthews P.M, Jenkinson M, Beckmann C.F, Hansen P.C, Calvert G.A. Color of scents: Chromatic stimuli modulate odor responses in the human brain. Journal of Neurophysiology. 2005;93::3434–3441. [PubMed: 15689393]
  70. Pangborn R.M. Influence of colour on the discrimination of sweetness. American Journal of Psychology. 1960;73:229–238.
  71. Pangborn R.M, Hansen B. The influence of colour on discrimination of sweetness and sourness in pear-nectar. American Journal of Psychology. 1963;76:315–317.
  72. Philipsen D.H, Clydesdale F.M, Griffin R.W, Stern P. Consumer age affects response sensory characteristics of a cherry flavoured beverage. Journal of Food Science. 1995;60:364–368.
  73. Prescott J. Flavour as a psychological construct: Implications for perceiving and measuring the sensory qualities of foods. Food Quality and Preference. 1999;10:349–356.
  74. Prescott J. Psychological processes in flavour perception. In: Taylor A.J, Roberts D, editors. In Flavour perception. London: Blackwell Publishing; 2004. pp. 256–278.
  75. Prescott J, Ripandelli N, Wakeling I. Binary taste mixture interactions in PROP non-tasters, medium-tasters and super-tasters. Chemical Senses. 2001;26:993–1003. [PubMed: 11595676]
  76. Prescott J, Soo J, Campbell H, Roberts C. Response of PROP taster groups to variations in sensory qualities within foods and beverages. Chemical Senses. 2004;26:993–1003. [PubMed: 15276811]
  77. Reed D.R. Birth of a new breed of supertaster. Chemical Senses. 2008;33:489–491. [PMC free article: PMC3342591] [PubMed: 18562444]
  78. Rolls E.T. Smell, taste, texture, and temperature multimodal representations in the brain, and their relevance to the control of appetite. Nutrition Reiews. 2004;62::S193–S204. [PubMed: 15630935]
  79. Rolls E.T, Baylis L.L. Gustatory, olfactory, and visual convergence within the primate orbitofrontal cortex. Journal of Neuroscience. 1994;14:5437–5452. [PubMed: 8083747]
  80. Roth H.A, Radle L.J, Gifford S.R, Clydesdale F.M. Psychophysical relationships between perceived sweetness and colour in lemon- and lime-flavoured drinks. Journal of Food Science. 1988;53::1116–1119, 1162.
  81. Rozin P. “Taste-smell confusions” and the duality of the olfactory sense. Perception & Psychophysics. 1982;31::397–401. [PubMed: 7110896]
  82. Schroeder C.E, Lindsley R.W, Specht C, Marcovici A, Smiley J.F, Javitt D.C. Somatosensory input to auditory association cortex in the macaque monkey. Journal of Neurophysiology. 2001;85:1322–1327. [PubMed: 11248001]
  83. Seymour S.K, Hamann D.D. Crispness and crunchiness of selected low moisture foods. Journal of Texture Studies. 1988;19:79–95.
  84. Shankar M.U, Levitan C.A, Prescott J, Spence C. The influence of color and label information on flavor perception. Chemosensory Perception. 2009;2:53–58.
  85. Shankar M.U, Levitan C, Spence C. “Grape expectations”: Does higher level knowledge mediates the interpretation of multisensory cues? Consciousness & Cognition. 2010;19:380–390. [PubMed: 19828330]
  86. Shimojo S, Shams L. Sensory modalities are not separate modalities: Plasticity and interactions. Current Opinion in Neurobiology. 2001;11:505–509. [PubMed: 11502399]
  87. Skrandies W, Reuther N. Match and mismatch of taste, odor, and color is reflected by electrical activity in the human brain. Journal of Psychophysiology. 2008;22:175–184.
  88. Small D.M. Crossmodal integration—insights from the chemical senses. Trends in Neurosciences. 2004;27:120–123. [PubMed: 15046082]
  89. Small D.M, Gerber J.C, Mak Y.E, Hummel T. Differential neural responses evoked by orthonasal versus retronasal odorant perception in humans. Neuron. 2005;47:593–605. [PubMed: 16102541]
  90. Small D.M, Prescott J. Odor/taste integration and the perception of flavour. Experimental Brain Research. 2005;166:345–357. [PubMed: 16028032]
  91. Small D.M, Veldhuizen M.G, Felsted J, Mak Y.E, McGlone F. Separable substrates for anticipatory and consummatory food chemosensation. Neuron. 2008;57:786–797. [PMC free article: PMC2669434] [PubMed: 18341997]
  92. Spence C. The ICI report on the secret of the senses. London: The Communication Group; 2002.
  93. Spence C, Levitan C, Shankar M.U, Zampini M. Does food colour influence flavour identification in humans? Chemosensory Perception. 2010;3:68–84.
  94. Spence C, Shankar M.U. The influence of auditory cues on the perception of, and responses to, food and drink. Journal of Sensory Studies. 2010;25:406–430.
  95. Spence C, Shankar M.U, Blumenthal H. ‘Sound bites’: Auditory contributions to the perception and consumption of food and drink. In: Bacci F, Melcher D, editors. Art and the senses. Oxford: Oxford Univ. Press; In press. To appear in.
  96. Spence C, Zampini M. Auditory contributions to multisensory product perception. Acta Acustica united with Acustica. 2006;92:1009–1025.
  97. Stevenson R.J, Boakes R.A. Sweet and sour smells: Learned synesthesia between the senses of taste and smell. In: Calvert G.A, Spence C, Stein B.E, editors. In The handbook of multisensory processes. Cambridge, MA: MIT Press; 2004. pp. 69–83.
  98. Stevenson R.J, Boakes R.A, Prescott J. Changes in odor sweetness resulting from implicit learning of a simultaneous odor-sweetness association: An example of learned synesthesia. Learning and Motivation. 1998;29:113–132.
  99. Stevenson R.J. The psychology of flavour. Oxford: Oxford Univ. Press; 2009.
  100. Stevenson R.J, Oaten M. The effect of appropriate and inappropriate stimulus color on odor discrimination. Perception & Psychophysics. 2008;70:640–646. [PubMed: 18556925]
  101. Stevenson R.J, Tomiczek C. Olfactory-induced synesthesias: A review and model. Psychological Bulletin. 2007;133:294–309. [PubMed: 17338601]
  102. Stillman J. Colour influences flavour identification in fruit-flavoured beverages. Journal of Food Science. 1993;58::810–812.
  103. Stillman J.A. Gustation: Intersensory experience par excellence. Perception. 2002;31:1491–1500. [PubMed: 12916673]
  104. Strugnell C. Colour and its role in sweetness perception. Appetite. 1997;28::85.. [PubMed: 9134097]
  105. Tyle P. Effect of size, shape and hardness of particles in suspension on oral texture and palatability. Acta Psychologica. 1993;84:111–118. [PubMed: 8237451]
  106. Varela P, Chen J, Karlsson C, Povey M. Crispness assessment of roasted almonds by an integrated approach to texture description: Texture, acoustics, sensory and structure. Journal of Chemometrics. 2006;20:311–320.
  107. Verhagen J.V, Engelen L. The neurocognitive bases of human multimodal food perception: Sensory integration. Neuroscience and Biobehavioral Reviews. 2006;30:613–650. [PubMed: 16457886]
  108. Vickers Z.M. Crispness and crunchiness in foods. In: Sherman P, editor. In Food texture and rheology. London: Academic Press; 1979. pp. 145–166.
  109. Vickers Z.M. Relationships of chewing sounds to judgments of crispness, crunchiness and hardness. Journal of Food Science. 1981;47:121–124.
  110. Vickers Z.M. Pleasantness of food sounds. Journal of Food Science. 1983;48:783–786.
  111. Vickers Z.M. Crispness and crunchiness—A difference in pitch? Journal of Texture Studies. 1984;15:157–163.
  112. Vickers Z.M. Crispness and crunchiness—Textural attributes with auditory components. In: Moskowitz H.R, editor. In Food texture: Instrumental and sensory measurement. New York: Marcel Dekker; 1987. pp. 45–66.
  113. Vickers Z.M. Sound perception and food quality. Journal of Food Quality. 1991;14:87–96.
  114. Vickers Z.M, Bourne M.C. A psychoacoustical theory of crispness. Journal of Food Science. 1976;41:1158–1164.
  115. Vickers Z.M, Wasserman S.S. Sensory qualities of food sounds based on individual perceptions. Journal of Texture Studies. 1979;10:319–332.
  116. von Békésy G. Neural volleys and the similarity between some sensations produced by tones and by skin vibrations. Journal of the Acoustical Society of America. 1957;29:1059–1069.
  117. Welch R.B, Warren D.H. Immediate perceptual response to intersensory discrepancy. Psychological Bulletin. 1980;3:638–667. [PubMed: 7003641]
  118. Wheatley J. Putting colour into marketing. Marketing. 1973;67:24–29.
  119. White T.L, Prescott J. Chemosensory cross-modal Stroop effects: Congruent odors facilitate taste identification. Chemical Senses. 2007;32::337–341. [PubMed: 17307760]
  120. Yau N.J.N, Mc Daniel M.R. The effect of temperature on carbonation perception. Chemical Senses. 1992;14:337–348.
  121. Yeomans M, Chambers L, Blumenthal H, Blake A. The role of expectancy in sensory and hedonic evaluation: The case of smoked salmon ice-cream. Food Quality and Preference. 2008;19:565–573.
  122. Zampini M, Sanabria D, Phillips N, Spence C. The multisensory perception of flavour: Assessing the influence of colour cues on flavour discrimination responses. Food Quality & Preference. 2007;18:975–984.
  123. Zampini M, Spence C. The role of auditory cues in modulating the perceived crispness and staleness of potato chips. Journal of Sensory Studies. 2004;19:347–363.
  124. Zampini M, Spence C. Modifying the multisensory perception of a carbonated beverage using auditory cues. Food Quality and Preference. 2005;16:632–641.
  125. Zampini M, Wantling E, Phillips N, Spence C. Multisensory flavour perception: Assessing the influence of fruit acids and colour cues on the perception of fruit-flavoured beverages. Food Quality & Preference. 2008;18:335–343.
  126. Zellner D.A, Bartoli A.M, Eckard R. Influence of colour on odor identification and liking ratings. American Journal of Psychology. 1991;104:547–561. [PubMed: 1793126]
  127. Zellner D.A, Durlach P. Effect of colour on expected and experienced refreshment, intensity, and liking of beverages. American Journal of Psychology. 2003;116:633–647. [PubMed: 14723247]
  128. Zellner D.A, Kautz M.A. Colour affects perceived odor intensity. Journal of Experimental Psychology: Human Perception and Performance. 1990;16:391–397. [PubMed: 2142207]
  129. Zellner D.A, Whitten L.A. The effect of colour intensity and appropriateness on color-induced odor enhancement. American Journal of Psychology. 1999;112:585–604. [PubMed: 10696268]

Footnotes

*

For example, coffee and tea are indistinguishable (with both having a bitter taste) if drunk while holding one’s nose pinched shut. Whereas the taste of a lemon only actually consists of sour, sweet, and bitter components, most of the flavor we normally associate with the taste of a lemon actually comes from the terpene aroma, one of the constituent chemicals that stimulate the olfactory mucosa via the nasopharynx (i.e., retronasal olfaction). Odor molecules may reach the receptors in the olfactory epithelium (i.e., the area located in the rear of the nasal cavity) traveling inward from the anterior nares or through the posterior nares of the nasopharynx. Most typically, orthonasal olfaction occurs during respiratory inhalation or sniffing, whereas retronasal olfaction occurs during respiratory exhalation or after swallowing. People usually report experiencing odors as originating from the external world when perceived orthonasally, and as coming from the mouth when perceived retronasally (Rozin 1982). Importantly, the latest cognitive neuroscience evidence has highlighted the fact that somewhat different neural structures are used to process these two kinds of olfactory information (Small et al. 2005, 2008; see also Koza et al. 2005).

*

The individual differences in taste sensitivity most extensively studied are those for the bitterness intensity of PROP [and phenylthiocarbamide (PTC) in earlier work]. Supertasters, medium tasters, and nontasters rate the bitterness of PROP as very to intensely strong, moderate to strong, and weak, respectively. Research using taste solutions have identified other differences in the three taster groups (see Prescott et al. 2004). Different PROP taster groups reported different taste intensities and liking of other bitter, salty, sweet, and fat-containing substances. The three different PROP taster groups are known to possess corresponding genetic differences. In particular, studies of taste genetics have revealed the existence of multiple bitterness receptor genes (Kim et al. 2004; see also Bufe et al. 2005; Duffy 2007).

*

However, it should also be noted that the relatively small number of participants were tested in each category (i.e., four non-tasters, five medium tasters, and five supertasters), thus placing a caveat in terms of generalizing from Zampini et al.’s (2008) findings. In future studies, taster status should therefore be assessed with much larger sample sizes.

*

It is interesting to note that the participants in Levitan et al.’s (2008) study were able to maintain such inappropriate beliefs about differently colored Smarties tasting different, despite the objective evidence that people perceive no difference in their flavor, and the fact that they have presumably had extensive previous exposure to the fact that these colors provide no useful information in this foodstuff.

*

However, it is important to note that, to the best of our knowledge, no neuroimaging studies have as yet been conducted to investigate the role of auditory cues on multisensory food perception (cf. Spence and Zampini 2006; Verhagen and Engelen 2006).