Product Experience

PREFACE

‘With this collection of chapters,’ say Hendrik Schifferstein and Paul Hekkert, ‘we hope to lay down the basics for the field of product experience research’. With these words, Schifferstein and Hekkert set forth a daunting challenge, but a necessary and important endeavor. Although industrial design has long been concerned with products, the emphasis has been on the physical structures of the devices, the forms and shapes, the materials, and the manufacturing processes to be followed. These are all important, to be sure, but today the focus has shifted from objects to the experiences that result from interacting with them. To understand experience we need to go beyond shape and form, even beyond simple ergonomics. We need to understand how psychology, the social sciences, communication, and business shape a person’s experience. This book provides an important first step. Design is hot: It is the topic of the day. Moreover, design is now coming to be understood in the broader sense, not just in the styling that provides a pretty face to products. Design is now recognized as a process applicable to all phases of product development. Design is, moreover, a multidisciplinary endeavor, requiring knowledge of a wide range of topics. The result is that the field of product design is entering a renaissance period. It’s a new world. It’s a challenge. And, of course, challenges are opportunities. But where to get all this new knowledge? Product Experience provides a powerful start. Schifferstein and Hekkert have packed an incredible array of information into the 27 chapters of the book, introducing the reader to multiple perspectives of product experience. From the senses to shopping, the chapters cover a rich gamut of issues. Schifferstein and Hekkert set the stage well with their introduction that reviews the many chapters,

xix

xx

PREFACE

providing structure and guidance. Then comes the meat of the book, the 27 chapters by recognized authorities, followed by Schifferstein and Hekkert’s closing reflections, neatly tying things together. The result encompasses many of the varied components and perspectives involved in understanding how products shape experience. Product Experience, the book, provides an essential reference work for students, instructors, and practicing designers. How do people experience products? What is the nature of the interaction, that dynamic interplay between the perceptions, actions, and understanding of the person experiencing the product? For the answers, start with this book. Don Norman1 Palo Alto, California Evanston, Illinois

1

Donald A. Norman. www.jnd.org [email protected] Nielsen Norman group and Northwestern University

LIST OF CONTRIBUTORS

The chapter(s) contributed to are indicated by the numbers in parentheses. Katja Battarbee, IDEO, Palo Alto, CA 94301, USA (19) Christina Bodin Danielsson, School of Architecture and Built Environment, The Royal Institute of Technology, 10044 Stockholm, Sweden (26) Stella Boess, Department of Industrial Design, Faculty of Industrial Design Engineering, Delft University of Technology, 2628 CE Delft, The Netherlands (12) Ünal Ö. Boya, Marketing Department, Appalachian State University, Boone NC 28608, USA (25) Reinhart Butter, Department of Industrial, Interior and Visual Communication Design, Ohio State University, Columbus, Ohio 43210, USA (14) Armand V. Cardello, Office of the Technical Director, US Army Natick Soldier Center, Natick, MA 01760-5020, USA (4) John M. Carroll, School of Information Sciences and Technology, The Pennsylvania State University, University Park PA 16802, USA (21) John Clarkson, Department of Engineering, University of Cambridge, Cambridge CB2 1PZ, United Kingdom (6) Gerald C. Cupchik, Department of Psychology, University of Toronto at Scarborough, Department of Life Sciences, Scarborough, ON, Canada M1C 1A4 (9) Michiel P. de Looze, TNO, 2130 AS Hoofddorp, The Netherlands (18) Pieter M.A. Desmet, Department of Industrial Design, Faculty of Industrial Design Engineering, Delft University of Technology, 2628 CE Delft, The Netherlands (15)

xxi

xxii

List of contributors

Jörgen Eklund, Department of Mechanical Engineering, Division of Quality and Human Systems Engineering, Linköping University, SE-58183 Linköping, Sweden (20) Ann Marie Fiore, Department of Textiles and Clothing, Iowa State University, Ames IA 50011, USA (27) Lawrence L. Garber, Department of Business Administration, Elon University, Elon, NC 27244, USA (25) Marc Hassenzahl, Economic Psychology and Human–Computer Interaction, University of Koblenz-Landau, Universität Campus Landau, 76829 Landau, Germany (11) Paul Hekkert, Department of Industrial Design, Faculty of Industrial Design Engineering, Delft University of Technology, 2628 CE Delft, The Netherlands (10) Michelle C. Hilscher, Department of Psychology, University of Toronto at Scarborough, Department of Life Sciences, Scarborough, ON, Canada M1C 1A4 (9) Eva M. Hyatt, Marketing Department, Appalachian State University, Boone NC 28608, USA (25) Shigekazu Ishihara, Department of Kansei Ergonomics, Hiroshima International University, Hiroshima, Japan (20) Jeroen Jansz, Amsterdam School of Communications Research (ASCoR), University of Amsterdam, 1012 CX Amsterdam, The Netherlands (23) Heimrich Kanis, Department of Industrial Design, Faculty of Industrial Design Engineering, Delft University of Technology, 2628 CE Delft, The Netherlands (12) David V. Keyson, Department of Industrial Design, Faculty of Industrial Design Ingeneering, Delft University of Technology, 2628 CE Delft, The Netherlands (22) Ilpo Koskinen, University of Art and Design Helsinki, 00560 Helsinki, Finland (19) Klaus Krippendorff, The Annenberg School for Communication, University of Pennsylvania, Philadelphia, PA 19104-6220, USA (14) Helmut Leder, Faculty of Psychology, University of Vienna, 1010 Wien, Austria (10) Herbert L. Meiselman, US Army Natick Soldier Center, Natick MA 01760-5020, USA, (24) Helena M. Mentis, School of Information Sciences and Technology, The Pennsylvania State University, University Park PA 16802, USA (21) Ruth Mugge, Department of Product Innovation Management, Faculty of Industrial Design Engineering, Delft University of Technology, 2628 CE Delft, The Netherlands (17) Mitsuo Nagamachi, User Science Institute, Kyushu University, Fukuoka 815-8540 Japan (20) Harold T Nefs, School of Psychology, University of St. Andrews, St. Mary’s College, St. Andrews, KY16 9JP, United Kingdom (1) Marsha L. Richins, College of Business, University of Missouri, Columbia, MO 65211, USA (16) Axel Roesler, Division of Design, University of Washington, Seattle, WA 98195, USA (7, 8)

List of contributors

xxiii

Hendrik N.J. Schifferstein, Department of Industrial Design, Faculty of Industrial Design Engineering, Delft University of Technology, 2628 CE Delft, The Netherlands (2, 5, 17) Jan P.L. Schoormans, Department of Product Innovation Management, Faculty of Industrial Design Engineering, Delft University of Technology, 2628 CE Delft, The Netherlands (17) Simon Schütte, Department of Mechanical Engineering, Division of Quality and Human Systems Engineering, Linköping University, SE-58183 Linköping, Sweden (20) Marieke H. Sonneveld, Department of Industrial Design, Faculty of Industrial Design Engineering, Delft University of Technology, 2628 CE Delft, The Netherlands (2) Charles Spence, Department of Experimental Psychology, University of Oxford, Oxford, OX1 3UD, United Kingdom (5) Ed S. Tan, Amsterdam School of Communications Research (ASCoR), University of Amsterdam, 1012 CX Amsterdam, The Netherlands (23) René van Egmond, Department of Industrial Design Faculty of Industrial Design Engineering, Delft University of Technology, 2628 CE Delft, The Netherlands (3) Thomas J.L. van Rompay, Department of Marketing Communication and Consumer Psychology, Faculty of Behavioral Sciences, Twente University 7500 AE Enschede, the Netherlands (13) Peter Vink, TNO, 2130 AS Hoofddorp, the Netherlands (18) Paul M. Wise, Monell Chemical Senses Center, Philadelphia, PA 19104-3308, USA (4) David Woods, Institute for Ergonomics, Ohio State University, 210 Baker Systems, Columbus OH 43210, USA (7, 8)

INTRODUCING PRODUCT EXPERIENCE PAUL HEKKERT AND HENDRIK N.J. SCHIFFERSTEIN

People live in a world in which they are surrounded by designed artifacts and services, products that were created by (other) people to serve some purpose: To get from one place to another; to clean the house; to cook; to feed and protect oneself; to contact someone; to have fun; to retrieve information; and so on. As a result, we have cars and bicycles, tissues and vacuum cleaners, shopping malls and living rooms, mobile phones and weblogs, computer games and festivals, archive systems and SMS messaging, and to use these products we need to interact with them. Notwithstanding the fact that the way in which people interact with a product is clearly product-dependent, they always use their senses to perceive it, they use their motor system and their knowledge to operate or communicate with it, and during the interaction they process the information they perceive, they may experience one or more emotions, and they are likely to form an affective evaluation of the product. Thus, although the interaction may be product-specific, the processes that are activated during the interaction are similar over products. As a consequence, it should be possible to develop an overall theoretical framework that guides the study of how people experience products. In the current book, we bring together existing work from several different research areas, and make a first attempt to integrate this work, as the starting point for the development of an overall, encompassing framework of product experience.

Theoretical views on product experience Before we can develop a conceptual framework, we first need to define the concept that we are interested in. In this book, we define the field of ‘product experience’ as the research area that develops an understanding of people’s subjective experiences that result from interacting with products. By focusing on products we restrict ourselves to physical objects or non-physical designs that have a utilitarian function, thereby excluding works of art and other non-utilitarian artifacts. Building on our previous definitions Product Experience Copyright © 2008 Elsevier Ltd.

1

2

Product Experience

(Hekkert, 2006; Schifferstein and Cleiren, 2005), we define the subjective product experience as the awareness of the psychological effects elicited by the interaction with a product, including the degree to which all our senses are stimulated, the meanings and values we attach to the product, and the feelings and emotions that are elicited. It can be debated whether these psychological consequences are always affective in nature. According to Russell (2003), core affect – the combination of pleasure and arousal – is, to varying degrees, ‘involved in most psychological events’ [p. 148, emphasis added]. Not all authors in this book embrace our definition of product experience. Some hold a different view and explain in their chapters where their definition originates from, and why they think it fits their research purposes better. These different perspectives from various fields of scientific endeavor add to the richness of our understanding of the concept. Our definition of product experience is quite broad compared to the definitions adopted by other authors and, thereby, generally encompasses their definitions. Therefore, in our view, all these authors investigate some aspects or antecedents of ‘product experience’, even though their reference frame may be narrower than ours. For example, Carroll and Mentis (Chapter 21) give an overview of how research on user experiences that emerged in the field of human–computer interaction yielded new insights that enriched research on product usability in the area of human factors. Battarbee and Koskinen (Chapter 19) use a designer’s perspective on product experience and stress the importance of social interaction in the formation of experiences. Cupchik and Hilscher (Chapter 9) explicitly limit the term ‘experience’ to special life events. They state that ‘experience refers to uniquely meaningful life events with both cognitive and affective qualities’ [emphasis added]. The contrast between Cupchik and Hilscher’s definition and ours becomes explicit in the two ways in which the term ‘experience’ can be translated into German or Dutch. All experiences, including very common, day-to-day experiences, are referred to with the German word ‘Erfahrung’ (Dutch: ‘ervaring’), whereas experiences of special, memorable events are better captured by the word ‘Erlebnis’ (Dutch: ‘belevenis’). The Erlebnis definition is closest to the views found in business literature on the Experience Economy (e.g. Pine and Gilmore, 1998). According to Pine and Gilmore, ‘an experience occurs when a company intentionally uses services as the stage, and goods as props, to engage individual consumers in a way that creates a memorable event’ (p. 98). Pine and Gilmore suggest that producers and retailers need to create special experiences for their customers, beyond simply offering good products. Their notion of experience thus resembles the uniquely meaningful events Cupchik and Hilscher refer to. Such experiences can occasionally be bound to individual products, but will more likely arise from designed spaces, installations, and attractions typically encountered in exhibitions and amusement parks. The design of these kinds of overwhelming and compelling events is often denoted as ‘experience design’. Designers and marketers may wonder whether it is always necessary to create an ‘Erlebnis’, or whether it will be sufficient to deliberately create, alter or improve the ‘Erfahrung’ that people have when they interact with products. According to Schmitt (1999), for example, the ultimate goal of experiential marketing is to create a desirable, coherent, and consistent customer impression that enhances the brand image. For companies it may be worthwhile to create possibilities for potential consumers to explore their products in a specific context, as part of a branding strategy. When people repeatedly encounter a particular brand within the context of a pleasant experience, they are more likely to develop a favourable attitude towards this brand. In addition, characteristics of the atmosphere during the experience (e.g. modern, fresh, impressive) may become associated with the brand.

3

Introducing product experience

The approaches sketched above, by Pine and Gilmore and Schmitt, were developed from a marketing perspective, in which the purchase and consumption of the product plays a central role. This deviates from the user perspective that is employed in most chapters of the current book. Many everyday experiences involve people who simply use and enjoy products. These everyday experiences are central to the book: The desire when seeing the new Apple iPod; the comfortable feel of a screwdriver in your hand; the awareness of the friendly whistle of a teapot; the frustration over the poor quality of an online help system; the delicious smell of a freshly baked apple-pie; the relief of smoothly parking your car in a narrow space. Understanding such experiences will allow designers and companies to ‘design for experience’.

Human–product interaction As argued at the beginning of this chapter, a product experience or ‘Erfahrung’ always results from some interaction the user has with a product. This interaction is not necessarily restricted to instrumental or non-instrumental (e.g. playing with your pen) physical action, but may also consist of passive (often visual) perception, or even remembering or thinking of a product (e.g. anticipating using your new stereo) (Desmet and Hekkert, 2007). Furthermore, the experience does not only result from the interaction, but also accompanies and guides the interaction, and thus affects the interaction. In sum, experience and interaction are fully intertwined and in order to explore people’s experiences of products, we need to thoroughly understand the constituents or building blocks of human–product interaction. Figure 1 provides a model of human–product interaction. In trying to obtain an overview of the literature covering the field of product experience, we got the impression that most scholars tend to approach human–product interaction from one of three possible perspectives. These three perspectives form the basic structure of the present book: The human beings with their systems and skills (Part I); the interaction itself with its different components (Part II); or a product (domain) with its specific properties (Part III). Independent from their surroundings and social context, humans are biologically equipped with a number of systems that make it possible for them to interact with their environment: A motor system to act upon the environment; sensory systems to perceive changes in the environment; and a cognitive system to make sense of the environment and to plan actions. Products are part of this environment. The motor capacities are needed to explore products, interact with them, and operate them. Sensory systems allow people to perceive a product and assess what kind of product it is. They provide feedback on people’s actions. Furthermore, they ‘tell’ a person whether a sensation (visual, auditory, tactual, olfactory, or gustatory) is pleasurable or should be avoided. Cognitive

HUMAN Motor system

PRODUCT

Motor skills

Sensory systems

Sensitivity

Cognitive system

Cognitive skills

Instincts

Concerns

INTERACTION

FIGURE 1 Model of human–product interaction.

Sensory properties

Structural properties

Possibilities for behavior

Composition

Functionality

Labels

Materials Technology

4

Product Experience

capacities link perceived information to stored knowledge to interpret the incoming information; they elicit memories of previous usage and evoke associations with other products. Finally, people are born with a limited set of instincts, which make them explore the world to try to satisfy basic needs. Through interaction with an environment, all these human capacities gradually develop into skills, expertise, and concerns (such as goals, intentions, and preferences). Skills and concerns can only be defined in relation to an outside world: We have skills in something, sensitivity for something, a preference for something, and our goals and intentions are directed towards something. The chapters in Part I of this book start out from one or more of these systems or skills, and try to explain experiential effects (and limitations) of human–product interaction on the basis of the specific qualities of these human capacities or characteristics. Analogously, a product perceived in isolation has (only) a number of structural or formal properties, such as its size, weight, and shape. Physical products are made of materials with specific technical characteristics (e.g. chemical constitution, heat conductivity, elasticity). Furthermore, products have a composition relating the various constituent parts to the whole. In addition, more and more products make use of embedded technology (electronics, sensors, and other digital components) to operate them. Finally, all kinds of labels (e.g. brand name, usage information, price information) may be attached to the product. It is, however, in the interaction with people that products obtain their meaning: On the basis of what is perceived sensorially (e.g. softness, freshness, loudness) products reveal cues of how to use them, and they reveal their function. Perceived product properties will only be of interest to an individual if they are somehow instrumental in fulfilling needs; only in relation to people we can determine what behaviour a product allows for, and what its primary or secondary functions might be. Studies that start out from the product (Part III) typically focus on one particular (set of) products and on how their product- or domain-specific characteristics affect people’s experiences. Often they include case studies of, for instance, how people experience household equipment, office interiors, or computer games. Product experiences depend on the way in which a person interacts with a product. Although phenomenologically experienced as a whole, at least three major components can be distinguished in product experiences (Desmet and Hekkert, 2007; Hekkert, 2006). The aesthetic response is characterized by feelings of (dis)pleasure that are based on the sensory perception of the object; the object looks beautiful, feels pleasurable, or sounds nice. In addition, people try to understand how a product must be operated or which actions it affords, and people attribute all kinds of expressive, semantic, symbolic, or other connotative meanings to it. The interactions with a product can help a person to reach a goal or can obstruct him or her in attaining that goal, and thereby lead to various emotional responses. Together these components shape the overall product experience. The chapters in Part II focus on one of these components of the experience, or they study a specific experiential state typified by a term like (dis)comfort, engagement, quality, or attachment. Finally, the way an interaction unfolds depends on the context in which this interaction takes place. This context can vary from the physical circumstances literally surrounding the interaction, such as the lighting conditions under which a product is perceived or the qualities of the space in which a meal is consumed, to the activities or experiences that have preceded the interaction at stake, and to the broader cultural and social situation that determines how people interact with and experience products (see e.g. Chapters 14, 24, and 26). It is clear that driving the same car may be experienced

Introducing product experience

5

completely differently when on holiday, in a hurry for an important meeting, testing it in order to make a purchase decision, or with the love of your life sitting next to you.

Empirical approaches to studying product experiences Given that product experience is defined as the awareness of the psychological effects elicited by the interaction with a product, it is not surprising that most research in this area typically assesses people’s subjective reports of their experiences with products. In the majority of studies, products or product parts are varied or manipulated under naturalistic conditions, and the effect of the manipulation on the subjective reports is assessed. In experimental studies the manipulations are typically done in a more or less systematic way, in order to isolate various underlying factors. In case studies and design projects, on the other hand, the manipulations are determined by wishes, demands, and limitations given by the product and its usage context, company goals, and designer capabilities. These two approaches yield complementary outcomes that both contribute to our knowledge on product experiences. The subjective reports may consist of either qualitative (e.g. in-depth interviews, diaries) or quantitative (e.g. responses on rating scales, preference rank orders) data. The ensemble of approaches used in these studies assures the ecological validity of the outcomes. However, subjective responses can only be obtained through introspection, and cannot be assessed objectively. Therefore, in several chapters self-reports are related to more objective measures, such as physical characteristics or instrumental measures of products, measures of information processing or decision making, consumer choice data, or psychophysiological measures and brain imaging data (see Chapters 1, 3, 18 and 20). Adding more objective measures and analyzing the relationships with subjective measures enables an internal validity-check on the subjective reports.

An overview of contributing scientific disciplines Research on product experience is situated at the intersection of several scientific (sub)disciplines. Because product experience research focuses on subjective experiences of people, all product experience research falls within the discipline of psychology. However, due to its multi-faceted nature, experience research crosses borders between several sub-disciplines of psychology that are usually distinguished, such as psychology of perception, cognitive psychology, and psychology of emotion. Acknowledging these different sub-disciplines, many questions can be asked concerning the interaction between a person and a product. How do people use their senses in experiencing products? How do people understand how to use a product? Why are people attracted to some products and not to others? On what grounds do people perceive a product as smart, solid, stupid, or splendid? Which memories, associations, and emotions does a product evoke? Why do people develop a bond with a product? These are the kinds of questions that will be addressed in this book. Although the contents of some chapters are clearly linked to one of these subdisciplines of psychology (e.g. Chapters 1–5 to perception; Chapters 7, 8, and 12 to cognition; and Chapters 15 and 16 to emotion), other chapters build on all sub-disciplines, and cover an applied field of knowledge. Most of these applied disciplines have their tradition in the social and behavioural sciences, such as psychological aesthetics, human factors, marketing, and consumer science. Others, however, have their roots in the technical sciences, such as mechanical and material engineering, and human–computer interaction (HCI). Analogously to how all these disciplines together contribute to and define

6

Product Experience

FIGURE 2 Disciplines contributing to the field of product experience.

the multi-disciplinary field of industrial or product design, they also converge in this relatively new domain of research: Product experience (see Figure 2). We now discuss each of these contributing disciplines briefly, and we indicate which (often new) areas or fields of study within each discipline ‘move’ into the emerging domain of product experience. Philosophers and psychologists have extensively studied people’s responses to works of art. From the field of philosophical aesthetics, the work of John Dewey has been most influential to the domain of product experience. In his ‘Art as Experience’, Dewey (1934) analyzes people’s involvement with artworks from a phenomenological perspective (see Chapters 9, 13, and 19). Ever since psychology was founded as a field of science, psychologists have had a keen interest in aesthetic experiences and evaluations of ‘objects’, such as simple patterns, faces, paintings, and landscapes (see e.g. Berlyne, 1971; Fechner, 1876). In their studies they generally looked for the principles governing people’s perception and appreciation of these manifestations by applying more general theories of perception, motivation, cognition, and emotion. More recently, psychologists have discovered products as an interesting subject area to investigate these aesthetic or pleasure principles (e.g. Crozier, 1994). Most of this work fits perfectly in the domain of product experience (see Chapters 10 and 11). The discipline of ergonomics or human factors traditionally focuses on the usability of products (in itself already an experiential goal). For a long time, the discipline limited itself to the perceptual and cognitive processes involved in product understanding, and to the physical or motor skills and processes enabling (or limiting) product use. In the current book, the perceptual systems and the way they operate in interaction with products are covered in Chapters 1–6, while the cognitive abilities and skills and their

Introducing product experience

7

effects on the interaction with products are covered in Chapters 6–8; human motor abilities and skills affecting interaction are predominantly discussed in Chapter 6. One of the ways in which products can be made easy to use is by making it self-evident how a product should be operated (see Chapter 12). However, this perspective has been expanded. At the end of the twentieth century the ergonomic discipline increasingly began to focus on other subjective experiences arising from the use of products, including research on satisfaction, pleasure (e.g. Jordan and Green, 2002), and comfort and convenience (Vink, 2005, see Chapter 18). Mechanical and material engineering have developed from having a singular focus on the technical/physical properties of artifacts and their effects on the durability, reliability, production, and (technical) performance of products, to studying, quantifying, and modeling the relationship between such properties and sensorial and other subjective responses in terms of meaning and aesthetics (e.g. Ashby and Johnson, 2002). This movement is most prominently seen in Kansei Engineering, a branch of engineering that started in Japan in the 1970s (see Chapter 20 for an extensive account of this approach). Technology-driven research focuses on how products can be created with new technologies that may be beneficial to potential users. The majority of this research consists of studies on the application of digital or smart technologies in human–computer interaction. Designers are interested in exploring new functionalities and interaction possibilities that can be created with these new technologies (see e.g. Moggridge, 2007). Because these technologies are applied in more and more everyday products, the findings obtained in the field of human–computer interaction become increasingly relevant for the entire product spectrum. In addition, within the HCI field we also see a shift from usability research to user experience research, variously looking at experiences such as presence, fun, trust, or engagement (e.g. Blythe, Overbeeke, Monk, and Wright, 2003). Various issues of user-product interaction with respect to digital technology are discussed in Chapters 7 and 8. In addition, the experience of functional interfaces (Chapter 21), intelligent products (Chapter 22), and computer games (Chapter 23) is extensively treated in Part III of this volume. The field of marketing studies how products find their way to customers. Traditionally, the marketer can make use of the four instruments in the marketing mix to bring the product to market in a profitable way: product, price, promotion, and distribution (Kotler, 1984). Research on product experience will typically be focused on the subjective evaluation of a physical product or service. This may concern a first encounter with the product in a store (Chapters 25 and 27), or a repeated encounter during product usage (Chapters 16 and 17). Also in the field of consumer research, research attention has shifted from information processing approaches with a focus on utilitarian value and price, to the emotional experiences associated with product consumption (e.g. Holbrook and Hirschman, 1982). In summary, the field of product experience research encompasses research from each of these disciplines. However, to fully understand human product experience, we need to use approaches that allow us to build bridges between these various fields of expertise. In our final ‘reflections’ we will further elaborate this view on the future of the research domain. People’s experiences with products are by definition subjective; probably no two experiences are alike. Individual differences in terms of, for instance, gender, age, expertise, and (cultural) background may account for the variety in people’s experiences, and some of these differences are discussed in this book. Finally, this book is not only about understanding product experience: We want to understand this phenomenon in order to contribute to ‘design for experience’. Hence, many authors address the design implications of their theories and findings, and they provide directions or guidelines to

8

Product Experience

increase the likelihood that a particular designed interaction will lead to the intended experience. Given that there are many scientific areas relevant for the study of product experiences, it is no surprise that the authors who contributed to this book differ considerably with respect to their scientific background and expertise. To preserve the coherence in the book, we have stimulated all authors to link their views and research to other related areas, to combine theoretical overviews with empirical studies, to provide concise descriptions of the methods they used, and to describe how their results may be used in design practice. It has proven to be a hard task for many authors to leave the field of expertise in which they feel comfortable, to explore largely unknown, but nonetheless fascinating, neighboring areas. This additional effort has made this book a unique collection of chapters that try to build bridges, for example, from neurophysiology and traditional psychophysics to product semantics, from universal aesthetic principles to design applications, and from human physical capabilities to product and consumption emotions. With this collection of chapters we hope to lay down the basics for the field of product experience research.

REFERENCES Ashby, M. and Johnson, K. (2002). Materials and design: the art and science of material selection in product design. Oxford, UK: Butterworth-Heinemann. Berlyne, D. E. (1971). Aesthetics and psychobiology. New York: Appleton-Century-Crofts. Blythe, M. A., Overbeeke, K., Monk, A. F. and Wright, P. C. (Eds.) (2003). Funology: from usability to enjoyment. Dordrecht: Kluwer Academic Publishers. Crozier, W. R. (1994). Manufactured pleasures: psychological responses to design. Manchester: Manchester University Press. Desmet, P. M. A. and Hekkert, P. (2007). Framework of product experience. International Journal of Design, 1, 57–66. Dewey, J. (1934). Art as experience. New York: Berkley Publishing Group. Fechner, G. T. (1876). Vorschule der Ästhetik. Leipzig: Breitkopf und Härtel. Hekkert, P. (2006). Design aesthetics: principles of pleasure in design. Psychology Science, 48, 157–172. Holbrook, M. B. and Hirschman, E. C. (1982). The experiential aspects of consumption: consumer fantasies, feelings, and fun. Journal of Consumer Research, 9, 132–140. Jordan, P. W. and Green, W. S. (2002). Pleasure with products: beyond usability. London: Taylor and Francis. Kotler, P. (1984). Marketing management: analysis, planning and control. London: Prentice-Hall. Moggridge, B. (2007). Designing interactions. Cambridge, MA: MIT Press. Pine, B. J. and Gilmore, J. H. (1998). Welcome to the experience economy. Harvard Business Review, 76(July/ August), 97–105. Russell, J. A. (2003). Core affect and the psychological construction of emotion. Psychological Review, 110, 145–172. Schifferstein, H. N. J. and Cleiren, M. P. H. D. (2005). Capturing product experiences: a split-modality approach. Acta Psychologica, 118, 293–318. Schmitt, B. (1999). Experiental marketing. Journal of Marketing Management, 15, 53–67. Vink, P. (2005). Comfort and design – Principles and good practice. Boca Raton: CRC Press.

1

ON THE VISUAL APPEARANCE OF OBJECTS HAROLD T. NEFS University of St. Andrews, Scotland, UK

1. ON VISUAL APPEARANCE 1.1. Introduction When looking around in the world, humans have to figure out what it is they see, what the shapes of things are, what kind of materials they are made of, and what their potential uses or dangers are. In addition to meaning, aesthetic value or affordance, the visual system also extracts some more fundamental object properties. Examples include the shape, size, glossiness, and lightness of an object. Counter-intuitively maybe, these aspects of the percept are not objective qualities because they are constructed in and by the human mind. Some of them have obvious correlates in the physical world, but this is often not the case. For example, objects might look flatter when they are made from a different material (e.g. Khang, Koenderink and Kappers, 2003; 2004). The position, that perceived object shape is determined exclusively by the objects’ physical shape, is therefore clearly untenable. Likewise, perceived material and perceived illumination characteristics are also not determined exclusively by their respective physical counterparts. The problem of what objects look like is certainly not a trivial problem. It determines whether people approach an object carefully or not, if and how they pick it up, etc. In this chapter we examine the difficulties that humans face when looking at the real world. In particular we will examine the perceived shape, material properties, and illumination of objects with special emphasis on the perceived shape. A question that has inspired and frustrated many scholars over the last few millennia is why objects look the way they do. In the ancient Greek tradition it was thought Product Experience Copyright © 2008 Elsevier Ltd.

11

12

Product Experience

that objects send out exact little copies of themselves to the eyes where they are mysteriously inspected by the mind (Howard, 2002). Aristotle, however, rejected the idea that objects emitted substance to the eyes, and he clearly had the idea of the rectilinear propagation of light. Although this seems quite obvious from our modern vantage point, for a very long time it was not properly understood how image formation in the eye was accomplished. In our modern understanding we know that an image of the world is projected onto the back of the eyes where it is further processed and transported to the brain through the optic nerve. The visual image is a two-dimensional projection of the threedimensional world on the retina. In spite of the fact that the depth dimension is lost in the process, humans report a clear impression of a three-dimensional world. Somehow the brain reconstructs the world from what is only available in a degenerate two-dimensional projection. The main problem for vision lies in the fact that for any visual image there are an infinite number of possible objects that give rise to the same image. This ambiguity not only lies in the fact that the depth dimension is lost in the projection onto the twodimensional retina, but also in the materials that the object is made of and how the object is illuminated. At first sight, the reconstruction of the world in the mind is thus an impossible task. However, if the visual system were to use some assumptions about the world, and constraints in the way it reconstructs the world, the task becomes much more manageable. Some of the assumptions that one can make about the world are very powerful in constraining the number of possible solutions, while still maintaining a high predictive power in the physical world. For example, it is reasonable to assume that what is smoothly connected in the visual image is also smoothly connected in physical space. It is a highly coincidental situation indeed that two unconnected surface patches still form a smoothly connected patch in the visual image. It is more likely that an area of constant luminance, or a smoothly changing area, corresponds to a single, smoothly connected surface in physical space and that an area with a sharp transition indicates two separate surfaces. Obviously, the more assumptions one makes, the further the class of possible surface solutions is restricted. However, the chance that the reconstructed object is not accurate also increases with each additional assumption. Before we can begin to understand how objects are perceived, we need to know about the physical behavior of objects. How they reflect light, and what the descriptors of shape are that can be used. In other words, what kind of information about the depth dimension is preserved in the projection from a three-dimensional object to a two-dimensional retinal image? A pattern in the retinal image can only be an effective cue to the threedimensional shape if it has a causal relationship with the real shape of the physical object. In order to understand how the physical world is represented in the mind, one thus needs to study the physical properties of objects under projection. From our knowledge of the physical behavior of objects we might be able to derive potential cues for the visual system to derive a three-dimensional reconstruction of the world. The idea that the observer is inextricably linked with its environment was very fashionable at times in the twentieth century. Gibson (1966), for example, stated that an understanding of the physical properties of the world is essential for understanding perception. Once the physics of object properties is sufficiently clear for its present purpose, the question can be raised of what physical correlates perceived object properties have. It is clear that the perceived shape of things must correspond sufficiently reliably to the physical shape otherwise the percept would be quite useless. However, the physical shape in itself is not what the visual system processes. It rather processes light and dark distributions and their changes over time in the visual image. It is well known from the literature, and from everyday life as well, that shapes might look different if they are illuminated

On the visual appearance of objects

13

from a different direction (e.g. Nefs, Koenderink and Kappers, 2005) or when you paint them in a different color. In other words, we want to know what object properties make objects look big, three-dimensional, rough, shiny, etc. As will be clarified later, there are physical relationships between shape, material and illumination once certain simple assumptions have been made. A further question emerges then. Namely, what is the internal coherence of the percept of an object as a whole? For example, one can come up with an algorithm to reconstruct the shape of an object from the shading pattern once an assumption is made on what the material and illumination conditions are. Changing these assumptions should change the reconstructed shape in systematic ways. However, is that actually the case? Does the visual system construct a coherent representation of the world, or are the percepts of shape, material and illumination essentially independent? The question that is raised is thus whether and how physical relationships between shape, material, and illumination are duplicated in perceptual relationships. In order to answer the questions that I raised above, I will first give in Section 2 the necessary theoretical background on the physical properties and relationships between shape, material, and illumination of objects in projection. Secondly, in Section 3, the physical correlates for perceived object properties are discussed. Thirdly, in Section 4, I will discuss perceptual organization and whether perceived object properties conform to physical constraints; that is, whether they form a coherent set. In addition I will discuss how additional cues can be used to form a coherent percept.

2. THE PHYSICAL WORLD 2.1. Shape Before starting a discussion on perceived shape, it is useful to agree on a measure to determine the extent to which two different object shapes are different or the same. The first step is to restrict shape to the ecological realm that can actually be seen. Every natural object is thoroughly fractured at the microscale and unstable, whereas at the macroscale it may be quite smooth and unchanging. The microscale is, however, not very interesting for visual perception. Likewise, the fact that the earth is round at the macroscale is not of any importance to perception since we are more concerned with smaller things. Given the restriction on the ecological realm, a transformational approach can be used to describe how two object shapes are different. For example, two objects might be the same in every way that can be discriminated by the spatial resolution of the visual system, except for their position or orientation in three-dimensional space. We can therefore say that these two objects are related by a translation and/or a rotation. All object shapes can be transformed into each other by some mathematical function. A prerequisite is that it is known which points on one object correspond to which points on the other object. Although this is not a trivial problem, let us for now assume that we do know. I think it is reasonable to say that two different sugar cubes that lie next to each other on a table and have the same global shapes are the same except for their position and orientation. That is, a sugar particle in the upper left corner of the first object corresponds to a sugar particle on one of the corners of the second cube, within a reasonable spatial tolerance, but not to a random particle somewhere in the middle. At the end of the nineteenth century, the German mathematician Felix Klein proposed to order transformations according to the properties that they leave invariant. This ordering has become known as the Klein umbrella. The transformations that are grouped

14

Product Experience

under the Klein umbrella are not the only possible transformations one can make from one object to another, there are many more, less restrictive, geometries possible, such as a topology. However, the Klein transformations have some important properties that are highly relevant for perception. For example, they all preserve connectivity and smoothness. That is, all points on a surface that are smoothly connected to another point are still smoothly connected to the same point after transformation. Therefore, this kind of transformation does not tear a surface apart, nor does it make holes in it. Cederberg (2001) gives a relatively accessible text on modern geometries. The transformations in the Klein umbrella are illustrated in Figure 1.1. Although we do not do so here for illustration reasons, the images of Figure 1.1, and the definition in Equation 1.1, can be expanded into three spatial dimensions. In the remainder of the chapter we are interested in three-dimensional shape rather than two-dimensional shape. The most restrictive transformation is when two different objects are the same in every way except for their position. These two objects are related by a translation or a rotation. We say that these objects are isometries of each other. An isometry transformation preserves the absolute distance between all points in an object. Isometries are illustrated in Figure 1.1 in the left most column. A more general transformation than the isometry preserves only the ratios of distances between points of an object and some point in the field. This is a called a similarity transformation. Dilation is a similarity transformation, but not an isometry. Note that all isometries are also similarities, because if absolute distances are preserved between all points, so are their ratios. Similarity transformations are illustrated in Figure 1.1 in the second column from the left. Storing a representation in memory in a similarity geometry instead of in a full Euclidean format has distinct advantages. For example, objects can be recognized irrespective of their size in either physical space or on the retina. That is, object representation and object recognition are size-invariant. The third group of transformations preserves parallelity but not necessarily position, distance or ratios of distances. All that is parallel in the original object is still parallel in Projectivity Affinity Similarity Isometry Translation

Rotation

Dilation

Strain

Collineation

Shear

FIGURE 1.1 The Klein umbrella. The black figure is in each case the original figure and the gray image is the transformed figure. From left to right isometries, similarities, affinities and projectivities are illustrated.

On the visual appearance of objects

15

the transformed object. These transformations are called affinities. Straining and a shearing are both affinities, but not similarities. Straining is increasing or decreasing the size of the object in one dimension only, whereas the dimension at right angles with the first one remains unchanged. Shearing translates the points in the image in one dimension as a function of the second dimension. For example, the more you go to the right, the more the points are translated upwards. It is like holding on to left side of the object while pulling it up on the right side. Straining and shearing are illustrated in the third column from the left in Figure 1.1. All similarities are also affinities, because they also preserve parallelity. The fourth group preserves collinearity. All points that are collinear, that is on a single straight line, in the original object are also collinear in the transformed object. Since areas of zero curvature have at least three collinear points, these areas thus remain invariant as well. These transformations do not add extra bumps or dimples on a surface. Transformations that preserve collinearity are called projectivities. All affinities are also projectivities. An interesting property is that the cross-ratio of four distinct collinear points is invariant. The cross-ratio can be interpreted as the ratio of ratios of distances. More complex, but also invariant is a five point cross-ratio in which the points do not have to be collinear. Storing mental object representations in a projective geometry can have distinct advantages for object recognition. Objects that are stored in a projective format are invariant for changes in viewpoint. For example, the fact that the cross-ratio of any set of four collinear points, or the five-point cross-ratio, on an object remains invariant after a projective transformation can be exploited in machine vision for automated object recognition. The transformations that are shown in Figure 1.1 look complicated at first glance, but they can be described by a simple matrix multiplication as in Equation 1.1. ⎡ xi ⎤ ⎡ a r u ⎤ ⎡ xi ⎤ ⎢ ⎥ ⎥ ⎢ ⎥ ⎢ ′ Equation 1.1: X ⫽ ⎢ yi ⎥ ⫽ A ⋅ X ⫽ ⎢ s b v ⎥ ⋅ ⎢ yi ⎥ ⎢ ⎥ ⎢ p q 1⎥ ⎢ ⎥ ⎢⎣ hi ⎥⎦ ⎥⎦ ⎢⎣ hi ⎥⎦ ⎢⎣ In this equation, X is the set of points of the first object and X⬘ is the set of points on the transformed object. The points in the sets X and X⬘ have been put in the same order. That is, point Xi corresponds to point X⬘i. The two objects are two-dimensional, xi and yi are the coordinates of point i. We added a third dimension, h. The triplets {xi, yi, hi} are homogeneous coordinates. Cederberg (2001) describes the convenience and use of homogenous coordinates in more detail. The thing to remember about homogeneous coordinates is that all {xi, yi, hi} are equivalent to {xi/hi, yi/hi, 1}. The Matrix A is the transformation matrix. The parameters a and b strain the object, s and r shear the object in the image plane. Rotations can be obtained by specific combinations of a, b, s, and r. The parameters u and v translate the object in the image plane, and p and q shear the images in the extra dimension h; the latter induces a projective transformation in the image plane.

2.2. Material For the perception of material properties it is important to know how the reflection of objects can be described. The restriction is to the ecological realm that can actually be seen. Furthermore, objects that emit light themselves, such as lamps, or objects that store energy and emit it over a prolonged period of time, such as ‘glow-in-the-dark’ paint, are not considered here.

16

Product Experience

FIGURE 1.2 Lambert’s Law. Light beam B is spread out over a larger surface area than beam A. Therefore, the illumination per surface unit area is less in the case of beam B than in the case of beam A.

When light falls on a surface, it is reflected, absorbed, or passes through. The total amount of reflected light as a proportion of the light that falls on the surface is called the albedo or ‘whiteness’. Obviously an object cannot reflect more light than what falls on its surface. This has an interesting consequence. Namely, if light falls on a surface under an angle, the incident light is smeared over a larger area than when it hits the surface perpendicularly (see Figure 1.2). Therefore, the amount of reflected light per surface unit area depends on the angle of illumination relative to the surface. The apparent darkening of a surface depending on the illumination angle is called shading. Shading is a primary cue for the visual system for shape perception. The relationship between luminance and the angle of incidence is known as Lambert’s Law. Lambert’s Law can be expressed as in Equation 1.2: Equation 1.2: λ(n, i) ⫽ i ⋅ n ⫽ Iin * Cos(β ) In this equation, λ is the surface luminance. Luminance is thus the amount of light that is reflected from the object to the eye, whereas illuminance is the amount of light that shines on the object. The surface normal is denoted by n, and i is the vector describing the direction of illumination. The length of i is the illumination intensity Iin. The angle of incidence, that is the angle between the surface normal, n, and the direction of the incoming light, i, is denoted by β. These notions are illustrated in Figure 1.3. Each surface has its own reflection characteristics on top of Lambert’s Law. These characteristics are known as the Bidirectional Reflection Distribution Function, or BRDF. In general, the BRDF depends both on the angle of incidence and also on the viewing angle. The observed luminance can then be described as the product of the intensity of the illuminant, the dot product of the surface normal and the direction vector of the illuminant, and the BRDF as in Equation 1.3. Equation 1.3: λ(n, i, p) ⫽ i ⋅ n * BRDF(n, i, p) The vector p is the viewing direction. Nicodemus et al. (1977) give a good description of reflectance properties.

On the visual appearance of objects

17

FIGURE 1.3 The surface normal, n, is the vector that is perpendicular to the surface. The illumination direction is indicated with i. Vector p indicates the viewing direction. The angle α that is spanned between n and p is the viewing angle. The angle β that is spanned between n and i is the angle of incidence.

FIGURE 1.4 Computer renderings of a slightly bumpy sphere. The objects have, from left to right, a Lambertian, specular, backscatter and asperity BRDF. The two rows depict the same objects and BRDFs, but illuminated from different directions. The top row is illuminated from the viewpoint of the observer. The bottom row is illuminated from an area to the right of and above the observer.

In the special case that the BRDF equals one for all incidence and viewing angles, we say that the surface is a Lambertian surface. In that case, the viewing direction, p, does not appear on the right-hand side of Equation 1.3. Therefore, the luminance of a Lambertian surface is independent of the viewing direction. The incoming light is scattered from the surface in all directions irrespective of the angle of incidence. Many of the less-sophisticated computer graphics algorithms render objects using Lambertian surfaces since the luminance computation is easy, fast, and gives a reasonably realistic threedimensional impression. There are, however, no known materials that actually reflect light in a Lambertian fashion even though many materials come close. Figure 1.4 shows images of the same slightly bumpy sphere that are rendered with different BRDFs. From left to right in Figure 1.4 we used a Lambertian, specular, backscatter and asperity BRDF. A nearly Lambertian BRDF can, for example, be found on objects carved from chalk. In the case of specular reflection, incident light is mostly

18

Product Experience

reflected in the direction where the angle of incidence is mirrored in the surface normal (i.e. α ⫽ β in Figure 1.3). All mirrors, including Christmas balls, have a specular BRDF. Objects often show a combination of a Lambertian and specular BRDF. These objects have a Lambertian-like shading, but with added specular highlights, and are perceived as being ‘glossy’. Specular highlights are the bright spots on a glossy object where the illumination direction and the viewing direction are exactly mirrored in the surface normal. Examples of these BRDFs include glossy fruit, plastic, and porcelain. In the case of a backscatter BRDF, incident light is mostly reflected back into the direction of the light source. Backscatter BRDFs can be found on the retroreflectors on bicycles. Retroreflectors cast the light back towards the headlights of an approaching car. An asperity BRDF can frequently be found on materials with short hairs such as peaches and velvet. An asperity BRDF makes the areas near the contour of the object brighter. The two horizontal rows in Figure 1.4 depict the objects when illuminated from 0 and 45 degrees away from viewing direction. For illustration purposes, the illuminant is made to be stronger for the backscatter and asperity BRDFs than for the Lambertian and specular BRDFs (see also Pont and Te Pas, 2006). Some materials are partly translucent. That is, incoming light penetrates the surface and is scattered or reflected inside the object. This light is not absorbed, but eventually leaves the object again. A most-clear example is the reflectance of a diamond. The light that falls on a diamond penetrates the surface and is then reflected again and again in the facets inside the diamond until it finally finds a way out. The particular way a diamond is shaped causes the light to bounce so many times that the point where the light eventually leaves the diamond is unpredictable and highly unstable with small changes in the illumination direction. This gives a diamond its distinctive fire. Another example of a translucent material is human skin. Incoming light is scattered below the surface through the skin layers. The translucency of the skin gives it its slightly reddish appearance and softens the edges of cast shadows on the body. I will not discuss translucency in this chapter any further. Nicodemus et al. (1977) describe an extended form of the BRDF that includes subsurface scattering as well.

2.3. Illumination A beam of light has intensity, direction and a wavelength spectrum. It always propagates in a straight line unless it is reflected. The illumination in a scene can all come from one direction, such as a scene that is directly illuminated by the sun. The illumination is in this case parallel. Common forms of illumination are illustrated in Figure 1.5. In the case of a spotlight or point light source that is relatively close to the scene, the illumination is convergent or divergent. Usually, however, there is light coming from multiple directions. Illumination that comes from all directions is called diffuse or ambient. In the case that illumination comes from only one half of all possible directions, it is called hemispherically diffuse. For example, in the case of an overcast sky, the majority of illumination is coming from all directions in the upper hemisphere (the sky) and very little from the ground. Of interest is to note that ambient light in combination with parallel light softens cast shadows. Parallel light gives hard cast shadows that hide many features of the object whereas it enhances surface imperfections (tiny bumps and dimples). Photographers often make use of ambient light from a relatively small range of directions with a ‘soft box’ or an umbrella to make the image esthetically more appealing. This kind of light is much more ‘forgiving’ of surface imperfections.

On the visual appearance of objects

19

FIGURE 1.5 Illumination conditions. Illumination can be parallel, divergent or convergent. Often light comes from all directions which is called ambient or diffuse. In the case light comes from only half of all possible directions, it is called hemispherically diffuse.

2.4. Color Of special interest in the appearance of objects is their color as it can set the objects apart from the environment and might signal their use or condition. For example, a slightly brownish tint on an apple signals that is rotten. Law-Smith et al. (2006) found that there are visual cues in female faces that are indicative of high estrogen levels and which are associated with ratings of femininity, attractiveness, and healthiness. Law-Smith et al.’s facial images suggest that a slightly reddish tint is one of those cues. Humans have four different types of photoreceptors that allow them to see light with wavelengths between roughly 350 nm (violet) and 750 nm (red). Three photoreceptors are associated with color vision whereas the fourth photoreceptor is associated with ‘night vision’. Wyszecki and Stiles (1982/2000) give an elaborate overview on color science, as well as a wealth of quantitative data on the subject. Objects reflect light for different wavelengths to different amounts and thereby create different colors. Elementary theory on color mixing can be found in any undergraduate textbook on sensory psychology. There are many systems available to order and describe colors. At the heart of all systems are the so-called CIE color matching functions. CIE stands for ‘Commission Internationale de l’Eclairage’ (International Committee on Illumination). The CIE color matching functions were first reported in 1931 and have served since then as the standard definition of color, although other color matching functions have been measured as well. The color matching functions describe the intensities of three light sources with different spectra that, when they are combined, look the same as a light source of a single wavelength. Light that is composed of only a single wavelength is called monochromatic. Monochromatic light is, for example, used in, typically yellowish, sodium streetlights because they are particularly energy efficient. Usually, natural light consists of a spectrum of different wavelengths.

20

Product Experience

The three light sources that were used to measure the CIE color matching functions were quite arbitrarily chosen. In fact one can perform the same procedure with three different light sources as long as they are sufficiently far apart in wavelength or color. It is relatively straightforward to calculate the correspondence between sets of color matching functions made with different light sources. I have plotted the CIE color matching functions in Figure 1.6A. The points of monochromatic lights form a continuous curve in the space spanned by the three light sources. A ‘color cone’ can be constructed by connecting the origin with all monochromatic points. All possible colors that can be perceived have a corresponding coordinate within the color cone. Usually, a cross-section is made through the color cone at which the intensity of the light is perceived as constant, as in Figure 1.6B. In Figure 1.6B all data points lay on a plane. This plane forms the CIE (x, y)-chromaticity diagram that might be familiar to the reader. Colors can be described with their coordinates in this plane, along with their intensity. Note that some colors, namely the purples, are not on the color-matching curve. There are no wavelengths that correspond to this color, because they can only result from a combination of a ‘reddish’ and a ‘bluish’ wavelength. Purples can be found on what is called the purple plane, the

B

A

Blueish

Blueish

Gr

ee

ish

dd Re Gre

nis

h

enis

dish

Red

h

C

h dis

Greenish

d

Re

FIGURE 1.6 Colors and object colors. (A) The color cone in the CIE (x, y, z) color space. (B) The positioning of CIE (x, y) chromaticity diagram in the color space. (C) All object colors in the CIE (x, y, z) color space that can be made with illuminant CIE-D65.

On the visual appearance of objects

21

flat plane in Figure 1.6A, or purple line, the straight line in Figure 1.6B, in case of a chromaticity diagram. This plane closes the gap between the red side and blue side of the color cone. The chromaticity coordinates for any light source can easily be found by multiplying the color matching functions with the spectrum of the light source. The color matching functions can, for example, be obtained from the Color and Vision Laboratories of the Institute of Ophthalmology at the University College London (http://www.cvrl.org). It is possible that different spectra yield the same chromaticity coordinates. The colors of these light sources are thus the same. This phenomenon is known as metamerism. Usually, objects with metamer colors can easily be distinguished by looking at them through a colored filter or by shining light of a different spectrum on them. For example, a trick that is sometimes employed by the colorblind to distinguish red and green is to view them through a red filter. Red and green are metamer colors for the colorblind. Viewing a red object through a red lens does not change its intensity, but a green light that is viewed through a red filter turns to black. One particular shortcoming of the CIE (x,y)-chromaticity diagram is that it does not give a quantification of how well two different colors can be discriminated. Furthermore, how far two different colors have to be apart in the chromaticity diagram in order to be seen as distinct depends on the color as well. Additional transformations can be made to rectify this problem, as is for example done in the CIE (x⬘,y⬘) chromaticity diagram and the CIE (L * u * v) and (L * a * b) color spaces. Not all colors can be made given a certain illumination spectrum. If there is no light in the red part of the spectrum, it is impossible to make an object look red, whatever its spectral reflection function. For example, the color of an object is very different in daylight than under the illumination of yellow streetlights. In fact, since sodium streetlights are monochromatic, there are no colors any more! All light that comes into the eye is of the same wavelength, irrespective of the material that has reflected it to the eye; only the intensity of the light differs. The casual observer might not even notice the absence of color under these conditions though, because there is still a lot of variation in light intensity. Given a certain illumination spectrum, it is possible to calculate all possible colors that can be made from shining this light on a surface. The resulting colors are called object colors. An object color is the color that the observer identifies as the color of an object. Object colors can thus, depending on the spectral composition of the illumination, take on different values. The object colors for a given illumination spectrum fill only a part of the color cone. The set of object colors thus fills a different volume, often called ‘color solid’, for each specific illumination spectrum. The volume is filled with all colors that can result from all possible reflectance functions and given a certain illumination spectrum. I have created such a color solid in Figure 1.6C. I have taken a standard illuminant, namely ‘CIE-D65’, and calculated all colors that can result under that illumination. The CIE has specified several standard illuminants for this purpose. There are many systems to order different object colors. I will briefly mention one particularly popular model, namely the Munsell Color system. The Munsell system orders object colors on three dimensions namely: Luminance; hue; and saturation. Luminance is a value between black and white. Hue can be described as ‘red’, ‘blue’, ‘purple’, and so on. Saturation is the amount of hue as opposed to being gray. Pastel colors, for example, have a low saturation. The system is composed of a set of differently colored chips that can be arranged in a solid, much like in Figure 1.6C. The chips are chosen such that their discriminability increases in equal steps. The Munsell color system does not take into account how colors of materials change over viewing, illumination angle, and spectral composition of the light source. Calibrated chip sets for the Munsell are commercially

22

Product Experience

available. More detail on the CIE color matching functions, the Munsell color system, etc., can be found in Wyszecki and Stiles (1982/2000). Object colors as discussed so far do not form the entire story though. There are a few restrictions. First, the BRDF is dependent on the wavelength of illumination. A striking example is the Compact Disc. Light with the same angle of incidence but different wavelengths is reflected in different directions on a Compact Disc. A full description of the color of an object therefore involves the viewing and illumination directions as well. Secondly, the perceived color of objects also depends on the color of the environment. The chromaticity coordinates and the Munsell system is only valid when the object or light is viewed against a gray background. I will address this further in Section 3.2. Obviously, this has important consequences for the product designer, as the typical illumination and viewing conditions in which the product is going to be sold or used must be taken into account.

2.5. The SMI triangle It can be seen in Equation 1.3 that the luminance of a surface patch, as seen from the eye, depends on three components, namely (a) the surface normal; (b) the BRDF; and (c) the illumination. All that the visual system is presented with, however, is the object’s luminance in the direction from the surface to the eye. This gives rise to an interdependency of object properties which I will denote here as the Shape–Material–Illumination Triangle, or SMI triangle for short. The SMI triangle has two degrees of freedom. That is, you can choose any two vertices of the SMI triangle to be whatever you want, but the third one is then fixed. When reconstructing the world, all three vertices of the SMI triangle are a priori unknown. This does not mean, however, that one cannot make an educated guess. However, if one wants to find, for example, a solution for an object’s shape, one has to make assumptions on what the material and illumination conditions are. Ikeuchi and Horn (1981), for example, describe an iterative algorithm to reconstruct shape from shading. This algorithm assumes the BRDF and the illumination conditions to be known. One of the interesting questions that can be raised is what the resulting ambiguities are, given some reasonable assumptions about the real world. Let us first assume that every pair of neighboring points that are smoothly connected in the image are also smoothly connected in three-dimensional space. Vice versa, every pair of neighboring points that are not smoothly connected in the image are also not smoothly connected in three-dimensional space, that is, connected but not smoothly, or not connected at all. Furthermore we assume that illumination and material are homogeneous over the object. These two assumptions already reject trivial solutions such as paintings on canvas or highly unlikely light sources on flat surfaces such as computer beamer projections. Consequently, paintings and projections are perceived just as if they were three-dimensional scenes. The SMI triangle is best illustrated by looking at a specific example of an ambiguity in more detail. Although this example concerns the ambiguity of shape, similar examples can be made for illumination and material properties. One of the best-known ambiguities in constructing the surface of an object is the so-called bas-relief ambiguity, which translates from French as ‘low sculpture’. Bas-reliefs are relatively flat sculptures that still give a very good impression of depth. Bas-reliefs can be found throughout the centuries. In spite of the fact that the bas-relief ambiguity has been known for a very long time, its mathematical structure and relationship with the illumination direction and intensity has been derived only recently (Belhumeur, Kriegman and Yuille, 1999). The bas-relief ambiguity is shown in Figure 1.7. The ambiguity lies in the fact that the same image can be obtained from objects that are scaled or sheared in the depth dimension when the illumination direction and intensity is changed at the same time.

On the visual appearance of objects

23

FIGURE 1.7 The bas-relief ambiguity. The top row shows an untransformed globular object from a frontal viewpoint, and the same object as seen from the top. The other objects show a transformed object with a transformed illumination direction. In these images, albedo is kept constant. The object in the second row from the top is sheared with a value of 0.3. The third row from the top is strained in depth in the direction towards the observer with a factor of 0.7. The bottom row shows the object when it is both strained (1.2) and sheared (⫺0.3). Despite the large differences in 3D shape, the images in the left column are nearly indistinguishable.

24

Product Experience

The angle of incidence can be expressed as the difference between the directions of the surface normal and the direction of illumination. An affine transformation changes the direction of the surface normals in a linear manner. This gives us an intuitive understanding of the bas-relief ambiguity: If the surface normal is changed to the same extent as the illumination direction, the same luminance results and the images are therefore indistinguishable. Cast and body shadows are preserved exactly. However, in order to keep the shading pattern exactly the same in the image, a transformation in albedo is required as well. For example, when an object is flattened the contrast in the image is lowered and the object is then typically perceived as being flatter. In order to correct for this effect, the albedo of the surface has to be changed locally such that the surface becomes darker the more the surface normal turns away from the illumination direction. However, the necessity to transform the albedo becomes only apparent in very strong bas-reliefs. In Figure 1.7, I did not correct for albedo. Interestingly, introducing an ambient component in the illumination affects the contrast in the image as well; ambient light illuminates all parts equally irrespective of the surface normal, and therefore lowers the contrast within the image. Adding an ambient term can thus counterbalance the effects of straining and shearing. When an ambient component is added, the contrast in the image is reduced which leads to a flattening effect and, when the ambient term is lowered, it leads to higher contrast and to an enhanced impression of depth in the object.

3. OBJECT APPEARANCE 3.1. Shape Several methods have been used in the past to measure the perceived shape of an object. These measures involve local depth comparisons, cross-sections, depth–width ratios, and surface attitude probes. For example, Koenderink, Van Doorn and Kappers (1992) introduced a popular method based on measuring the surface attitude. The method involves a surface attitude probe or a so-called ‘gauge figure’. The probe typically consists of a small circle with a little stick that is superimposed on the image. It resembles the shape of a pushpin. The aspect ratio and orientation of the gauge figure can be changed such that it looks as if the probe rotates in three-dimensional space. The probe is located on the image of an object and the observer sets the probe such that it looks like a small circle that is painted on the surface, with the little stick perpendicular to the surface. Across trials, the probe is placed at numerous points all over the image. The setting of the gauge figure is usually, but not necessarily, measured in slant and tilt. Slant is the angle away from the line of sight and tilt is the angle around the line of sight. Slant and tilt are illustrated in Figure 1.8. An example of the gauge figure task is shown in Figure 1.9. Figure 1.9A shows what the observer typically sees in an experiment during a single trial. The featured object in this case is an image of a globular virtual object that was created from a computer algorithm to simulate a three-dimensional shape with a glossy BRDF that was illuminated from a specific direction. Usually this kind of research is done on images of objects on a computer monitor in a darkened room rather than with real three-dimensional objects, for practical reasons. These images might be photographs taken of real objects or renderings of virtual objects that are created from a specific computer algorithm. In Figure 1.9B, I have accumulated all probe settings in one image. From the probe settings a surface can be recreated as in Figure 1.9C.

On the visual appearance of objects

25

FIGURE 1.8 Slant and tilt. A number of different circular patches are shown on a sphere. The slant of a patch is the angle at which the surface patch is turned away from the line of sight. Tilt is the angle around the line of sight. For example, patch A has a slant of 67 degrees and a tilt of 45 degrees. Patch B has a slant of 45 degrees and a tilt of 135 degrees.

FIGURE 1.9 The gauge figure task. (A) shows what the observer typically sees. The pushpin-like gauge figure can be manipulated by the observer and has to be oriented such that it looks like a circle painted on the object’s surface with the little stick as the outward surface normal. All observer settings are accumulated in (B). The settings are integrated into a ‘perceived’ surface in (C), for illustration purposes seen from a different viewing angle.

It should come as no surprise that the perceived surfaces of shaded objects show likeness with the ground truth up to a certain point. Human observers are also consistent in perceiving the three-dimensional shape of shaded objects, although perceived surfaces vary a lot in straining and shearing in depth when the procedure is repeated over different observers. Some observers set the gauge figures consistently to larger values than other observers. This might be interpreted as though these observers see more depth than others. However, it might also mean that observers map the aspect ratio of the gauge figure differently to a surface gradient in the mental world. There is somewhat less variance in straining and shearing when the procedure is repeated for the same observer over different sessions than when repeated over different observers. It has been found that most of the variance in perceived surface shape between observers or between tasks can be accounted for by affine transformations (Koenderink et al., 2001).

26

Product Experience

We can now address the question of the effects of material and illumination conditions on perceived shape. A change in illumination direction can have significant effects on perceived object shape. Curran and Johnston (1996), for example, found that the perceived curvature of spheres is affected by the position of the light source. Perceived curvature decreased as light source tilt (rotation in the image plane) increased and as light source slant (rotation away from the line of sight) decreased. Nefs, Koenderink and Kappers (2005; 2006) also found significant effects of illumination direction on the perceived depth and on the overall attitude of the object. The effective cue seems to be the overall contrast in the image. Less contrast in the visual image seems to be interpreted as the object being flatter. Increasing the level of ambient light makes the surfaces thus appear flatter, since contrast is reduced. In addition to straining and shearing the perceived surface, a change in illumination direction also induces deformations in the perceived surface that cannot be described by affine transformations. Koenderink, Van Doorn, Christou and Lappin (1996a,b) for example found systematic distortions of the perceived shape of human torso replicas with changes in illumination direction. For all observers in their study, those parts in the image that had a higher luminance were consistently interpreted as being nearer. Christou and Koenderink (1997) also concluded that even with images of simple ovoid shapes, subjects’ settings of surface attitude were systematically biased in the illumination direction, and were consistent with a regression to image luminance gradients. That is, dark parts are perceived as deeper and brighter parts of the objects are perceived as being closer. Nefs et al. (2005) also found that there are non-linear changes in perceived surface relief when the direction of illumination is changed. They described the change that was induced by changing the illumination direction as a shift in the position of the parabolic curves in the perceived surface. Parabolic curves are the sets of points on the object where the surface goes from being convex or concave to being saddle-shaped. In general, these points form closed curves on the surface. The shift in the convex, saddle-shaped and concave areas in Nefs et al.’s experiment is shown for one observer in Figure 1.10. The dark gray part represents a convex area, the black part is saddle-shaped and the light gray part is convex. The parabolic curves in the perceived surface reconstruction are located on the transitions from convex to saddle-shaped and from saddle-shaped to concave. The white jagged lines indicate where the parabolic curves of the physical shape are located. The knowledge of the effects of different BRDFs on perceived surface shape is somewhat less advanced. However, it is reasonable to conclude from Figure 1.4 that the BRDF clearly does have an effect. Most research has focused on the presence of specular highlights. Figure 1.9 shows an example of an object with three distinct specular highlights. There are good reasons for investigating specular highlights. First is a practical reason, specular highlights are easy to make in computer graphics and therefore easily available for researchers to play with. Secondly, specular highlights are not easily confused with Lambertian shading because of their high intensity. And thirdly, it has been argued that specular highlights are potentially informative of shape: Specular highlights are in general found on specific areas on an object (e.g. Longuet-Higgins, 1960; Oren and Nayar, 1996). Specular highlights have, for example, the tendency to cling to areas of high curvature and move rapidly over areas of low curvature when the illumination direction or the viewing direction is changed. This is also the reason why specular highlights follow predictable paths over the surface with changes in illumination or viewing direction. Specular highlights tend to move over the parabolic curves of an object. In the case of purely specular objects in complex environments we see a compression of the reflection of the world at areas where the object is strongly curved, and an expansion at areas of low curvature (e.g. Flemming, Dror and Adelson, 2003; Flemming, Torralba

On the visual appearance of objects

27

FIGURE 1.10 The top row shows the stimulus as used in Nefs et al. (2005). Observers made gauge figure judgments on these two images. In the bottom row, two diagrams are shown that indicate the convex, hyperbolic and concave areas in the perceived surface. The dark gray areas are convex, the black areas are hyperbolic and the light gray areas are concave. The jagged white lines show the positions of the parabolic curves of the real, physical shape are, which were the same for both objects. Reprinted by permission from Pion, London: Perception, 34, 275–287, copyright 2005.

and Adelson, 2004). For example, the reflection of the world in a Christmas ball makes everything look small, whereas the same world viewed in a flat mirror appears to have its original size. On objects that have both flat and curved areas, the world is thus compressed to a different extent depending on the surface curvature. The scientific evidence that the addition of specular highlights to otherwise Lambertian reflecting objects induces differences in perceived surface shape is contradictory though. On the one hand, there are studies that support an effect of specular highlights. Todd and Mingolla (1983) reported that the presence of specular highlights on cylindrical surfaces enhances the perception of surface curvature. Norman, Todd and Orban (2004) showed that shape discrimination was improved when specular highlights were added. Todd, Norman, Koenderink and Kappers (1997) reported that observers’ judgments were significantly more reliable when specular highlights were added to Lambertian surfaces when objects were viewed stereoscopically. Blake and Bülthoff (1990; 1991) showed that specular highlights can be used by observers to disambiguate convex/concave surfaces in stereoscopic vision. On the other hand, there are studies that do not report significant effects of the presence of specular highlights. Mingolla and Todd (1986) found that changing the illumination direction influenced the judged shape of ellipsoid surfaces, but the presence or absence of specular highlights did not have a significant effect. Nefs et al. (2006)

28

Product Experience

also reported effects of illumination, but not of specular highlights, on shape perception for a range of globular objects. Norman, Todd and Phillips (1995) reported that observer settings were no less accurate for objects containing just specular highlights than for Lambertian objects or objects having both Lambertian shading and specular highlights. There are a couple of explanations for the different conclusions in the literature. An important reason is probably the specific shape of the objects that are used. For example, in the case of cylinders, specular highlights form a straight line over the cylinder. And in the case of an object with a large curvature contrast, the specular highlights may trace ridges and parabolic curves over the object. Khang, Koenderink and Kappers (2004) looked into the effects of Lambertian, specular, asperity, and backscatter BRDFs on perceived surface shape. They found that spheres and ellipsoids that were rendered with asperity or backscatter BRDFs were perceived as being flatter than those rendered with a Lambertian or specular BRDF. Hue has no measurable effect on perceived shape (e.g. Claessen, 1996). Troscianko et al. (1991) also showed that iso-luminant gradients from red to green do not affect the perceived slant of a surface. An iso-luminant gradient from red to gray (saturation), however, did have an effect. There is a small interesting effect of hue on the perception of depth though when an object is viewed with two eyes. This effect is best seen when a red object is displayed in front of a blue background. The red part is sometimes seen to stand in front of the blue background. This effect is known as chromostereopsis. It is believed that this effect is caused by the different refraction indices for red and blue light in the cornea of the eye, which results in a binocular disparity between the red and blue parts of the image. Usually the effect is small though and has little consequence for visual perception. You can buy special ‘ChromaDepth™ 3-D’ glasses that enhance chromostereopsis by separating the refraction indices for red and blue further (Chromatek®).

3.2. Material A reduction in illumination makes an object on a gray background look as if it has a lower albedo when the luminance of the background is kept constant. If the background luminance is reduced, that is the contrast between the object and the background remains constant, the perceived albedo of the object is fairly constant over a large range. This is known as luminance constancy. Likewise there is color constancy. The color of an object does not look much different in the evening compared to midday, especially for the colors that have a low saturation. Nevertheless, the spectrum of light entering the eye at the beginning of the evening is quite different from the spectrum at midday. If the evening and midday colors of the object would be shown simultaneously against a gray background, the difference would be striking. What is important for perception is not so much the light intensity or its spectrum, but the luminance or color contrast with the environment. In many ways the human observer has no respect for absolute intensities, perception is foremost concerned with relative intensities. This principle comes back in many different forms in sensory psychology. For example, in luminance and color contrast illusions, but also in motion perception as in the Duncker illusion. The Duncker illusion is best illustrated by the apparent motion of the moon rushing against the moving clouds. Three contrast illusions, namely luminance, curvature and size, are illustrated in Figure 1.11. When predicting or evaluating the appearance of objects in terms of albedo, color, size, motion etc., it is thus advisable to take the environment into account. The effects of luminance constancy and color constancy have also been described elsewhere (e.g. Palmer, 1999). A large part of the literature on material perception is devoted to the perception of lightness/albedo. Todd, Norman and Mingolla (2004) investigated the lightness of objects

29

On the visual appearance of objects

(A)

Luminance contrast

(B)

Curvature contrast

(C)

Size contrast

FIGURE 1.11 Contrast illusions. (A) Luminance contrast. The gray patch in the white circle is perceived as being darker than the gray patch in the black circle, even though they have the same gray value. (B) Curvature contrast. The physically straight line appears to bend. (C) The black disk is perceived as larger when it is surrounded by small circles than when it is surrounded by large circles, although the physical size is the same.

as a function of the presence of specular highlights. Lightness is here defined as the fraction of the amount of reflected light that is reflected in a Lambertian manner over the total amount of incident light. Todd et al. concluded that observers were quite able to discount the luminance intensity of specular highlights when judging lightness. An implication of this is that a mere glossy finish of a product does not change the perceived albedo of the underlying material. That is, objects do not look whiter (as opposed to gray) with a layer of varnish. Khang et al. (2003) investigated whether the perception of surface albedo is influenced by the mode of lighting in 3D geometrical shapes. They rotated a Platonic solid (in this a case a cube (six faces), a dodecahedron (12 faces) or an icosahedron (20 faces)) on a computer monitor. One of the faces of the solid was rendered with an albedo different from the other faces. Observers had to indicate whether there was a face with a different albedo from the other faces present in the object or not. It was found that observers performed best when the illumination was ambient, then hemispherically diffuse, and they performed worst when the illumination was parallel. It was also found that the number of faces that could be seen at any time was important. Observers performed best with the icosahedron, then with the dodecahedron, and they performed the least well with the cube. The symmetry of the object was found not to be an important variable in identifying the faces of polyhedrons with different albedo. The identification accuracy was the same whether the polyhedron was regular or irregular. Of practical use is that parallel light can hide slight variations in albedo better than other illumination modes. In case the aim is to detect imperfections of albedo on the surface it is best to use an ambient illumination.

3.3. Illumination Pentland (1982) proposed that the direction of illumination in smoothly shaded convex objects could be estimated from the first-order derivative of the image intensity and demonstrated that estimates of the illumination direction that are based on this idea correspond

30

Product Experience

well with estimates made by human observers. This study shows that observers have an idea where the light in a scene is coming from, which might in turn be important for shape and material perception. Khang et al. (2006) investigated the effects of material on perceived illumination direction. They showed that the type of BRDF influenced the accuracy of illumination settings. They concluded that observers might not take the effects of BRDF into account when judging the illumination direction, despite the large changes in the image intensity that it brings about. It has been suggested that observers are sensitive to the color of specular highlights (Yang and Maloney, 2001). Although not strictly true for all materials, the spectral composition of specular highlights is for many materials more similar to the light source than to the spectral composition of Lambertian shading of the object. This effect is often seen for varnished or wet materials. For example, when shining a white light on a glossy red plastic ball, there will be a white, rather than a red, specular highlight on the ball. On the other hand, the specular reflection of metals, such as copper and gold, is not color neutral to the light source; the specular reflection of gold is distinctly yellow for white illumination.

4. PERCEPTION 4.1. Perceptual organization The visual system is not a passive machine, but actively tries to make sense of the surrounding world. It reconstructs the world on the basis of the visual input into meaningful units. Gibson (1979) formulates this notion as: ‘Perceiving is an achievement of the individual, not an appearance in the theatre of his consciousness’. Several organizing principles have been identified in the past and are now commonly known as the Gestalt Laws. Central to Gestalt theory is the idea of ‘Prägnanz’. The Law of Prägnanz can be formulated as: ‘The psychological organization will be as “good” as the prevailing conditions allow’ (Koffka, 1935). The objective of the visual system is to achieve an organization that is as simple and regular as possible. In the context of shape, it means that a visual image is organized into the simplest and most regular shape possible given the visual image. This is a rather vague definition, because what exactly defines ‘good’ and what is a ‘good shape’? Several factors, or laws, have been described that contribute to perceptual organization, or to a good shape. The most important factors are: Good continuation; Proximity; Equality; Closure; Symmetry; and Common fate. The Gestalt factors are illustrated in Figure 1.12. Good continuation means that curves and surfaces are perceptually continued when they appear interrupted. The visual system identifies curves and surfaces that are good continuations of each other and combines them into a single unit. The factor of proximity means that points, lines, etc. that are close to each other are grouped together. Equality refers to the tendency of the visual system to group parts of the visual system together that share some feature, such as color or shape. Closure means that parts of the image that form a closed area or volume tend to be seen as such. Symmetry in this context means a figure is the same as its mirror image. Finally, dot, lines, etc. that move in a common direction and speed are seen together. This is known as common fate. Although we did not go into that in this chapter since all our objects are stationary, structure-from-motion, that is common fate, is often an important cue for shape perception. For example, a well-camouflaged bird may go unnoticed as long as it keeps perfectly still. As soon as the bird starts to move,

On the visual appearance of objects

31

FIGURE 1.12 The Gestalt factors: Good continuation, Proximity, Equality, Closure, Symmetry and Common fate. The arrows in the bottom right picture indicate a motion in that direction.

camouflage is broken, because the observer sees a coherent motion in the bush. A fact that is generally underappreciated in textbooks, but which should be mentioned for good measure, is that many demonstrations of the Gestalt laws in the literature are subject to the time that is given to view them. Like any other process, perceptual organization takes time. The more time that is given to view the scene, the more complex and detailed the resulting organization may become. Irregularities that are at first overlooked are incorporated at later stages. A fairly irregular shape may be perceived as regular when it is shown very briefly, but at longer durations the irregularities become clear. The Gestalt principles are highly important in visual perception since the visual system tries to order the world accordingly. Sometimes however, they lead to incorrect or bistable interpretations. Many visual illusions are therefore associated with the Gestalt laws. A classic example of bistability is the well-known Rubin face/vase illusion where the perceptual organization into a foreground and a background switches between two states. In this illusion the observer flips between a percept of two faces or a vase in the foreground. More interesting, artistic illustrations of bistability than the face/vase illusion can be found in the ‘Slave market with a disappearing bust of Voltaire’ by Salvador Dali, and in ‘Vanity’ by Charles Allen Gilbert. ‘Vanity’ was recently given new life in an advertisement for Dior that won an award for best European advertising photography in the Epica 2002 contest. Anderson and Winamer (2005) recently published one of the most fascinating new illusions. They showed that because of organizing an image into a foreground and background, the lightness of the depicted objects is strongly modified. This illusion is shown in Figure 1.13. The discs on the right of Figure 1.13A are perceived as black, while partly occluded by a white mist. The discs on the left of Figure 1.13A are perceived as white, while occluded by a dark fog. The images of all discs are exactly the same though. Likewise, the images of the two sets of chess pieces in Figure 1.13B are exactly the same, despite their strikingly different appearance in terms of albedo. Anderson and Winamer showed that this illusion is not a mere luminance contrast illusion. They showed this by

32

Product Experience

FIGURE 1.13 An illustration of the Anderson and Winamer (2005) illusion. In (A) the discs in the two images are exactly the same; in (B) the two sets of chess pieces are identical. Reprinted by permission from Macmillan: Nature, 434, 79–83, copyright 2005.

rotating the ‘mist’ over 90 degrees. This manipulation destroys the illusion in spite of the fact that the contrast in the image is unchanged. One of the problems with Gestalt theory is to quantify the contribution of each Gestalt law and to describe exactly what the contribution means. What exactly is a good shape and why is a square a better shape than a rectangle? The key is believed to be the simplicity of the resulting organization: The simpler the organization, the better. Simplicity can be described as the number of information bits that are necessary to describe the scene. Koffka (1935) gives an early example of this idea. Koffka suggested that a circle is more likely to be seen as a mere line than a triangle, because every element of the circle contains the principle of the whole. That is not the case for the triangle. That is, it requires more bits of information to describe a triangle than a circle. Yet another way to put it is that the circle has a higher level of symmetry than a triangle. Several coding systems have been proposed in order to evaluate the simplicity of the structure (e.g. Leeuwenberg, 1971; Van der Helm and Leeuwenberg, 1996).

On the visual appearance of objects

33

Important concepts in these approaches are regularity and symmetry. The effects of different kinds of symmetry in Same/Different tasks have, for example, been investigated by Leeuwenberg and Van Lier (2005). In addition to geometrical considerations for perceptual organization it may be argued that the visual system makes assumptions on what is statistically likely in the real world. For example, the majority of light usually comes from above, rather than from the floor. It makes sense to be biased towards light from above if the situation is ambiguous. In addition, Hill and Bruce (1993; 1994) found that people have a preference to see objects as convex rather than concave. It is often unclear whether a perceptual bias is based on geometric considerations, or on statistical likelihood. Regularity in the visual image, for example, is a very unlikely coincidence if it does not exist in three-dimensional space as well. Consider for example a sphere. From any viewpoint the sphere will appear as a circle in the visual image; that is, spheres yield stable projections. Ellipsoids (egg-shaped objects) project in general to ellipses in the visual image but in accidental viewpoints they project as circles; these projections are unstable. That is, if you see a circle it is more likely projected from a sphere than from an ovoid. A circle in the visual image is just too good to be incidental. Gibson (1979) takes the idea of perceptual organization one step further in his theory of information pick-up. According to Gibson not perceptual qualities, such as color and intensity, are perceived, but rather the information that is provided in their changes over space and time. Mere perception of color or curvature is unsatisfactory to Gibson. True perception is all about seeing the structure in the variation of the world and the structures that remain invariant over time. Gibson suggested that perception is thus not concerned with positions but with places, with objects rather than 2D-shapes and with substances (materials) rather than luminance. This is very much what we are after when discussing the appearance of objects: Object appearance is not about the perception of luminance, it is all about inferred shape, albedo, BRDF, illumination, etc. of the object. What observers are often particularly interested in is what you can do with an object. Observers specifically look for information in the visual image that allows them to infer what it is they can do with an object. A prominent feature in Gibson’s theory is therefore the inferred function, called affordance, of an object. The affordance of an object translates roughly into ‘this object is for … (raking, hammering, throwing, etc.)’. The affordance of an object is not only a property of the object, but also of the observer. For example, a big tree trunk does not afford throwing for a child, but it does for an adult interested in caber tossing.

4.2. The SMI triangle Koffka (1935) writes that prior to his time it was believed that the visual system could extract properties of the visual image independently of each other. For example, color could be extracted independently from motion. Koffka himself, however, believed that this was impossible. More recently, however, evidence has been published that suggests that his position is not quite tenable. The mere fact that you need some light in order to make a pattern move over the retina does not mean that you cannot perceive motion without being able to see light. There are, for example, patients who can see light but not movement (e.g. Zihl, Voncramon and Mai, 1983), and there are also cortically blind patients with relatively intact motion perception (e.g. Azzopardi and Cowey, 2001). Although evidence from patient studies has several problems, it suggests that the coupling of motion and light perception is not a simple fact. The problem of whether perceived object qualities have an inherent relationship goes one step further. These object

34

Product Experience

properties are not qualities of the visual image, but of the representation that is created from that. There are good reasons to expect a relationship between object properties just as there are good reasons not to expect one. A good reason for perceived properties to be coupled is of course the SMI triangle. Gregory (1973) showed a relationship between perceived convexity and illumination direction. He showed that both a mask and a mould of a face look like convex faces, but that the light seems to come from a different direction. That is, a convex mask that is illuminated from above looks like a convex face that is illuminated from above, whereas a concave mould of a face that is illuminated from above looks like a convex face illuminated from below. Johnston, Hill and Carmen (1992) showed that this also holds for faces turned upside down and for hollow potatoes as well. In these cases illumination and shape perception seem to be coupled. It is a common mistake to consider the perceived illumination direction as the reason why the hollow face is seen as convex. The more likely reason is that there is a prior for convexity (Langer and Bülthoff, 2001). Note also that the hollow face illusion is never bistable. Mingolla and Todd (1986) also showed a correlation between perceived shape and perceived illumination direction when observers viewed relatively flat egg-like objects, but not when they saw egg-like objects of greater depth. Te Pas and Pont (2005) found that the perception of BRDF and illumination on spherical shapes was in many cases confounded. In many of the objects that they tested, a change in BRDF was instead perceived as a change in illumination conditions. They did not report whether the perception of BRDF and illumination were correlated on a trial-by-trial basis. Interestingly, when the task was to compare the BRDF of two objects, each with a different BRDF but illuminated the same, observers consistently reported the BRDF to be different. When the task was about the illumination direction, and the objects differed in illumination but the BRDF was the same, observers were less consistent in reporting the illumination to be different. This suggests that the processes underlying material and illumination perception are at least partly decoupled.

4.3. Additional cues Apart from shading there are usually more cues to shape, material and illumination available that are largely independent of the SMI triangle. The shape of the object in the visual image is a very good cue for three-dimensional shape. For example, Mamassian and Kersten (1996) concluded that observers used the occluding contour of the object rather than shading to estimate the local surface orientation of croissant-shaped objects. The occluding contour is where the object curves around to occlude itself from sight. That is, the occluding contour is the edge of the object’s silhouette. Erens, Kappers and Koenderink (1993) showed that the presence of an occluding contour is necessary for the perception of shape-from-shading. The absence of a well-defined contour renders the sign of the surface shape (convex/concave) essentially ambiguous. Furthermore, even in images of objects that lack any shading, such as silhouettes, there are sufficient cues to the three-dimensional structure, such as the occluding contour, in order to get to a reasonably accurate three-dimensional impression. Tse (2002), for example, argues convincingly that the three-dimensional shape of objects can be quite accurately predicted when unlikely combinations of object shape and viewpoints are rejected. Several authors have shown that the perception of material properties is influenced by perceived shape. The perception of shape is in these cases induced by contours and suggestive groupings of surfaces. Knill and Kersten (1991) showed that perceived surface curvature influences perceived albedo. Perceptions of surface attitude and curvature

On the visual appearance of objects

35

shape were induced by suggestive contours. Gilchrist (1971) showed that perceived lightness depends on the relationship of the luminance of surfaces that lie in the same depth plane, but not on surfaces that are merely adjacent in the retinal image. Soranzo and Agostini (2006) showed that increasing perceptual belongingness of surface patches into larger units improved the perception of lightness constancy. Object surface texture deforms in specific ways, depending on the surface attitude relative to the viewing direction. Also, because of linear perspective, the density of the texture increases with increasing distance. Note that the texture in the visual image strains in one direction because of a change in surface attitude of the object, but shows dilation, when the distance is changed. Here we make the assumption that texture is uniform over the object and without a specific direction, but there is a dependency of texture and shape, much like we have seen for shape, material, and illumination. Curran and Johnston (1996) have shown that the effect that the position of the light source has on perceived shape is reduced when texture cues are added. Pont and Koenderink (2003) showed that the direction of illumination is accurately predicted from the direction of the texture in the visual image. That is, bumps and pits in the surface create systematic distortions in the texture in the visual image, because they cast small shadows on the surface that tend to elongate in the direction of illumination. They also showed that illumination direction could be estimated in a similar way in complex scenes. Koenderink et al. (2003) showed that observers are good at estimating the tilt of the illumination direction, but are less proficient at estimating the slant of the illumination direction. Te Pas and Pont (2005) reported also that observers are more reliable in their estimation of illumination direction when three-dimensional texture was present on the object. Three-dimensional texture did not prove helpful in discriminating objects with different BRDFs though. The effects of three-dimensional texture on shape perception have not been evaluated systematically yet. In cases where a higher accuracy of illumination perception is required it makes good sense to add three-dimensional texture to the objects in the scene. For example, when one wants to use the information on the illumination direction to estimate shape and material properties of other nearby objects that are not textured but are only shaded. Stereoscopic vision is another frequently cited cue to three-dimensional shape. Because the two eyes look at the world from slightly different viewpoints, the images in the two eyes are slightly shifted relative to each other. This shift is known as binocular disparity. In general, however, it is not the solution to unambiguous perception. First, binocular disparity works best only in a limited range of distances, and secondly, there is a considerable part (about 30%) of the general population that does not have the stereoscopic resolution that one would wish for (Richards, 1970). Thirdly, there is considerable uncertainty in the perceived distance of the object, which causes unpredictable effects on depth scaling (e.g. Drga and Harris, 2005). And fourthly, stereoscopic vision works only for relatively high spatial frequencies such as edges, lines, texture, etc. For example, Arndt, Mallot and Bülthoff (1995) showed that a reversal of the left and right half images of a stereoscopic image of a shaded sphere does not turn a convex sphere into a concave bowl. Only when texture is added does the sphere turn inside out, but still only reluctantly. Apparently, the prior for convexity is much stronger than the cue provided by stereoscopic viewing of shading. Several models have been proposed of how different cues can be integrated to resolve ambiguities in shape perception (e.g. Landy et al., 1995). Cue accumulation models do not in general take material and illumination ambiguities into account. They do not tell whether, for example, shape perception becomes more stable when cues to illumination are specified. There are no formal models that incorporate all these cues. The question of

36

Product Experience

whether reduction of ambiguity in one of the vertices of the SMI triangle leads to more perceptual stability in the other two vertices, and how that might best be achieved is still largely an open question, though with obvious consequences for disambiguating environments where shape and material information of objects is crucial for picking them up in the correct way, manipulating or recognizing them. For example, conditions that allow for making a good estimate of the BRDF might provide the observer with a clue to the type of material an object is made from, such as metal or cloth, which determines how hard you can squeeze it when picking it up. Other important applications are in virtual or augmented reality applications where the number of visual cues is often limited, and as a result shape or material might be become ambiguous.

5. CONCLUSION In this chapter I have described the roles of shape, material, and illumination for object appearance at three different levels, namely: At the physical level, the relationship between the physical world and the mental world, and finally at the level of the mind. First I discussed the physics and mathematics of shape, material, illumination and color. I made clear that there are inherent ambiguities in visual object perception. For example, the color of an object is not only determined by its reflectance function, but also by its illumination. This is important in conditions where objects have to stand out of the environment irrespective of illumination, such as traffic signs or the clothes of road workers. Traffic signs have to be clearly visible, also at night when the only illumination comes from the sodium streetlights that we discussed earlier. A retro-reflective property of the traffic sign or the clothes of road workers that reflects the headlights of cars back to the source, as well as takes advantage of the more complex light spectrum, is clearly a desirable property at night. Another area in which metamerism is important is where objects have to blend in with the environment, such as in military camouflage. It would make sense to be well aware of the possibility that metamere colors are not metamere under different illumination conditions, when the observer uses a color filter, or even when the observer looks from a different direction. I have also illustrated that ambiguities of shape must fall within a certain class once some assumptions have been made. For example, we have seen how the bas-relief ambiguity is restricted to an affine class of shapes. In the second part, I discussed how physical parameters affect perception. I discussed a method to measure perceived shape. Perceived shape can be significantly altered by changes in material and illumination. An important factor in the perception of material, color and albedo, is contrast. The visual system operates mostly on contrast in luminance, color, etc., rather than its absolute value. For example, when a product needs to draw attention, it is better to increase the contrast in luminance or in the density of the surface texture or in the color, of the product with the environment rather than increasing the overall illumination levels. The luminance, color, etc. of the environment must therefore always be taken into account if one wants to predict what a product will look like, especially where it is important that objects are presented favorably, such as in shop windows and on supermarket shelves. Contrast is also an important factor for improving visibility of products for people with low vision. Rather than increasing the overall luminance, although that might also be beneficial in many cases, increasing the color contrast or luminance contrast with the environment will make the product more visible. In the third part, I discussed how the visual system tries to organize the visual image into a sensible impression of the world. The Gestalt laws describe several factors that

On the visual appearance of objects

37

contribute to a perceptual organization. Proper structuring of the environment can disambiguate perception, overcome luminance contrast illusions, or just as well result in a visual illusion. In the design situation, ambiguity in the perceptual organization might, for example, be exploited in order to create an interesting artistic product, but the designer might also want to avoid ambiguity as much as possible depending on the application in mind. I discussed how the perception of shape, material and illumination might be coupled. This is important because it is a potential way to disambiguate the visual scene. It might be possible to make the material that an object is made of less ambiguous by providing cues to the shape, such as a clearly visible contour and surface texture. Or, observers might be able to recognize that an object is less flat when it is clear to them that the illumination is highly ambient. For example, in interactive virtual reality or augmented reality applications where accurate perception of depth and distance is often required, the presence of visual cues is often limited which may result in a high level of ambiguity under these conditions. However, the ambiguity might be resolved by adding more visual cues. Adding additional independent visual cues to shape, material and illumination such as surface texture and 3D texture, binocular disparity, specular highlights, sharp contours, coherent motion, and regularity in the spatial layout, either in a real or a virtual environment, may all contribute to a more accurate visual perception of the product.

ACKNOWLEDGMENTS I wish to thank the members of the Vision Lab at the University of St. Andrews, and Samira Bouzit of the School of Psychology at the University of Southampton for providing invaluable feedback on the draft version of this chapter.

REFERENCES Anderson, B. L. and Winamer, J. (2005). Image segmentation and lightness perception. Nature, 434, 79–83. Arndt, P. A., Mallot, H. A. and Bülthoff, H. H. (1995). Human stereovision without localized image features. Biological Cybernetics, 72, 279–293. Azzopardi, P. and Cowey, A. (2001). Motion discrimination in cortically blind patients. Brain, 124, 30–46. Belhumeur, P. N., Kriegman, D. J. and Yuille, A. L. (1999). The bas-relief ambiguity. International Journal of Computer Vision, 35(1), 33–44. Blake, A. and Bülthoff, H. H. (1990). Does the brain know the physics of specular reflection? Nature, 343, 165–168. Blake, A. and Bülthoff, H. H. (1991). Shape from specularity: Computations and psychophysics. Philosophical Transactions of the Royal Society of London: Series B: Biological Sciences, 331(1260), 237–252. Cederberg, J. N. (2001). A course in modern geometries. NY: Springer. Christou, C. G. and Koenderink, J. J. (1997). Light source dependency in shape from shading. Vision Research, 37(11), 1441–1449. Claessen, J. P. (1996). Shaped by colour. Doctoral thesis, University of Delft, The Netherlands. Curran, W. and Johnston, A. (1996). The effect of illuminant position on perceived curvature. Vision Research, 36(10), 1399–1410. Drga, V. F. and Harris, J. M. (2005). Distance perception during binocular viewing via stereo-goggles at different display distances. Perception, 34 (Suppl.), 189–189. Erens, R. G. F., Kappers, A. M. L. and Koenderink, J. J. (1993). Perception of local shape from shading. Perception and Psychophysics, 54, 145–157. Flemming, R. W., Dror, R. O. and Adelson, E. H. (2003). Real-world illumination and the perception of surface reflectance properties. Journal of Vision, 3, 347–368. Flemming, R. W., Torralba, A. and Adelson, E. H. (2004). Specular reflections and the perception of shape. Journal of Vision, 4, 798–820. Gibson, J. J. (1966, 1983). The senses considered as perceptual systems. Westport, CT: Greenwood Press.

38

Product Experience

Gibson, J. J. (1979, 1986). The ecological approach to visual perception. Hillsdale (NJ), London: Lawrence Erlbaum Associates. Gilchrist, A. L. (1971). Perceived lightness depends on perceived spatial arrangements. Science, 195, 185–187. Gregory, R. L. (1973). The confounded eye. In: R. L. Gregory and E. H. Gombrich (Eds). Illusion in nature and art, pp. 49–96. London: Duckworth. Hill, H. and Bruce, V. (1993). Independent effects of lighting, orientation, and stereopsis on the hollow-face illusion. Perception, 22, 887–897. Hill, H. and Bruce, V. (1994). A comparison between the hollow-face and ‘hollow potato’ illusions. Perception, 23, 1335–1337. Howard, I. P. (2002). Seeing in depth. Volume 1, Basic mechanisms. Toronto: I. Porteous. Ikeuchi, K. and Horn, B. K. P. (1981). Numerical shape from shading and occluding boundaries. Artificial Intelligence, 17, 141–184. Johnston, A., Hill, H. and Carmen, N. (1992). Recognising faces: effects of lighting direction, inversion and brightness reversal. Perception, 21, 365–375. Khang, B. G., Koenderink, J. J. and Kappers, A. M. L. (2003). Perception of surface reflectance of 3D geometrical shapes: influence of the lighting mode. Perception, 32, 1311–1324. Khang, B. G., Koenderink, J. J. and Kappers, A. M. L. (2004). Shape constancy does not hold for images rendered with different types of material surfaces. Journal of Vision, 4(8), 79–79. Khang, B. G., Koenderink, J. J. and Kappers, A. M. L. (2006). Perception of illumination direction in images of 3-D convex objects: influence of surface materials and light fields. Perception, 35, 625–645. Knill, D. C. and Kersten, D. (1991). Apparent surface curvature affects lightness perception. Nature, 351, 228–230. Koenderink, J. J., Van Doorn, A. J., Christou, C. G. and Lappin, J. S. (1996a). Shape constancy in pictorial relief. Perception, 25, 155–164. Koenderink, J. J., Van Doorn, A. J., Christou, C. G. and Lappin, J. S. (1996b). A perturbation study of shading in pictures. Perception, 25, 1009–1026. Koenderink, J. J., Van Doorn, A. J. and Kappers, A. M. L. (1992). Surface perception in pictures. Perception and Psychophysics, 52(5), 487–496. Koenderink, J. J., Van Doorn, A. J., Kappers, A. M. L., Te Pas, S. F. and Pont, S. C. (2003). Illumination direction from texture shading. Journal of the Optical Society of America, 20(6), 987–995. Koenderink, J. J., Van Doorn, A. J., Kappers, A. M. L. and Todd, J. T. (2001). Ambiguity and the ‘mental eye’ in pictorial relief. Perception, 30, 431–448. Koffka, K. (1935). Principles of Gestalt psychology. London: Kegan Paul, Trench, Trubner. Landy, M. S., Maloney, L. T., Johnston, E. B. and Young, M. (1995). Measurement and modeling of depth cue combination: in defense of weak fusion. Vision Research, 35(3), 389–412. Langer, M. S. and Bülthoff, H. H. (2001). A prior for global convexity in local shape-from-shading. Perception, 30, 403–410. Law-Smith, M. J., Perrett, D. I., Jones, B. C., et al. (2006). Facial appearance is a cue to oestrogen levels in woman. Proceedings of the Royal Society of London B, 273, 135–140. Leeuwenberg, E. L. J. (1971). A perceptual coding language for visual and auditory patterns. American Journal of Psychology, 84(3), 307–349. Leeuwenberg, E. L. J. and Van Lier, R. (2005). Symmetry cues for matching mirrored objects. Spatial Vision, 18(1), 1–23. Longuet-Higgins, M. S. (1960). Reflection and refraction at a random moving surface. I. Pattern and paths of specular points. Journal of the Optical Society of America, 50(9), 838–844. Mamassian, P. and Kersten, D. (1996). Illumination, shading and the perception of local orientation. Vision Research, 36(15), 2351–2367. Mingolla, E. and Todd, J. T. (1986). Perception of solid shape from shading. Biological Cybernetics, 53(3), 137–151. Nefs, H. T., Koenderink, J. J. and Kappers, A. M. L. (2005). The influence of illumination direction on the pictorial reliefs of Lambertian surfaces. Perception, 34(3), 275–287. Nefs, H. T. Koenderink, J. J. and Kappers, A. M. L. (2006). Shape-from-shading for matte and glossy objects. Acta Psychologica, 121(3), 297–316. Nicodemus, F. E., Richmond, J. C., Hsia, J. J., Ginsberg, I. W. and Limperis, T. (1977). Geometrical considerations and nomenclature for reflectance #NBS MN_160. Washington: National Bureau of Standards, Department of Commerce. Norman, J. F., Todd, J. T. and Orban, G. A. (2004). Perception of 3D shape from specular highlights, deformations of shading, and other types of visual information. Psychological Science, 15(8), 655–570. Norman, J. F., Todd, J. T. and Phillips. F. (1995). The perception of surface orientation from multiple sources of optical information. Perception and Psychophysics, 57(5), 629–636.

On the visual appearance of objects

39

Oren, M. and Nayar, S. (1996). A theory of specular surface geometry. International Journal of Computer Vision, 24(2), 105–124. Palmer, S. E. (1999). Vision science: Photons to phenomenology. Cambridge, MA: MIT Press. Pentland, A. P. (1982). Finding the illumination direction. Journal of the Optical Society of America, 72, 448–455. Pont, S. C. and Koenderink, J. J. (2003). Illuminance flow. Computer analysis of images and patterns. Lecture Notes in Computer Science, 2756, 90–97. Pont, S. C. and Te Pas, S. F. (2006). Material–illumination ambiguities and the perception of solid objects. Perception, 35, 1331–1350. Richards, W. (1970). Stereopsis and stereoblindness. Experimental Brain Research, 10, 380–388. Soranzo, A. and Agostini, T. (2006). Does perceptual belongingness affect lightness constancy? Perception, 35, 185–192. Te Pas, S. F. and Pont, S. C. (2005). A comparison of material and illumination discrimination performance for real rough, real smooth and computer generated smooth spheres. Proceedings APGV 2005, ACM SIGGRAPH, 75–81. Todd, J. T. and Mingolla, E. (1983). Perception of surface curvature and direction of illumination from patterns of shading. Journal of Experimental Psychology: Human Perception and Performance, 9(4), 583–595. Todd, J. T., Norman, J. F., Koenderink, J. J. and Kappers, A. M. L. (1997). Effects of texture, illumination and surface reflectance on stereoscopic shape perception. Perception, 26, 806–822. Todd, J. T., Norman, J. F. and Mingolla, E. (2004). Lightness constancy in the presence of highlights. Psychological Science, 15(1), 33–39. Troscianko, T., Montagnon, R., Le Clerc, J., Malbert, E. and Chanteau, P. L. (1991). The role of colour as a monocular depth cue. Vision Research, 31(11), 1923–1930. Tse, P. U. (2002). A contour propagation approach to surface filling-in and volume formation. Psychological Review, 109(1), 91–11. Van der Helm, P. A. and Leeuwenberg, E. L. J. (1996). Goodness of visual regularity: a nontransformational approach. Psychological Review, 103(3), 429–456. Wyszecki, G. and Stiles, W. S. (1982/2000). Color science: Concepts and methods, quantitative data and formulae. New York: John Wiley. Yang, J. M. and Maloney, L. T. (2001). Illuminant cues in surface color perception: tests of three candidate cues. Vision Research, 41, 2581–2600. Zihl, D., Von Cramon, D. and Mai, N. (1983). Selective disturbance of movement after bilateral brain damage. Brain, 106, 313–340.

2

THE TACTUAL EXPERIENCE OF OBJECTS MARIEKE H. SONNEVELD Delft University of Technology, Delft, The Netherlands

HENDRIK N.J. SCHIFFERSTEIN Delft University of Technology, Delft, The Netherlands

1. INTRODUCTION A child’s first tactual experiences with objects mostly involve being touched, for example by the latex gloves of the midwife, the towels she cleans the baby with, and the textiles of the clothes that separate the baby’s skin from its mother. Once children grow old enough to reach out and touch what surrounds them, their tactual experiences become active. They hold, squeeze, and swing whatever comes into reach. Their world becomes an exciting environment, in which they learn to develop their physical skills through manipulating balls, dolls, grandpa’s spectacles, bicycles, skates, and in which they learn how to avoid unpleasant encounters, such as with the sharp edges of the table. When tired of exploring and playing, a cuddly toy may wait for them to keep them company. And although children are aware that they are the active agent in kicking the ball and riding the bicycle, it is not always clear whether they are cuddling the toy or whether the toy is cuddling them; touching becomes interactive. This unavoidable reciprocity is characteristic for the senses of touch. Seeing does not imply being seen, neither does hearing imply being heard. But touching implies being touched simultaneously. Touching and being touched are integrated into one phenomenon; the tactual experience. In this chapter, we provide an overview of the different domains from which tactual experiences can be described and explored. These different domains are summarized in the Tactual Experience Guide that is discussed below (Sonneveld, 2007). A leading motive during the construction of this guide was to enable the description of the aesthetic aspects of tactual experience: What are the influential factors that make objects pleasant or unpleasant to touch? Product Experience Copyright © 2008 Elsevier Ltd.

41

42

Product Experience

Tactual experiences are common to everyday life. Nevertheless, it seems as if people hardly ever talk about them, and they appear to find it very difficult to talk about how objects feel. Suppose that you ask a person to describe how the pen he usually writes with feels. He would probably pick up the pen and start to manipulate it, turn it over, hold it in different positions, consider the different surface textures of the pen by stroking it, swing it between his fingers, possibly slightly hit the table with it, and put down some lines on a piece of paper. Obviously, this manipulation is different from the interaction when actually writing with it, or when carrying it around. Sooner or later the person will become aware of the complexity of the question and will reply: ‘What do you mean, how it feels? When I write with it, play with it, carry it, or what?’ We cannot simply ask people how an object feels, we need to take into account that the answer depends on the nature and context of the interaction with the object. In addition, people seem to lack the vocabulary to describe their tactual experiences of interacting with objects. To support designers as well as users in describing and aesthetically assessing the experience of interacting with objects, Sonneveld (2007) developed the Tactual Experience Guide. This guide helps people to describe their tactual experiences with objects, by offering a consistent framework of the different aspects of tactual experience (the content of the tool), and by offering a format that guides people through this experience (the design of the tool). The framework is based on a qualitative analysis of personal descriptions of pleasant and unpleasant tactual experiences with objects. These descriptions were collected through a printed questionnaire completed by 46 participants, 24 men and 22 women, varying in age from 18 to 67 years. The data analysis resulted in a set of themes that characterize the different aspects of tactual experience. These themes were grouped along the different domains of tactual experience that form the basis of the structure of the Tactual Experience Guide (Figure 2.1) and will be used as the outline of the current chapter: Moving with the object (Section 3); perceiving tactual

movements tactual properties

tactual experience affective behavior gut feelings

sensations

FIGURE 2.1 Overview of the five domains of tactual experience presented as a mind-map structure.

The tactual experience of objects

43

properties of the object (Section 4); sensing physical sensations (Section 5); experiencing the affective behavior of the object (Section 6); and experiencing the gut feelings involved (Section 7). Throughout this chapter, the descriptions of the different aspects of tactual experience will be illustrated with examples derived from this study. This chapter starts out with a description of the meaning of touch, based on a literature overview approaching touch from different perspectives. Next, the five domains of tactual experience are described, complete with background information for each specific domain. The chapter concludes with a section on the development of product designers’ aesthetic sensitivity for the tactual experience in human–product interaction, and recommendations for future research in the field of tactual aesthetics.

2. THE MEANING OF TOUCH 2.1. Touch: Physical encounters and awareness of oneself Physical interaction with the world is not limited to the hands, it involves the whole body. Physical engagement with the world, the awareness of touching and being touched, makes people aware of being a physical body themselves, sharing the physical world with other physical objects. It is within this embodied encounter that the ‘I’ experiences itself and its surrounding world simultaneously, making this encounter the basis for selfawareness (see Bermudez, Marcel and Eilan, 1995, for an overview). Although people can see their body, they need to sense their body to be aware of themselves. Touch allows sensing one’s own body, sensing the borders between the self and the outside world, and the interaction between the two. The neurologist Oliver Sacks emphasized this aspect of the physical experience of the body as the foundation for self-awareness in his descriptions of patients with disturbed self-perceptions. For example, a patient who did not experience his leg as part of himself any more tried to throw the alien leg out of the bed (Sacks, 1984; 1987). Such situations of disturbed tactual sensations may also occur in non-pathological cases such as when waking up with a numb arm or having an anaesthetized cheek at the dentist. When touching these body parts, they feel alien, as ‘not part of me’; they are experienced as ‘dead’ matter. These findings could be summarized in the statement ‘you are what you feel’ (Bergsma, 1987). A world in which touch is poorly addressed is likely to weaken the feeling of being in contact with the world, which may lead to a disappearing feeling of self-awareness. Paradoxically, touch may also blur the boundary between the experienced self and the world. The rubber hand illusion provides a striking example; when one is watching a rubber hand being stroked, while one’s own unseen hand is synchronously stroked, one may attribute the rubber hand to one’s own body and ‘feel like it’s my hand’ (Tsakiris and Haggard, 2005). Another phenomenon that blurs the experience of the border between the self and the outside world is people’s capacity to feel through objects, by ‘incorporating’ these objects as part of their own body (Polanyi, 1967). For example, a blind person feels the world at the end of his white stick, and a carpenter feels the wood through his saw.

2.2. Touch: A foundation for knowledge of the material world In his philosophy on education, Dewey states that the material world people live in and through forms the basis of learning and personal growth, because it is the basis of their

44

Product Experience

‘experiential world’. According to Dewey, one only learns and grows through and from experience, in physical interaction with a material world (Dewey, 1997 [1938]). Physical interaction puts the body, and thereby the tactual senses, back into experience as the foundation of knowledge. People can see the shapes and colors of the physical world, hear the events that take place, smell it, but it is only through touch that people learn about its materiality. The sight and sound of a frog, for example, give some clues for suppositions about its tactual properties, but it is only through touch that one learns about its physicality: Its weight, temperature, wetness, the texture and elasticity of its skin, its force and movements, and so on. The experience of touching a frog (and being touched by it) embodies what learning through touch is about. From this point of view, people actually need touch to get to know and understand the world. Likewise, people need touch to know and understand the man-made objects they are manipulating to grasp their meaning (Lakoff and Johnson, 1999). This knowing through bodily experiencing is different from the knowledge gained through thinking as deduction from theory.

2.3. Touch: A foundation for feelings and emotions Touching is being in physical contact and, as such, is the basis for the feeling of being in contact. Within this contact, touch is a strong basis for the development of feelings of affection and intimacy (Fields, 2003; Montagu, 1971) and is necessary for physical and mental development. Touch during the first phases of life has to be loving and protecting in order for a person to develop into a healthy, empathic human being. It is through touch that one experiences that one is safe and cared for. Experiments with rats have shown that touch-deprivation leads to growth retardation and withering (Montagu, 1971). This need for touch seems so primordial that monkey infants deprived from their mother prefer a terry-cloth surrogate mother without milk to hang on to, over a wire surrogate mother with milk. This is confirmed for people by observations of children who grew up in Romanian orphanages, where touch was infrequent due to understaffed situations. The need for loving touch remains throughout people’s lives, a phenomenon referred to as touch hunger (Fields, 2003). These insights on affective and emotional aspects of touch are based on studies of people touching people, and not on human–product interactions. However, some observations suggest that people’s affective and emotional development and well-being may also be affected by the way they are touched by objects. For example, the experiment with the monkey infants showed that being touched by a non-living terry-cloth surrogate mother contributed to the infant’s well-being. Furthermore, transitional objects such as a blanket or a teddy bear, described by Winnicott (1964) as objects that allow the child to feel safe in a world where the mother is temporarily absent, are illustrations of this affective meaning of touch embodied by objects. Another example is the cuddly walls developed for demented elderly to promote emotional well-being.

2.4. Touch: A communication channel for affection Touch implies contact and thus bodily involvement, whereas seeing and hearing are distant senses and thus are more apt to create distance and ‘objectification’ in social contact. Touch is therefore often considered as our most social sense. Interpersonal touch tells us whether we are safe, cared for and have value (Fields, 2003; Finnegan, 2002). Fagan (1998) suggested that touch is the first language that children learn to communicate

The tactual experience of objects

45

interpersonal affection. Fagan distinguished between various kinds of touch: Ritual touch (e.g. a handshake), athletic touch (e.g. in contact sports), punishing touch, nurturing touch, intimacy-evoking touch and sexual touch. These categories are not mutually exclusive; a formal handshake in a greeting ritual, for example, may feel intimate if it feels softer or lasts longer than expected. To be able to function properly in social contacts, one has to be able to understand (and express) the language of touch properly. Thayer (1982) established a taxonomy of touch based on and illustrated by the relationships people have: A functional–professional type (the doctor touching his patient for examination), social–polite type (the handshake), the friendship–warmth type, the love– intimacy type, and the sexual type. This sequence of categories can be characterized by increasing intimacy: More body parts get involved, longer and more frequent instances of touching, and an increasing variety of the types of touch involved. These social aspects of interpersonal touch that contribute to the meaning of touch may also apply to our relationships with products. The way in which people interact with products may express different kinds of affective relationships, ranging from the functional–professional to the intimate.

3. TACTUAL INTERACTION 3.1. Active and passive touch In human–object interactions, we can distinguish between touching an object and being touched by an object. These two distinct phenomena are referred to as active and passive touch, respectively (Gibson, 1962). Active touch produces a perception of the object being touched: One is exploring the object’s properties (objective pole). On the other hand, passive touch with the same object (being touched by the object) gives an internal sensation: One experiences the sensations in the body, what is being done to the body (subjective pole) (Gibson, 1962). To illustrate this distinction, imagine picking up a glass of wine, handling it in your hands, gently turning it to move the wine: You perceive its shape, its temperature, its fragility, and the movement of the liquid. On the other hand, imagine lying on the bench of a masseur who is putting hot stones on your back: You sense the pressure on your back, the warming of your skin, but you do not sense the shape and the size of the stone. In the active mode, if you focus on your hand you are able to feel the pressure of the glass against your skin, and in the passive mode you may perceive the weight of the stones on your back. Thus touching and being touched occur simultaneously in a physical encounter. Apparently, in actively reaching out to manipulate and touch the world your attention is directed towards the object, whereas in being touched your attention is directed towards the sensations caused by that object. But, in interaction, one can be made aware of both. The difference between touching or being touched is not only a matter of attention and being active or passive. It is also related to the body parts involved in the interaction, because the skin of the different body parts differs in the suitability for active or passive touch. The skin of the palm of the hands and the soles of the feet seems especially suited for touching, whereas the hairy skin, covering the rest of the body, is better equipped for signaling the locus of events touching the body (Bolanowski, 2004). Touching and being touched is not limited to the contact between the body and an object. People have the capacity to touch the environment through other objects (Burton, 1993). Some of these objects are our own non-neural extensions, such as nails, teeth, and

46

Product Experience

hair, in anatomy and physiology referred to as accessory organs (Saladin, 2001). But our interest here focuses on the man-made objects people touch through: People touch the bread through the knife they cut with, they touch the road through the bicycle they are riding on, the tennis ball through the racket they hit with, and the paper through the pen they are writing with. Although touching through intermediate objects may be different from direct touch (Lederman and Klatzky, 1999), it forms an interesting part of people’s everyday tactual experiences. In the following paragraphs, we first discuss the properties one perceives when actively touching objects, followed by the sensations perceived when being touched by objects.

3.2. Exploration strategies When you start to think about it, it is an astounding achievement of tactual perception that we perceive and recognize an object as one object. For example, when holding a glass of wine the skin is touched at different places with different pressure intensities. Joints, muscles and tendons have specific positions and apply specific forces to different parts of the glass. Therefore, it is remarkable that all these impressions are integrated into the perception of one glass of wine. This capacity of identifying an object as one object can be fooled: Aristotle discovered that rolling a pen over two crossed fingers with one’s eyes closed gives the impression of rolling two pens over the fingers. Nevertheless, generally when people use only their sense of touch, they identify the integrity of objects correctly. When people are presented with an object, they typically first try to identify it: They want to know what it is (e.g. see Gibson, 1962; Schifferstein and Cleiren, 2005). Empirical studies have shown that the tactual (haptic) system is quite rapid and accurate in recognizing three-dimensional, familiar objects. Klatzky, Lederman, and Metzger (1985) demonstrated that blindfolded subjects recognized 96% of common objects within 5 seconds, and 94% in 1–2 seconds. Tactual perception of an object and its properties is dependent on movement: It is through movement, in interaction, that people perceive tactually. Klatzky et al. (1985) studied the movements made by blindfolded people when physically exploring tactual properties of objects, and concluded that people use specific exploratory movements for the perception of specific tactual properties. In addition, these authors found that for a consistent tactual perception of the object, tactual scanning should be done in a systematic way. These tactual scanning strategies have to be learned: Experience in touching is needed to be able to explore efficiently and accurately (Davidson, 1985). Figure 2.2 shows an overview of the exploratory procedures distinguished by Klatzky et al. (1985) and indicates the corresponding tactual properties these procedures are specifically suited for. Turvey (1996) observed and researched ‘dynamic touch’ as an exploratory procedure. This procedure implies that people swing objects to ‘get a feel for them’. Dynamic touch is functional for the perception of geometrical properties, size, and weight, but especially appropriate for determining the moment of inertia of an object: Its reaction to rotation. Dynamic touch forms the basis for understanding how to use an object as a tool: How to hit a nail with a hammer and where to hit the ball with your tennis racket.

3.3. Motivations to move To understand how people interact with objects, it is important to understand why they interact in the way they do (Hodgson, 2001). In other words, we need to understand

47

The tactual experience of objects

Lateral Motion (Texture)

Pressure (Hardness)

Unsupported Holding (Weight)

Enclosure (Global Shape; Volume)

Static Contact (Temperature)

Contour Following (Global Shape; Exact Shape)

Moving Parts

FIGURE 2.2 Overview of the exploratory procedures that people use to determine specific tactual properties of objects (adapted from Klatzky et al., 1985).

their motivation for interaction. This motivation is evidently elicited by and reflected in the function of a product, but it is not necessarily limited to that function. In this Section, we give an overview of the different motivations we encountered in studies on tactual experience (see Figure 2.3): • Interaction for practical, functional use, as a tool: In many cases, objects are used with a functional goal, as a tool. The intention of the interaction is directed towards the outside world, for practical reasons. Evident examples are the use of scissors to cut paper, the use of a knife to slice bread, the use of a camera to take a picture, or the use of a car to get somewhere. However, the functional goals do not necessarily align with the goals the object was intended for. For example, scissors may be used to open a jar of paint. • Interaction to play: Another form of interaction occurs when the object is used for non-functional reasons, for playing in the broadest sense of the word, including playing sports or just messing around. Some objects are actually intended to play with; in this case the motivation is inherent to the object’s function. Examples are tennis rackets and yo-yos. But many people play with objects that were not initially meant to play with. This playing has a specific character: It involves physically moving and interacting with the object, just for the sake of the resulting sensation. It is sometimes referred to as ‘thoughtless’ playing with the object. However, in some cases the playing may involve a

48

Product Experience

to play? to explore?

to use?

how do I move … to take care? by accident?

to carry?

FIGURE 2.3 The map on Movements presents the different motivations people have to interact with an object: Why do you touch the object and how do you move?

challenge, like tearing up a coaster into as many parts as possible. In addition, playing may involve an element of cuddling, like when you stroke a pen against your skin, just because it feels so good. • Interaction to care for and to be taken care of: A specific theme in the interaction with objects is ‘to take care’. First, objects are used by people for personal care, that is, to brush one’s teeth, or to comb one’s hair. Furthermore, supporting someone can be regarded as a way of taking care; in that sense, chairs and beds take care of people. Again, this taking care can be the object’s primary function, like a coat that keeps someone warm, or a chair that supports, or it can be sought for independently from its function. For example, you can hold a coffee mug against your cheek to warm yourself. A second form of taking care is when people take care of the object: To wash it, repair it, store it, and so on. In other words, ‘taking care’ is a mutual aspect of the human–product interaction. • Interaction to explore: Regardless of its function, an object can be touched for the sake of exploring it, because it is unknown and a person wants to discover how it feels. This motivation is not necessarily restricted to unknown objects. A familiar object can also be touched just for the sake of touching it, to make contact with it. • Interaction to carry: Some interactions derive from the fact that objects are movable or portable, which involves a specific kind of interaction: Carrying the object. This carrying can be done in different ways: In a pocket, on your back, in your hands, and so on. • Interaction by accident, by coincidence: Furthermore, some interactions are not intentional or prompted by a specific motivation, but just happen by accident: People accidentally sit on something, or bump into something.

49

The tactual experience of objects

4. TACTUAL PROPERTIES OF OBJECTS Although people perceive an object as a whole, rather than as the sum of its different properties, this paragraph focuses on the perception of the different properties (Figure 2.4). For each property, the characteristic exploratory movements (see Section 3.2, Klatzky et al., 1985) are described, together with the possible dimensions on which people perceive these properties. These perceptual dimensions have been derived from experiments using sorting tasks, where participants sorted materials on the basis of their tactual similarities (e.g. see Giboreau, Navarro, Faye and Dumortier, 2001). Overall, tactual properties can be related to: The substance, i.e. the materials the object is made of: Its hardness, elasticity, plasticity, temperature and weight. The surface of the object: Its texture and patterns. The structure, i.e. the geometrical aspect of the object: Its global shape, exact shape, volume and weight distribution (balance). The moving parts of the object: The way the parts move in relation to one another.

elasticity?

movements?

force?

hardness?

material properties?

moving parts?

texture? how do I perceive its ...

geometry?

temperature?

shape? weight? size/ volume?

balance?

FIGURE 2.4 The map on Tactual Properties of an object presents the different properties perceived in tactual experience: How do you perceive the object’s tactual properties?

50

Product Experience

4.1. Hardness, elasticity and plasticity The hardness, stiffness and elasticity of an object’s materials are explored when people exert pressure on the object, for example when they squeeze the object. Other possible movements are pulling, pushing on it, and bending or wrenching it. These movements have in common that they try to transform the object. These properties evaluate the material’s resistance against, or compliance to, transformation: Hardness and softness are explored when exerting pressure (Klatzky, Lederman and Reed, 1987), and stiffness and flexibility, are explored when bending and wrenching (Ashby and Johnson, 2002). Once the material is transformed, the material’s elasticity and springiness depend on the way the material behaves when the pressure is released: Does it come back to its initial shape, or does it stay transformed? If it returns to its initial shape, the material is perceived as elastic. If it remains transformed (e.g. clay), this material property is referred to with the term ‘plasticity’.

4.2. Temperature People react differently to temperatures that are extremely high or low, compared with, temperatures that are close to their body temperature. Very high and very low temperatures are perceived instantaneously at contact, causing a strong withdrawal reflex. Temperatures that are not threatening need time to be perceived: People will leave their hands on a location for a while to perceive the difference between their body temperature and the temperature of the object. People perceive the warmness or coldness of objects, because objects with a temperature above or below body temperature cause a temperature flow. An object feels cold if it extracts warmth from the skin. To produce this sensation, the object does not only need to have a temperature below body temperature, it also has to extract warmth at a fast rate: The material needs to have low temperature resistance. Examples of such materials are glass and metal (Ashby and Johnson, 2002). Materials with high temperature resistance, such as wood or plastics, generally feel ‘warm’ even if their temperature is below body temperature. Due to the process of temperature flow, the temperature of objects changes over time when they are held, which eventually leads to neutral thermal perception (not warm, nor cold). The larger the difference between the object temperature and the skin temperature, the more accurate the temperature perception is (Tritsch, 1988).

4.3. Texture and patterns Texture is related to the properties of the material the object is made of and to the structure of the surface resulting from production techniques and surface treatment. Surface texture includes patterns, such as structured or randomly distributed details. Texture is explored through stroking the surface of an object. Stroking is necessary for the detection of fine surface textures (Hollins and Risner, 2000). Textures with coarser patterns may also be perceived through static touch (Lederman, 1981). In addition, surface texture is perceived when holding an object, thereby assessing the grip on that object (friction). Texture perception is probably one of the most studied tactual phenomena (Craig and Rollman, 1999), but researchers do not agree on an unequivocal set of perceptual dimensions for surface texture. Hollins et al. (1993) found that participants judged texture on three dimensions, the first two of which were the most important: Rough/smooth and soft/hard. The third dimension was related to the elasticity (‘springiness’) of the surface. In a follow-up study, the first two dimensions were again found, but the third

The tactual experience of objects

51

dimension was now defined as sticky/slippery (Hollins et al., 2000). Hollins et al. found bumpy/flat as an additional fourth dimension, but this dimension was not independent from the first three. In both experiments, a set of widely varying materials was used, such as sandpaper, velvet, and wood. Picard et al. (2003) investigated the perceptual dimensions of everyday tactual textures of textiles, and came up with the following set of four dimensions: Soft/harsh, thin/thick, relief/no relief, and hard/mellow. Some of the differences in outcomes between these studies may have depended on the presented materials. The perception of the roughness of a surface is not equal for all body parts: The lips and the fingers are most sensitive, while the heel, the back, and the thigh are least sensitive (Stevens, 1990). Furthermore, roughness perception is dependent on the way one moves. For example, roughness perception becomes more intense when the applied finger force increases (Lederman, 1974). For practical applications, the grip of objects is very important. The grip is mainly dependent on the degree of friction between hands and objects, which is influenced by the condition of the skin, such as dry or sweaty, and the amount of dirt on the skin. Slightly wet hands offer a greater friction force, which is why people spit on their hands before executing a task requiring firm grip. But too much water (or perspiration) forms a layer between hand and object, which causes slipping. The use of soap reduces the coefficient of friction of the hands (O’Meara and Smith, 2001). For soapy hands, textured surfaces offer the best grip, whereas for dry hands, smooth materials perform best (see Bobjer, Johansson and Piguet, 1993; Buchholz, Frederick and Armstrong, 1988). Friction is also dependent on the size of the contact area: The larger the contact area, the more friction (Highley, 1977).

4.4. Shape and size of the object Geometrical properties of objects are explored when grasping the object, holding it, manipulating it, and following the contours with the fingers. In addition, the size and shape of bigger objects are explored through dynamic touch, by swinging and wielding them. Shape discriminations can be based on the following characteristics (Lederman and Klatzky, 1987): (1) abrupt surface discontinuities, such as edges (no edge versus edge) and holes (hole versus no hole, shallow hole versus deep hole); (2) continuous 3D surface contours, such as curved versus flat; (3) orientation of surfaces (horizontal, vertical, slant). The importance of various shape properties seems to differ when they are perceived through touch compared to when they are perceived through vision. For example proportion, a typical aspect of visual perception, is not directly nor spontaneously perceived through tactual exploration (Appelle, Gravetter and Davidson, 1980). Changes in curvature are considered very important when perceiving visually, but changes in orientation of the object (its position in space) are not. This is quite the opposite when perceiving through the tactual system (Goodnow, 1969). Curvature perception depends on the direction of the scanning hand’s movement: Symmetrically curved edges often feel skewed in the direction of the moving hand (Goodnow, Baum and Davidson, 1971). In addition, shape perception is influenced by what has been perceived previously, a so called after-effect. For example, after prolonged perception of a concave surface, a flat surface is perceived as convex, and vice versa (Vogels, Kappers and Koenderink, 2001). Size in tactual perception is referred to as volume, length and width (Lederman and Klatzky, 1987). After-effects have also been described for the perception of size. After prolonged perception of an object with a certain length, longer objects are perceived as shorter and shorter objects as longer than their actual length. In general, compared to

52

Product Experience

the visual space, the tactual space seems smaller. Thus objects that were first touched and then seen look smaller than expected on the basis of touch (Teghtsoonian and Teghtsoonian, 1970).

4.5. Weight and balance An object’s weight is explored by holding the object, and by moving it up and down. Weight distribution is explored through dynamic touch, by swinging and wielding the object, or by trying to hold it still in a specific position (Turvey, 1996). Kreifeldt (2001) studied the object’s moment of inertia, perceived as an object’s resistance to rotational movement. This moment of inertia depends on where you hold the object. He illustrates this with the example of a baseball bat. The bat has a specific weight and a specific centre of gravity, but the way it is experienced depends on where you hold it: At the hitting hand or at the holding end. Weight is perceived as heavy or light, and weight distribution as balanced or unbalanced. Charpentier was the first to demonstrate, in 1891, that the perceived weight of an object depends not only on its physical mass but also on its size (Murray et al., 1999). When holding two objects of equal mass but of different size, subjects will consistently report the bigger one as lighter (Murray et al., 1999). This illusion also occurs in purely tactual situations, where the subjects are not able to see the objects (Amazeen, 1997). However, this effect emerges only when the grip on the objects is loose; it does not occur for firm grip (Ellis and Lederman, 1999). The perception of weight is influenced by what has been perceived previously. After prolonged holding of two objects of different weight in each hand, the weight of two objects of the same weight is estimated as different (de Mendoza, 1979). Weight perception is also influenced by the temperature of the object. For all body parts, cold objects rested on the skin feel heavier than thermally neutral ones. Warmth intensifies weight perception as well, but this effect is not present in all body parts. For example, it is present on the forearm, but it does not occur on the forehead (Stevens, 1980). Weight perception also seems to be influenced by other tactual properties of the object. For example, when lifting an object with the distal pads of the thumb and index finger at its sides (precision grip) the perceived weight depends on the object’s surface texture: The smoother the texture, the heavier the object is perceived. This is explained by the fact that to prevent the object from slipping, a greater normal force has to be applied when the object’s surface is smoother (Flanagan et al., 1995). Objects may consist of various constructions and mechanisms, and of moving parts. In the field of Human Factors, movement in human–product interaction has been studied extensively. This has resulted in an overview of the different types of grips used when holding an object (for example precision grip, force grip, antenna finger), different types of movements (for example translation, rotation), force exertion, and related types of control mechanisms (for example buttons, handles, wheels). For an overview of the resulting descriptions the reader is referred to MacKenzie and Iberall (1994).

5. TACTUAL SENSATIONS: BEING TOUCHED BY OBJECTS Like tactual perceptions, tactual sensations depend on the movements one makes when touching an object (Gibson, 1963). However, sensations and perceptions differ in the way they emerge from touching an object: The perception of an object may be considered as invariant throughout moving, whereas tactual sensations vary while moving. To illustrate

53

The tactual experience of objects

this, consider touching a wooden cube. You will perceive its shape through enclosure and contour following, moving the cube around in your hands. Throughout these different movements, the perception of the shape is invariant: It is a wooden cube. But this is not the case for the tactual sensations involved: The pressing of the edges of the corners, or of the flat surfaces of the cube on your skin varies with every movement of the hands and with every position of the cube. Thus the perception of the tactual invariant properties of an object coincides with varying sensations (Gibson, 1963). A physical property of an object may evoke many different sensations, depending on the way one interacts with it. This Section presents the skin and body senses and the related tactual sensations (Figure 2.5).

5.1. The skin and the skin senses Our skin is our largest organ: In adults it has a surface of 1.5 to 2 m2, is 0.5 to 4 mm thick (depending on the body part), and amounts to about 15% of total body weight (Saladin, 2001). Two types of skin cover the body: The glabrous (hairless) skin of the palms of the hands and the soles of the feet, and the hairy skin covering the rest of the body. The two types of skin seem to be equipped for different functions: The glabrous skin is suited for active touch in exploring and manipulating the world, and the hairy skin is suited for passive touch in signaling the locus of events, because of the following differences (MacKenzie and Iberall, 1994): • The glabrous skin is thicker (especially the epidermis), tougher, and more resistant to pressure. • The epidermis of the glabrous skin contains fat pads on the fingers and the bulges on the palm of the hand. These fat pads make the skin comply with the grasped object, thus facilitating a stable grip.

vibration?

itch?

body postures ? how do I sense ...

body temperature?

muscle force? pressure? pain?

FIGURE 2.5 The map on Bodily Sensations involved in tactual experience: What do you sense in your body when touching the object?

54

Product Experience

• The glabrous skin has a papillary structure: The epidermal ridges form the palmar print and the fingerprints. This structure has a sensory function: It allows the sensors to register lateral pressure. Hence, they contribute to the accuracy of the sense of touch. Furthermore, the ridges are important in grasping. They offer more grip on the grasped object, much like the profile of tires offers more grip on the road. • The distribution of the sweat glands is denser in the glabrous skin of hands and feet than in other parts of the body. The glands also differ in the way they respond to stimuli: The glands in the hand respond more to applied force (thus again facilitating grip), whereas the glands in the hairy skin respond more to temperature, thus facilitating temperature regulation. • The hairy skin lacks the Meissner’s corpuscles, responsible for the sensations of light touch and vibration. Therefore, people are unable to perceive subtle tactual details such as texture differences with body parts covered with hairy skin. Both skin types consist of three different layers, the epidermis, the dermis and the hypodermis containing a variety of sensors: Mechanoreceptors, sensitive to mechanical transformation of the skin; thermoreceptors, detecting cooling or warming of the skin; nociceptors, involved in the sensation of pain when the skin is (almost) damaged. Once the skin sensors are stimulated, neural fibers conduct this sensory information to the central nervous system. For each type of sensor, the corresponding neural fiber can either be slowly adapting (firing continuously) or rapidly adapting (only firing when the stimulus changes). These differences in adaptation rate strongly influence the duration of tactual sensations. Some sensations vanish rapidly because the sensor is adapted to the new situation (for example small temperature differences), whereas other sensations will last over time (for example deep pain) because the sensors do not adapt at all.

5.2. The skin sensations The skin sensations can be divided into three types, according to the type of sensors involved (Saladin, 2001; Vander, Sherman and Luciano, 2001): 1. Touch sensations and sensations derived from touch, such as superficial and deep pressure, and vibration (mainly mechanoreceptors involved). 2. Warm and cold sensations (mainly thermoreceptors involved). 3. Pain sensations (mainly nociceptors involved). Overall, sensations can be considered on the aspects of location (where), quality (what), intensity (how strong) and duration (how long does it last) (Gibson, 1963). We distinguish between the following types of skin sensations here (Figure 2.5): • Light touch: Light touch is what one senses when being touched without the skin being deformed. Light touch is mostly detected by rapidly adapting sensors (Saladin, 2001; Vander et al., 2001). This rapid adaptation allows people to forget about the clothes that touch them during the day. • Pressure: Pressure is maintained touch. It is experienced when an object is pressing on one’s skin and, thereby, deforming the skin. Pressure sensors are slowly adapting. That is why the sensations of deep and heavy pressure are usually difficult to ignore. • Vibration: Vibration in the skin is experienced when rapidly adapting touch sensors are stimulated rhythmically, for example when the hand strokes a texture, or when one sits on a chair and a truck is driving by, causing vibration of the floor. Receptor organs in the upper layer of the skin are sensitive to low frequency stimulation, the deeper receptors to high frequency stimulation (Sekuler and Blake, 1994).

The tactual experience of objects

55

• Cold and warmth: Although people intuitively consider warm and cold two opposites of one dimension, the sensations of warming and cooling of the skin are elicited by two different sensory systems. The skin easily adapts to temperatures between 20 and 40ºC, thus resulting in a thermal neutral perception of the object. Below 20 and above 40ºC, one perceives a cold or a warm object, but the sensation does not adapt over time. Above 45ºC, the tissue starts to be damaged, and the sensation becomes one of pain (Ganong, 2001; Sekuler and Blake, 1994). • Pain: When pain is induced by stimulation of the skin it is called superficial pain. Pain from muscles, bones and joints is called deep pain. Pain as a bodily sensation is referred to as somatic pain. The subjective experience of pain is a complex phenomenon; it differs from the other senses because it is intimately related to the affective meaning of the circumstances in which it is experienced, and the motivations of the subject. An extensive discussion on pain perception can be found in Kruger (1996). • Itch and tickle: Itch and tickle are skin sensations elicited by stimulation of nonmyelinated fibers in the skin that are highly similar to pain sensors. Itch and tickle are produced by mild stimulation when moving something across the skin (Saladin, 2001; Vander et al., 2001). Itch can also be produced by chemical stimulation of the skin. The difference between tickle and itch is not clearly defined. • Physical pleasure: Olausson et al. (2002) discovered a system of non-myelinated, slow conducting sensors in the hairy skin, that responds when a person is being touched lightly, producing a faint sensation of pleasant touch, without producing the sensation of being touched. Therefore, it seems plausible to consider the experience of physical pleasure as a physical sensation per se. The researchers suggested that people might possess a special system for limbic touch, underlying emotional responses to caress-like skin-toskin contact between individuals. Because the study used a soft brush as a stimulus, this system is not confined to interpersonal touch, but includes touching by objects.

5.3. The body senses and sensations In addition to the skin sensations, active touch involves the two body senses: Proprioception that perceives body posture, and kinaesthetics that perceives body movement (Figure 2.5). Body posture and body movement are sensed through sensors in muscles, tendons and joint tissues (Saladin, 2001; Vander et al., 2001).

5.4. Tactual sensitivity Tactual sensitivity relates to the capacity to sense if one is touched, where one is touched, for how long, and with what intensity (Lederman and Klatzky, 1998). Because the sensors in the skin are not equally distributed, tactual sensitivity depends on the locus on the body. The fingertips and lips contain most sensors per mm2 (Stevens, 1990). In addition, sensitivity depends on the spatial aspects of the afferent neurons. Receptive fields of fibers in the upper layer of the skin are relatively small (2–4 mm) and overlapping. Thus they form a sensitive system to locate a point on the skin. The deeper receptors have larger receptive fields, making the location of a point on the skin less accurate (Sekuler and Blake, 1994). Finally, sensitivity is related to the size of the reception areas in the somatosensory cortex. These areas are not proportional to the different body parts: The lips and hands cover the largest area, whereas the back and the calf cover relatively small parts. These differences in sensitivity are reflected in the homunculus defined by Penfield (Sekuler and Blake, 1994) that depicts the different body parts proportional to their reception areas in the cortex (Figure 2.6).

56

Product Experience

FIGURE 2.6 Penfield’s Homunculus.

The sensitivity due to the distribution of sensors and to the spatial characteristics of the corresponding neurons throughout the body cannot be altered by training. However, the size of the reception areas in the brain can be enlarged by training and experience. For example, blind people do not possess better discriminative abilities than sighted people (Hanninen, 1972), but they have become better in recognizing objects (Berla and Butterfield, 1977) and patterns (Craig, 1988) through touch. Tactual sensitivity is not a static aspect of an organism, but varies in time. Sensitivity decreases through aging. For example, the spatial acuity of the skin of the fingertip deteriorates with age (Stevens and Choo, 1996). Various diseases or pathological conditions can disturb the tactual senses, such as diabetes, leprosy, multiple sclerosis and Parkinson’s disease (Pratorius, Kimmeskamp and Milani, 2003). Damage of the central nervous system or peripheral nerve tissue through accident or tumors may cause a loss of sensitivity as well (Franzen and Lindblom, 1976).

6. THE BODY LANGUAGE OF OBJECTS The affective aspects of a tactual experience with an object can be characterized as the experienced body language of the object. In physical interaction objects are experienced as expressing affective behavior through their physical reactions to our actions. This affective behavior can be described along a number of themes that are discussed below (Figure 2.7).

6.1. Personality A person’s personality is sometimes characterized by words typically used to describe material properties that can be perceived tactually: People are experienced as weak, strong, hard, soft, flexible, rigid, warm, or cold, and so on. Conversely, objects may be regarded as entities expressing personality. In this case, people talk about objects as if they possess human characteristics. For example, they may say about an object that ‘it is obeying, but with dignity’, or that ‘it feels strong and playful’. According to Govers (2004), product personality can be defined as the set of human characteristics that people use to describe an object. What makes the tactual characteristics particularly interesting in this respect is that people seem to transfer the perceived tactual qualities

57

The tactual experience of objects

transparency?

physical skills? power match?

feedback?

perfect match?

how do I experience its ...

emotions?

personality?

familiarity? intentions?

FIGURE 2.7 The map on the Affective Behavior of an object involved in tactual experience: How do you experience the object’s affective behavior?

directly to the product personality: A cold object expresses a cold personality, and a flexible product may be experienced as a flexible personality. Because objects seem to have a personality, they seem to become social entities and, consequently, may evoke feelings and emotions that usually only apply to the interpersonal domain, such as feelings of sympathy. For example, people may report that they feel sorry for an object that seems sad, because it has a broken part.

6.2. Intentions People have specific motivations to interact with objects (Section 3.3), and people experience objects as having intentions as well. Sometimes an object seems to have a will of its own: It wants (or refuses) to be explored, to take care, to cooperate, to play, and so on. This perceived intentionality emerges because objects seem to display a specific physical behavior in interaction. People move in order to experience an object. In return, when the object moves this is experienced as meaningful, intentional behavior (Heider and Simmel, 1944; Michotte, 1963). Thus when an object does not react, it is experienced as refusing, rejecting. And when a chair collapses under somebody’s weight, that chair is experienced as letting that person down. Sonneveld (2007) found the following intentions in participants’ descriptions: • Wanting to be touched and explored (e.g. ‘the balloon had some kind of “I don’t want to be in your hands” reaction’). • Wanting to cooperate (e.g. ‘my walking shoes are much too heavy, they feel like they do not want to walk at all, they are too tired to do so’).

58

Product Experience

• Wanting to play (e.g. rubber bands, springs, and some on/off switches elicit repetitive movements, because they seem to ask you to go on playing with them). • Wanting to take care of and, related, wanting to love or hurt somebody (e.g. objects that literally support and take care of people due to their function, such as furniture and clothes).

6.3. Integrity: Tactual feedback In interaction, an object provides people with information. On the one hand, it provides information about itself, for example about its properties, about what it is and what it is doing. On the other hand, it provides information about the physical world around it, about what is going on. With the information the object supplies, it can guide people in what they are trying to achieve, but it may also mislead them. Therefore, the way in which objects give tactual feedback is experienced as the integrity of the object. To start with, integrity is related to whether the object gives any feedback at all: Products can be rich or poor in tactual information. For example, touch screens do not let the user feel what they are actually doing, whereas other interfaces, such as steering wheels of cars, let you know exactly what is going on. Next, if objects give feedback about what is happening, they may or may not seem honest. For example, coffee mugs of porcelain provide information about the temperature of the coffee inside, whereas polystyrene foam cups do not. In contrast to the porcelain cups that provide the correct information, people feel fooled when the coffee in the polystyrene cup is much hotter than expected. How tactual feedback is experienced is largely dependent on the expertise of the user: An experienced car mechanic will get a lot more information through manipulation of the different parts of an engine than a layperson. The appreciation of the integrity of objects is not unequivocal; being teased or fooled may be part of a pleasant experience, depending on the context of the interaction. For example, in the case of a toy crocodile the lack of integrity makes it exciting: The crocodile bites when you press one of his teeth, but you never know which tooth it is going to be.

6.4. The perfect match When touching an object, people evaluate the way it fits them. People seem to enjoy the feeling that something feels perfectly right: The perfect match. The degree of match is primarily determined by the object’s geometrical properties (Section 4.4), such as the fit of shoes and clothes, the shape and balance of tools, and the temperature of a shower. In addition, the degree of match can be related to dynamic properties, such as when the object moves with you as if it dances with you. A perfect match may be experienced immediately during the first encounter, or it may emerge from an intensive interaction over time. For example, a fountain pen adapts to the writer’s hand, and scissors adapt to the hairdresser’s hand. Over time, these objects become so well adapted to the owner’s hand that they seem impossible to work with for others.

6.5. Familiarity: Feeling ‘mine’ or alien When an object is touched frequently, it becomes familiar, it feels like one’s own, and one recognizes it as such. In contrast, an unfamiliar object may be experienced as ‘alien’ or ‘someone else’s’. Being new and unfamiliar can be a burden: People reported that as a child they had difficulties in accepting new clothes, because they did not feel as ‘own’.

The tactual experience of objects

59

This theme differs from the perfect match in the sense that familiarizing does not necessarily lead to a perfect fit, it just increases the familiarity of the object. Vice versa, if the size is right an unfamiliar object may already fit perfectly at the first encounter. Familiarity grows over time; the memories of past experiences contribute to the current experience. In addition, touch allows people to recognize objects in an affective, intimate way. One person said ‘I was wondering if it was my grandmother’s cupboard that I used to play with. When I let my hands glide across the surface of the woodcuttings, I suddenly knew: It’s hers. This tactual memory was very strong’. On the other hand, to feel somebody else ‘through’ an object can be a bewildering experience, because it evokes unwanted feelings of intimacy. For example, when you sit down on a toilet seat that is still warm from the person who sat there just before you, it feels like the seat was not yet ready to be used again; it almost feels like a strange, intimate contact with somebody else. Feelings of familiarity may be enhanced by elements that make the product unique (see Chapter 17). First, an object may be unique in shape, texture, or other details, which allows the user to recognize it as his ‘own’ object. Next, small changes that occur over time due to the interaction with the object, such as small dents, scratches or other imperfections, allow the user to recognize the object as the familiar object.

6.6. Power match and being in control A physical encounter may elicit an assessment of power: Who is the strongest? Some products literally challenge people to arm wrestling, like a marmalade jar with a tight lid. Who does not want to win, and stick the lid with triumph in the air once the jar has been defeated? In addition, the power match addresses the question about who is in control in the interaction: Who leads whom? Power seems to be mainly explored in the first encounter. It is not only about winning and being the strongest, but also about exploring limits: How far can I go? How much can I bend this stick? How far can I stretch this band? Furthermore, this theme assesses the requested physical effort one has to invest in the interaction. It can be overwhelming to sense the power one has, to feel that one is completely in control, for example in driving a car. But the opposite can be exciting as well: Luna park attractions are exciting because people experience that they are completely out of control.

6.7. Challenge of developing physical skills Physical interaction involves physical skills. Objects differ in the way they challenge people to develop their skills. Some objects require great skills, such as musical instruments, and some do not, such as objects with push buttons. People may have to invest a lot of effort to develop the required skills, for example in the case of learning to play the guitar. Once acquired, the practice of a physical skill can be a source of pleasure, of flow (Csikszentmihalyi, 1990). People are programmed biologically to like to develop their physical skills and to exercise them (Veenhoven, 2006). To develop skills, one has to develop ‘tactual knowledge’ through interacting with the objects. For example, a masseur needs to ‘know’ from experience the human body in its different qualities, to be able to work with it. Likewise, a sculptor needs to ‘know’ the material he is working on from experience. This aspect is closely related to the theme of tactual feedback. Objects differ in the way they allow people to develop a personal style when developing these skills, often referred to as doing things with a ‘personal touch’. Some objects

60

Product Experience

prescribe skills in a rather straightforward way, such as pressing the buttons on a microwave oven. Other objects allow more freedom to develop one’s own style, such as making cocktails with a cocktail shaker. In contrast, some objects seem to have their own, personal user manual that only the user knows. This is a particular form of physical skill and of ‘knowing’, because it has nothing to do with an established skill. This skill is specific for objects with stubborn personalities that can only be managed in a particular way. The theme of physical skills is closely related to the theme of power. When a person needs physical skills to interact with certain objects, the objects will have power over that person as long as (s)he has not developed these skills. For example, a car seems uncontrollable when you do not know how to drive it, but you gain control over it over the course of your driving lessons.

6.8. Attention: Tactual transparency and tactual noise When touching an object, people are in contact with that object, but their attention is not necessarily directed towards the object. The theme ‘transparency’ refers to the capacity of the object to allow people to feel through the object, to incorporate it, and to direct their attention to something else in their environment. For example, when riding a bicycle, you feel the road surface through the handle-bars and through the saddle. However, an annoying tactual sensation can create ‘tactual noise’ that reduces the degree of transparency, for example, when the handles get sticky they will attract the majority of attention and, thereby, divert the attention that was first directed towards the road. Condoms and surgical gloves with true touch are typical examples of products for which manufacturers have tried to decrease tactual noise, but nevertheless these objects may stay in one’s focus of attention. Condoms are a good example of the dual design possibilities of tactual transparency; either they are designed as tactually transparent as possible, or the designer accepts the fact that people perceive them anyway and tries to design a particular tactual experience.

6.9. Conclusion on the themes The previous paragraphs discussed the different themes involved in the body language of objects. Although some themes are more relevant than others to understand and describe specific experiences, they are all useful to describe the different aspects of a tactual experience as a whole. The themes should, therefore, be seen as different aspects of the same phenomenon. Also, they are not mutually exclusive, but they are related to each other. Thus, whatever the perspective taken, the other themes will emerge as context.

7. THE FEELINGS INVOLVED IN TACTUAL EXPERIENCE In human–product interaction, the experienced affective behavior of the object is reflected in the feelings elicited by this behavior. These different types of feelings are described below (Figure 2.8): • Physical pleasure: Lust and pain or disgust. Obviously, pleasantness in tactual experience is related to lust and physical pleasure. The pleasure is generally characterized by descriptions like ‘it feels good’, with superlatives such as ‘delicious’ and ‘delightful’. When people are asked to focus on tactual experiences, they are often amazed by the

61

The tactual experience of objects

bodily reaction? self experience?

physical pleasure? what is my ...

action tendency?

vulnerability?

energy?

affective reaction?

FIGURE 2.8 Map on Gut Feelings involved in tactual experience with an object: What feelings do you have when touching the object?

pleasantness of the experience: ‘It was even better than I thought it would be’. The physical pleasure does not seem to have one single counterpart, some descriptions of unpleasantness involve pain (e.g. ‘painful’, ‘it hurts’, ‘it stings’), some refer to light or strong discomfort (e.g. ‘uncomfortable’, ‘annoying’, ‘terrible’), while others contain elements of disgust (e.g. ‘repulsion’, ‘a dirty feeling’). People did not describe their feelings of physical pleasure or pain and disgust elaborately; these descriptions often missed nuances. In many cases, these feelings were exemplified by expressive sounds, such as ‘wow’, ‘mmmm’, ‘ugh’, or ‘yuk’ and the experience seemed to be grounded in a physical reaction accompanied by bodily responses, such as goose bumps, shivers, and feelings of nausea. Physical (un)pleasantness should not be confused with the overall assessment of the experience, because in some cases physical unpleasantness is nevertheless appreciated. As one person responded: ‘Its sliminess was disgusting, but at the same time that was also what made it attractive to touch’. And another one said: ‘It hurts so good’. • Affection: Love and hate. The feelings involved in interactions with products seem to be related to the experience of mutual affection, of feeling love for the object, as well as feeling loved by the object. This theme reflects the perspective described in Section 2.4, considering touch as a communication channel for affection. People refer to this theme in various ways. First of all, they indicate that they feel love or feel loved (e.g. ‘it has cuddle value for me’, ‘I feel tenderness’), or that they feel hatred or feel being hated (e.g. ‘I see it as a necessary evil’, ‘It drives me mad’). In some cases, the degree of affection seems to depend on the degree of intimacy between the person and the product (e.g. ‘I have my own way of cuddling it’).

62

Product Experience

A related set of feelings involve respect and contempt. As one person states: ‘When you push it harder it still feels soft, but at the same time it gives some resistance, which evokes respect’. Another set of feelings addresses acceptance and rejection. For example: ‘It accepts me the way I am’. Furthermore, affection may involve feelings of attachment. • Vulnerability: Trust and fear of getting hurt. Touch involves the body in physical contact, and confronts people with their physical vulnerability. This theme involves feelings of trust and distrust. It has to do with people’s fear of getting hurt, and also with feelings of being reassured, of trusting objects, and feeling safe with them. In addition, mixed experiences may be reported as well: ‘Under water, the muddy soil was an unknown world, but once you overcame your fear, it was a source of pleasure’. This theme is also related to feelings of freedom and oppression. For example, whereas some clothes give ‘the feeling of being free, like Peter Pan’, other clothes can make you ‘feel oppressed, suffocating’. • Energy: Tension and relaxation. The physical interaction with an object influences a person’s energy level. The energy may increase or decrease, and the energy may be experienced as positive or negative energy. This leads to reports of being physically excited, stressed, relaxed or washed-out. • Action tendency: Approach and avoidance. Action tendencies are components of emotions. An action tendency implies that the actual touching did not occur yet. Hence, the tactual experience may already start before you actually have physical contact with the object; you could call it ‘touching with the eyes’. Similarly, once a person knows how bad something feels, he may try to avoid it. Beside a tendency to approach or avoid, people may experience a tendency to hold on to or let go, and to take care of or neglect, or even to destroy. • Tactual characteristics reflected in self-experience: The tactual properties of objects can be transferred to the self-experience of the person who touches the object. For example, touching something cold can make one feel cold; or touching something dirty can make one feel dirty. The same phenomenon occurs for the experienced personality aspects of the object. For example, when a woman experiences an object as impressive, using the object may make her feel more impressive herself. An attempt to characterize these themes all together led to the umbrella concept of gut feelings as the basic concept for the emotions and feelings experienced in tactual interaction (Figure 2.8). Gut feelings are characterized as feelings that emerge from a non-reflective, direct interaction with the world, and may be related to the visceral level of interaction defined by Norman (2002). Gut feelings are related to our intuitive orientation on the world; they are grounded in our physical actions and physiological reactions, rather than in our cognitive orientation to the world, which is grounded in our thoughts. This explains why it is difficult for people to talk about touch and feelings: It is a non-verbal (or as some would say pre-verbal), intuitive mode of interaction.

8. EDUCATING THE TACTUAL SENSES In design education, students need to develop their awareness, insight and sensitivity for the different aspects of aesthetic experience in human–product interaction. This education cannot be achieved by a mere transfer of knowledge through lectures and readings. It is generally acknowledged that personal hands-on experiences should be part of the educational setting. To educate the designer’s senses, it seems therefore appropriate to develop tools and methods that offer a conceptual framework about the sensory experience,

The tactual experience of objects

63

that is embedded in practical, sensory experiences. Such an approach is based on the interplay between cognitive learning and perceptual learning; once people have a set of concepts to describe what they feel, they will be able to perceive more nuances, and thus describe their experiences more extensively. And vice versa, the more they have experienced physically, the more they will be able to give content to the specified concepts (Chollet, Valentin and Abdi, 2005). Product design education does not seem to offer such educational tools and methods yet. But other domains, as for example art education or wine tasting, offer valuable insights in possible starting points for such educational tools and methods. In art education, several methods were developed to learn to look. These methods offer a set of concepts people may use when looking at and experiencing art. For example, Visser (1986) developed a framework that guides people through the process of experiencing art, focusing on the formal aspects of art perception, such as materials, composition, size, space, point of view, abstraction, and so on. On the other hand, Armstrong (2000) proposed a set of different stages that people may go through when experiencing art, such as gathering information, dreaming, contemplation, investment (engagement), and so on. Both approaches are characterized by a conceptual framework illustrated with examples that the perceiver must apply in his own situation to assimilate their meaning. In the domain of the sensory evaluation of food, tools and methods were developed to enable people to train their sensory sensitivity. For example, the sensory user’s manual for wine tasting (LaMar, 1997) provides an overview of the different sensory aspects of wine and instructions on how to drink it in order to perceive its different aspects. People who attend wine-tasting workshops are thus presented with words that describe sensory aspects, but these words remain empty shells when they are not supported by wine tasting itself. The concepts become meaningful when they are experienced physically. It is only through experience that the concepts become embodied knowledge, and thus recognizable in experience. Therefore, to become wine-tasting experts, people need to build up a personal, embodied ‘database’ of sensory experiences, linked to the framework that is offered, to be able to work with that framework when actually tasting wine. Likewise, tools and methods in the education of the tactual senses in product design should provide designers with a frame of thought introducing the sensory, perceptual and experiential aspects of the specific sensory domain, a structure to guide the designers through these aspects, and insight in appropriate ways to explore these aspects. Moreover, these tools should support designers to build up a personal, embodied database linked to the conceptual framework that is offered on tactual experiences, and to customize the framework to fit the personal world of experience. It is from this starting point that the Tactual Experience Guide was developed. It offers a basis for the development of the aesthetic sensitivity of product design students, provided it is embedded in a personal hands-on exploration of the presented concepts. To ensure these hands-on experiences, product design education should include design courses addressing the aesthetic aspects of the senses.

9. FUTURE DEVELOPMENTS By presenting the content of the Tactual Experience Guide (Sonneveld, 2007) and some of the qualitative data on which it is based, the present chapter has provided a conceptual framework for the description of tactual experiences through words. During design

64

Product Experience

courses at Delft University of Technology, using and evaluating the guide has demonstrated the usefulness of words to get insight into the experience, but simultaneously it has made clear that words are limited in expressing nuances in tactual aesthetics. Therefore, we should explore the possibility to include the use of images, sounds, tangible samples, and so on, when using the Guide. In addition, the evaluation suggested that the Guide could be accompanied by a toolbox containing objects and materials that illustrate the different aspects of tactual experience. As it is only through hands-on experiences that one gains insight in tactual aesthetics, such hands-on experiences should be provided when being introduced to the Guide. Another limitation of the Tactual Experience Guide is that it does not provide insight into the underlying relations between the different aspects it describes. In visual aesthetics, relations between visual properties are explored to understand the underlying principles of aesthetic experience (Kreitler and Kreitler, 1972). These principles include concepts such as contrast, composition, proportion, (dis)continuities, ambiguities, and so on. In addition, the domain of visual aesthetics explores the relationship between visual properties of objects and exposure to these objects, leading to insights such as the MAYA-principle: Most Advanced Yet Acceptable (see Chapter 10). The field of tactual aesthetics lacks insight in such underlying principles. To develop the research field of tactual aesthetics, future research should focus on the underlying principles of aesthetic experience, thus possibly revealing some universal tactual Gestalt principles, and revealing insight in relations between aesthetic experience and interaction factors such as exposure frequencies (Hekkert, 2006). Finally, the insight that tactual experience in human–product interaction can be described as the body language of two social entities opens up new ways to explore the aesthetics of tactual experience, inspired by inter-human behavior and relationships. Possible sources of inspiration are insights in body language (Mehrabian, 1972); attachment (Mugge, Schifferstein and Schoormans, 2006); personality studies (Govers, 2004; Totton and Jacobs, 2001); haptonomics (Veldman, 1996); and social well-being (Ryan and Deci, 2001). For example, Ryan and Deci (2001) concluded that well-being in a relationship between two people is dependent on three factors: One should feel understood, feel competent, and have fun. It is intriguing to translate these findings into requirements for well-being in human–product relationships and to research their implications. Likewise, according to Veldman (1996) experiencing the integrity of the other, and trusting the other, form a necessary and solid base for an exchange of affection. Again, this may offer interesting hypotheses for human–product relationships that could lead to new insights in product design. To conclude this brief overview of possibilities, Totton and Jacobs (2001) observed that people all need to feel understood and accepted by other human beings. Likewise, we could hypothesize that people all need to feel understood and accepted by at least some of the objects that surround them. These directions for future developments in Tactual Aesthetics are not exhaustive; other directions may be explored as well. Nevertheless, the directions mentioned above illustrate that the presented conceptual framework forms a consistent basis to build future theory on; it can be the starting point for developing new hypotheses and research questions, exploring the full diversity of the field of Tactual Aesthetics. In addition, we think that the content and the structure of the conceptual framework will help to bridge the gap between research and design. By supporting designers in developing their tactual sensitivity and exploring the tactual aspects of the objects in their design projects, new insights in Tactual Aesthetics may be obtained with relevance to researchers in the field.

The tactual experience of objects

65

REFERENCES Amazeen, E. L. (1997). The effect of volume on perceived heaviness by dynamic touch: with and without vision. Ecological Psychology, 9, 245–263. Appelle, S., Gravetter, F. J. and Davidson, P. W. (1980). Proportion judgments in haptic and visual form perception. Canadian Journal of Psychology, 34, 161–174. Armstrong, J. (2000). The intimate philosophy of art. London: Penguin Books. Ashby, M. and Johnson, K. (2002). Materials and design: the art and science of material selection in product design. Oxford: Butterworth-Heinemann. Bergsma, J. (1987). Identiteit, lichamelijke verstoring en autonomie. Lochem: De Tijdstroom. Berla, E. P. and Butterfield, L. H. (1977). Tactual distinctive features analysis: training blind students in shape recognition and in locating shapes on a map. Journal of Special Education, 11, 335–346. Bermudez, J. L., Marcel, A. and Eilan, N. (1995). The body and the self. London: MIT Press. Bobjer, O., Johansson, S.-E. and Piguet, S. (1993). Friction between hand and handle. Effects of oil and lard on textured and non-textured surfaces; perception of discomfort. Applied Ergonomics, 24, 190–202. Bolanowski, S. J. (2004). Passive, active and intra-active (self) touch. Behavioral Brain Research, 148, 41–45. Buchholz, B., Frederick, L. J. and Armstrong, T. J. (1988). An investigation of human palmar skin friction and the effects of materials, pinch force and moisture. Ergonomics, 31, 317–325. Burton, G. (1993). Non-neural extensions of haptic sensitivity. Ecological Psychology, 5, 105–124. Chollet, S., Valentin, D. and Abdi, H. (2005). Do trained assessors generalize their knowledge to new stimuli? Food Quality and Preference, 16, 13–23. Craig, J. C. (1988). The role of experience in tactual pattern perception: a preliminary report. International Journal of Rehabilitation Research, 11, 167–183. Craig, J. C. and Rollman, G. B. (1999). Somesthetis. Annual Review of Psychology, 50, 305–331. Csikszentmihalyi, M. (1990). Flow: the psychology of optimal experience. New York: Harper Collins. Davidson, P. W. (1985). Functions of haptic perceptual activity in persons with visual and developmental disabilities. Applied Research in Mental Retardation, 6, 349–360. de Mendoza, J.-L. J. (1979). Demonstration of an aftereffect occurring in the tactile-kinesthetic domain. Psychological Research, 40, 415–422. Dewey, J. (1997 [1938]). Experience and education. New York: Touchstone. Ellis, R. R. and Lederman, S. J. (1999). The material-weight illusion revisited. Perception and Psychophysics, 61, 1564–1576. Fagan, J. (1998). Thoughts on using touch in psychotherapy. In: E. W. L. Smith, P. R. Clance and S. Imes (Eds.) Touch in psychotherapy. New York: Guilford. Fields, T. (2003). Touch. London: MIT Press. Finnegan, R. (2002). Communicating. The multiple modes of human interconnection. London: Routledge. Flanagan, J. R., Wing, A. M., Allison, S. and Spenceley, A. (1995). Effects of surface texture on weight perception when lifting objects with a precision grip. Perception and Psychophysics, 57, 282–290. Franzen, O. and Lindblom, U. (1976). Tactile intensity functions in patients with sutured peripheral nerve. In: Y. Zotterman (Ed.) Sensory functions of the skin with special reference to man. Oxford: Pergamon Press. Ganong, W. F. (2001). Review of medical physiology. New York: McGraw-Hill. Giboreau, A., Navarro, S., Faye, P. and Dumortier, J. (2001). Sensory evaluation of automotive fabrics: the contribution of categorization tasks and non-verbal information to set-up a descriptive method of tactile properties. Food Quality and Preference, 12, 311–322. Gibson, J. J. (1962). Observations on active touch. Psychological Review, 69, 477–491. Gibson, J. J. (1963). The useful dimensions of sensitivity. American Psychologist, 18, 1–15. Goodnow, J. J. (1969). Eye and hand: differential sampling of form and orientation properties. Neuropsychologia, 7, 365–373. Goodnow, J. J., Baum, B. and Davidson, P. (1971). A haptic error: skew in symmetrical curve. Perception and Psychophysics, 10, 253–256. Govers, P. (2004). Product personality. Unpublished PhD dissertation, Delft University of Technology, Delft, the Netherlands. Hanninen, K. A. (1972). Review of the educational potential of texture and tactually discriminable patterns. Journal of Special Education, 5, 133–141. Heider, F. and Simmel, M. A. (1944). An experimental study of apparent behavior. American Journal of Psychology, 57, 243–259. Hekkert, P. (2006). Design aesthetics: principles of pleasure in design. Psychology Science, 48, 157–172. Highley, D. R. (1977). Frictional properties of the skin. Journal of Investigative Dermatology, 69, 303–305. Hodgson, J. (2001). Mastering Movement. New York: Routledge.

66

Product Experience

Hollins, M., Bensmaia, S., Karlof, K. and Young, F. (2000). Individual differences in perceptual space for tactile textures: evidence from multidimensional scaling. Perception and Psychophysics, 62, 1534–1544. Hollins, M., Faldowski, R., Rao, S. and Young, F. (1993). Perceptual dimensions of tactile surface texture: a multidimensional scaling analysis. Perception and Psychophysics, 54, 697–705. Hollins, M. and Risner, S. R. (2000). Evidence for the duplex theory of tactile texture perception. Perception and Psychophysics, 62, 695–705. Klatzky, R. L., Lederman, S. J. and Metzger, V. A. (1985). Identifying objects by touch: an ‘expert system’. Perception and Psychophysics, 37, 299–302. Klatzky, R. L., Lederman, S. J. and Reed, C. (1987). There’s more to touch than meets the eye: the salience of object attributes for haptics with and without vision. Journal of Experimental Psychology: General, 116, 356–369. Kreifeldt, J. G. (2001, June 27–29). Designing ‘feel’ into a product. Paper presented at the International Conference on Affective Human Factors Design, Singapore. Kreitler, H. and Kreitler, S. (1972). Psychology of the arts. Durham, NC: Duke University Press. Kruger, L. (1996). Pain and touch. San Diego: Academic Press. Lakoff, G. and Johnson, M. (1999). Philosophy in the flesh: the embodied mind and its challenge to western thought. New York: Basic Books. LaMar, J. (1997). Tasting wine: a sensory user’s manual. Retrieved 23 March 2007, from http://avalonwine. com/Tasting-Wine.php Lederman, S. J. (1974). Tactile roughness of grooved surfaces: the touching process and effects of macro- and microsurface structure. Perception and Psychophysics, 16, 385–395. Lederman, S. J. (1981). The perception of surface roughness by active and passive touch. Bulletin of the Psychonomic Society, 18, 253–255. Lederman, S. J. and Klatzky, R. L. (1987). Hand movements: a window into haptic object recognition. Cognitive Psychology, 19, 342–368. Lederman, S. J. and Klatzky, R. L. (1998). The hand as a perceptual system. In: K. J. Connolly (Ed.) The psychobiology of the hand. London: MacKeith Press. Lederman, S. J. and Klatzky, R. L. (1999). Sensing and displaying spatially distributed fingertip forces in haptic interfaces for teleoperator and virtual environment systems. Presence, 8, 86–103. MacKenzie, C. L. and Iberall, T. (1994). The grasping hand. Amsterdam: North Holland. Mehrabian. (1972). Non-verbal communication. Chicago, Il: Aldine-Atherton. Michotte, A. (1963). The perception of causality. London: Methuen. Montagu, A. (1971). Touching. New York: Columbia University Press. Mugge, R., Schifferstein, H. N. J. and Schoormans, J. P. L. (2006). Product attachment and product lifetime: the role of personality congruity and fashion. In: K. M. Ekstrom and H. Brembeck (Eds.) European Advances in Consumer Research, 7, 460–466. Duluth, MN: Association for Consumer Research. Murray, D. J., Ellis, R. R., Bandomir, C. A. and Ross, H. E. (1999). Charpentier (1891) on the size–weight illusion. Perception and Psychophysics, 61, 1681–1685. Norman, D. A. (2002). Emotion and design: attractive things work better. Interactions, 9(4), 36–42. Olausson, H., Lamarre, Y., Backlund, H., et al. (2002). Unmyelinated tactile afferents signal touch and project to insular cortex. Nature Neuroscience, 5, 900–904. O’Meara, D. M. and Smith, R. M. (2001). Static friction between human palmar skin and five grabrail materials. Ergonomics, 44, 973–988. Picard, D., Dacremont, C., Valentin, D. and Giboreau, A. (2003). Perceptual dimensions of tactile textures. Acta Psychologica, 114, 165–184. Polanyi, M. (1967). The tacit dimension. Garden City, NY: Doubleday. Pratorius, B., Kimmeskamp, S. and Milani, T. L. (2003). The sensitivity of the sole of the foot in patients with Morbus Parkinson. Neuroscience letters, 346, 173–176. Ryan, R. M. and Deci, E. L. (2001). On happiness and human potentials: a review of research on hedonic and eudemonic well being. Annual Review of Psychology, 52, 141–166. Sacks, O. (1984). A leg to stand on. London: Picador. Sacks, O. (1987). The man who mistook his wife for a hat and other clinical tales. New York: Harper and Row. Saladin, K. S. (2001). Anatomy and physiology: the unity of form and function. New York: McGraw-Hill. Schifferstein, H. N. J. and Cleiren, M. P. H. D. (2005). Capturing product experiences: a split-modality approach. Acta Psychologica, 118, 293–318. Sekuler, R. and Blake, R. (1994). Perception (3rd Ed.) New York: McGraw-Hill. Sonneveld, M. H. (2007). Aesthetics of tactual experience. Unpublished PhD dissertation, Delft University of Technology, Delft, the Netherlands. Stevens, J. C. (1980). Thermal intensification of touch sensation: further extensions of the Weber phenomenon. Sensory Processes, 3, 240–248.

The tactual experience of objects

67

Stevens, J. C. (1990). Perceived roughness as a function of body locus. Perception and Psychophysics, 47, 298–304. Stevens, J. C. and Choo, K. K. (1996). Spatial acquity of the body surface over the life span. Somatosensory and Motor Research, 13, 153–166. Teghtsoonian, R. and Teghtsoonian, M. (1970). Two varieties of perceived length. Perception and Psychophysics, 8, 389–392. Thayer, S. (1982). Social touching. In: W. Schiff and E. Foulke (Eds.) Tactual perception: a sourcebook. Cambridge: Cambridge University Press. Totton, N. and Jacobs, M. (2001). Character and personality types. Buckingham: Open University Press. Tritsch, M. F. (1988). The veridical perception of object temperature with varying skin temperature. Perception and Psychophysics, 43, 531–540. Tsakiris, M. and Haggard, P. (2005). The rubber hand illusion revisited: visuotactile integration and selfattribution. Journal of Experimental Psychology, 31, 80–91. Turvey, M. T. (1996). Dynamic touch. American Psychologist, 51, 1134–1152. Vander, A. J., Sherman, J. and Luciano, D. (2001). Human Physiology. The mechanisms of body functions. New York: McGraw-Hill. Veenhoven, R. (2006). Healthy happiness. Rotterdam: Erasmus University. Veldman, F. (1996). Haptonomie: Wetenschap van de affectiviteit (Haptonomics: Science of affectivity). Utrecht: Erven J. Bijleveld. Visser, A. (1986). Hardop kijken (Looking aloud). Nijmegen: SUN. Vogels, I. M. L. C., Kappers, A. M. L. and Koenderink, J. J. (2001). Haptic after-effect of successively touched curved surfaces. Acta Psychologica, 106, 247–263. Winnicott, D. W. (1964). The child, the family and the outside world. Middlesex, UK: Penguin.

3

THE EXPERIENCE OF PRODUCT SOUNDS RENÉ VAN EGMOND Delft University of Technology, Delft, The Netherlands

1. WHETHER TO BE SILENT In our daily life we are confronted with the sounds of industrial products. However, people are often not aware that they ‘use’ these sounds in a functional or experiential way. Many people use auditory cues ‘unconsciously’. For example, the sound of a water boiler informs one in which phase the boiling process is. Furthermore, people derive not only functional information from the sound, but also information concerning an object’s physical properties. For example, you hear if a big truck is coming towards you even if the truck is far away. This sound is quite different from the sound of an approaching sports car. Moreover, even if the sports car was nearby instead of far away – making the sound louder – the truck still ‘sounds’ bigger. This indicates that people estimate the size of objects on cues different from loudness. Studies have shown that people use sound to estimate the size and the shape of objects (Carello, Anderson and Kunkler-Peck, 1998; Grassi, 2005; Kunkler-Peck and Turvey, 2000), to estimate the size and speed of rolling balls (Houben, Kohlrausch and Hermes, 2004; 2005), and to identify material properties (Giordano and McAdams, 2006; Hermes, 1998). In addition, sound has been used in estimating the quality of engines of vacuum cleaners (Benko et al., 2004; 2005). These examples show that sound is an important carrier of information that people use in interaction with the environment and objects. Research in the field of product sound design has often dealt with the issue of how to make products silent (e.g. Park, Lee and Lee, 2004; Rittmueller, Mann and Holger, 1997). In my view to silence products is not our main goal (although sometimes very important), but the goal is to enhance the experience of a product with proper sound design. Silence should be one of the means available to a sound designer, and will Product Experience Copyright © 2008 Elsevier Ltd.

69

70

Product Experience

be dependent on the type of task a product will be used for. For example, it is wellknown that background noise that exceeds 40 dB(a) limits the concentration of people (Lindeman and Stekelenburg, 1981, p. 114). Therefore, one should consider this aspect when designing a product for office use. The fans of personal computers that emit noise at such a loudness level that they have been compared with jet engines are notorious. Or the noise certain ink jet printers make, caused by the transport of the paper and the movement of the ink cartridge. The former example is, of course, worse because personal computers make the noise during the entire day, whereas in the latter example one chooses the time for printing. Silence can also – and probably should – be used in the design of alarm sounds for hospital situations. Personnel often switch off these sounds because they are too loud and too irritating (Chambrin, 2001; Edworthy and Hards, 1999; Friesdorf, Buss and Gobel, 1999; Solsona et al., 2001). Alarms go off when the value of a measurement device breaches a certain threshold (e.g. if the heart rate suddenly drops). However, often only the value of the threshold has been implemented, and not a duration that would indicate how long a certain threshold can be breached without becoming too dangerous. In other words, there is no time-dependency. Consequently, alarms continue to go off after the threshold has been breached for only a short time, even if the situation is no longer critical. The experience of alarm sounds would improve considerably if these sounds did not depend on these short temporal breaches of thresholds. The term sound quality has often been used in relation to reducing noise or in relation to psychoacoustic measures as, for example, sharpness and roughness (e.g. Brandl, Biermayer and Pfluger, 1999; Ih et al., 2003; Keiper, 1997; Lyon, 2000). However, recent discussions suggest that, dependent on the definition of quality, other aspects – e.g. the association of meaning – should be introduced (e.g. Blauert and Jekosch, 1997; Jekosch, 2004). In this chapter, the term product sound experience will be used, because it is a more general concept (related to the emotional, semantic, and sensorial experience), and because the term is more neutral. It also is a term that is related to the human aspect and not to how the sound can be described in terms of spectral composition or signalto-noise ratios. The latter sound features may evoke or be related to a certain experience and are therefore important to understand, but a one-to-one mapping of certain features to an experience will be too simplistic. In the next paragraph the domain of product sounds will be defined.

2. THE DOMAIN OF PRODUCT SOUNDS The domain of product sounds can be considered as a subdomain of environmental sounds (e.g. Ballas, 1993, 2002; Ballas and Howard, 1987; Björk, 1985; Gaver, 1993a, 1993b; Kidd and Watson, 2003; Marcell et al., 2000). A number of studies investigated memory, recognition, identification and the descriptors of environmental sounds. Environmental sounds comprise the sounds that we hear during our daily life, or – in other words – the sounds that surround us. This means that environmental sounds comprise a diversity of sounds, e.g. car sounds, birds whistling, water sloshing, thunder, horns, bells, dogs barking, etc. Of course, these sounds are often culture or country dependent. In the desert, one will perceive different sounds than in the streets of New York. For example, Ballas (1993) has asked people to keep a diary of the sounds they heard. The sound of highway traffic occurred quite often. There will be many places in the world where this sound is not heard, indicating the importance of contextual information.

The experience of product sounds

71

Although sounds of products are with us during our daily life, a perceptual domain of product sounds has not been established. We define the domain of product sounds as being the sounds that are produced by industrial products, cars, and user interfaces. Among product sounds, we distinguish between consequential and intentional sounds. Consequential sounds (mechanical sounds, Västfjäll et al., 2002) are a consequence of the moving parts of products and are affected by the interaction when a person uses a product. For example, a toothbrush sound will be generated by the engine (its rpm), the gear, the transmission, and the pressure and movement applied by the user. Intentional sounds are deliberately added to a product. These sounds are synthesized or recorded. They are radiated through a loudspeaker or by a vibrating surface. User interface and alarm sounds belong to this category. The sounds are often more musical or speech-like than the consequential sounds. Consequently, the design and the experience of intentional sounds will differ from consequential sounds. Before describing the specific properties of the consequential and intentional sounds, a short introduction of the spectral and temporal structure of sound will follow in the next paragraph.

3. SPECTRAL AND TEMPORAL STRUCTURE OF SOUNDS Three perceptual domains of sound can be distinguished: Speech sounds, musical sounds, and environmental sounds (sometimes called everyday sounds). Certain sounds may be classified in more than one domain. In these cases, the context in which the sound is heard will be the determinant factor to relate it to a specific domain. For example, singing produces speech sounds (especially if one considers the Sprechstimme (‘speech-voice’)), but this does not warrant to state that every speech sound is a musical sound. Each domain has its own characteristic features. Sounds are described by an amplitude fluctuation as a function of time in the temporal domain and as a sum of sinusoids (overtones) in the spectral domain. Each event in the temporal domain can be mathematically transformed (i.e. Fourier transformation) into the spectral domain and vice versa. A Fourier-transform of a time-signal results in a series of sinusoids (with their relative amplitude and phases). This sum of sinusoids determines the spectral composition of the sounds. Natural sounds – and most synthetic sounds – are composed from several sinusoids called the overtone series. The number of overtones, their frequencies and amplitude fluctuations determine the sound color or timbre. Vowels and consonants can be considered as the basic constituents of speech. Vowels are recognized by their timbre characteristics. Formants are specific areas in the speech spectrum. The relationship between formants is an important aspect in the recognition of vowels. Consonants are noise or click-like sounds. Although speech has been applied in user interfaces, it has mostly been used to inform a user. Some interfaces have been developed to recognize a user’s speech commands, but they often fail. One of the reasons is that the systems have to be trained to recognize specific voices. Human speech recognition not only analyzes the sound, but in addition applies top-down knowledge on grammar, lexicon, and uses the meaning of the produced sentences. One fills in gaps that have not been perceived in the speech sound. In this chapter we will not consider the perception or design of speech in user interfaces. Two categories of musical instruments can be distinguished: (a) pitch-producing (e.g. violin, guitar, piano, etc.); and (b) ‘noise’ producing (e.g. drums, wood-blocks). These categories are not that strict. For example, timpani are often tuned in fourths, thus pitch can be attributed to these ‘noise-like’ sounds. Most sounds of pitch-producing instruments consist of overtones that are proportionally related by the harmonic series (1:2:3:4:5:6:7:8, etc.), or have small deviations from the harmonic series (1:2:3.01:4.02,

72

Product Experience

etc.). The deviations from the harmonic series introduce inharmonicity. This inharmonicity has a physical cause, e.g. the stiffness in metal strings. A sound with a fundamental of 100 Hz would consist of a series of harmonics of: 100 Hz, 200 Hz, 300 Hz, 400 Hz, 500 Hz, 600 Hz, etc. This is the ‘ideal’ sound that hardly exists. It is interesting to note that the harmonic series contains the intervals of the Western tonal system. The proportion 1:2 is the octave, 2:3, the fifth, 3:4, the fourth, 4:5 the major third, and 5:6 the minor third. In terms of pitches, if the fundamental tone is C then the series of overtones (until the 6th harmonic) will be the following: C, C, G, C, E, G. This series is often considered as the underlying principle of Western tonal harmony (see, e.g. Parncutt, 1989). The number of overtones between the harmonics determines how noiselike a sound is (the amount of tonalness decreases). This can be captured by the harmonicsto-noise ratio (see, e.g. Boersema and Weenink, 2006). It is also worthwhile to note that the onset (start) of a musical tone often introduces noise-like elements that are important in people’s judgment on the naturalness and quality of a sound (e.g. Fletcher, 1962). Musical sounds – and even melodies – are considered to be product sounds if they have been implemented in industrial products. For example, sounds in user interfaces, alarm sounds, and start-up sounds in computers are considered product sounds.

4. PRODUCT SOUNDS As mentioned before, product sounds are a perceptual subdomain of environmental sounds and also comprise the use of musical sounds (musical instruments or synthesized sounds). In most instances, synthetic sounds are implemented in the way they have been designed; they will not change due to product–user interaction. As a result the sound will be the same every time it is triggered. Although this simplifies the design of these sounds, the experience of the sound will be affected. The reason for this is that a perfectly designed sound will evoke a desired experience. However, in this case the experience of the sound will also depend on the repetition by which it is heard. A repeated sound that remains perceptually equal will come to be experienced more negatively. This appears to be in conflict with the mere exposure effect reported by Zajonc (1968). However, if one considers the nature of user interface sounds then the following explanation may be given. User interface sounds are often short, of a simple structure, and are repeated many times. In a meta-analysis of the relevant literature, Bornstein (1989) found that the mere exposure effect is stronger for complex stimuli, for multiple stimuli, and for brief stimuli that are not repeated that often. Most user interface sounds have just the opposite features and may therefore be disliked with repetition. Consequently, a sound designer should consider introducing small deviations in the spectral-temporal structure of the sounds. These fluctuations will result in minor differences between repetitions, but nonetheless all sounds will be perceived as similar to the original sound. The similarity will evoke the familiar experiential properties, whereas the deviations will avoid boredom or irritation. In the next paragraph, the spectral and temporal structure of product sounds will first be discussed without the influence user actions have on the sound. Then the effect of the user on the sound production will be discussed, and its consequences on the sound properties.

4.1. Spectral and temporal structure of product sounds The amount of structure in both the temporal and spectral domain of a product sound determines how well a sound can be recognized and remembered (Özcan and

The experience of product sounds

73

Van Egmond, 2007). Sounds with no-structure in the spectral domain will sound very noise-like. In this case, the sound will be hard to manipulate to achieve a proper experience. The lack of structure in the temporal domain (e.g. absence of a strong perceived periodicity in sounds with an engine or a hierarchical rhythmic structure in alarm sounds) will make it harder to encode a sound. If a sound is highly structured in both domains recognition and memory are high, whereas for sounds that are unstructured in both domains recognition memory will be low. Özcan and van Egmond (2007) distinguished three levels of structure in the spectral domain and three levels of structure in the temporal domain. In the spectral domain, a highly structured sound consists of an (harmonic) overtone series, and therefore will sound tonal. These sounds almost do not contain noise (or noise that is relatively low in intensity compared to the overtones). Medium structured sounds in the spectral domain stem from products that contain an engine with a certain rpm (revolutions per minute). The rpm will result in a perceivable periodicity in the sound that generates an (harmonic) overtone series. However, these sounds consist of a considerably large amount of noise caused by the moving of, for example, gears. In other words, the intensity of the noise is relatively high compared to the intensity of the overtones generated by the periodicity. Sounds that only consist of noise or (rapidly) changing spectral structure over time are the least structured. Not only will the memory and recognition decrease with the amount of structure, but the possibility of designing a sound experience for a product will be much more difficult if structure is lacking. We have also distinguished three levels of temporal structure for product sounds. Product sounds that are highly structured in the temporal domain will have a distinctive (hierarchical) rhythm-like structure, and will therefore be more easily remembered (see Povel and Essens, 1985). An example is the sound of an alarm clock with a distinctive rhythm. Products with engines that run at a certain rpm will have a cyclical temporal structure. These sounds are considered medium structured. The rpm causes a periodicity – one experience the cyclic temporal structure – but this periodicity produces continuous events, not discrete events in the temporal domain. In contrast to discrete events, continuous (cyclic) events in the temporal domain will not evoke a strong rhythmic pattern. Products that operate with airflow or water-flow will produce sounds with the least recognizable temporal structure. The aforementioned descriptions of the spectral and temporal structure of product sounds are without the influence of user actions. In other words, these are products that run stationary. The user actions will influence the spectral–temporal structure of a product sound and will be determined by the type of control system. Three control systems can be distinguished: Autonomous; partly autonomous; and user-dependent control systems. The comparability of the sounds produced by similar events decrease in the following way: Autonomous; partly autonomous; and user-dependent. The comparability of the sounds of similar events will affect the possibility of making proper predictions concerning experiential aspects. Products that have autonomous control systems (e.g. central heating systems) will produce sounds that are very similar (equal) if events are repeated. In fact, minor fluctuations in the sound will be more dependent on the external factors of the environmental context than that a user action will influence the sound. Consequently, the experience of the sound will be highly predictable. For example, a central heating system will be dependent on the temperature of the surrounding context that influences the time of heating and the temperature of the water that needs to be heated. The spectral–temporal composition of the sound of a product with a partly autonomous control system will be dependent on user actions. However, the deviations in the

74

Product Experience

spectral–temporal composition will be small. Consequently, the experience of the sound will still be predictable within small perceptual boundaries. For example, the sound event of a drip filter coffeemaker will be dependent on the amount of coffee it has to brew, how the filter is placed, etc. However, when the coffeemaker is switched on it will run autonomously. The spectral–temporal composition of products with user-dependent control systems can vary considerably. Two manifestations of sound can be discerned: (i) The stationary sound (i.e. the sound a product generates when it is switched on but not used); (ii) The sound when the product is in use. The experience of the sound of these products in use and when they are stationary differ. For a company this may mean that if they want to design the shelf-presence of a product sound then they should consider the experience of the sound when the product is switched on without any user action. For example, a toothbrush can often only be switched on and cannot actually be used in a store. The design of the stationary sound is easier than the design of a sound that pertubates perceptually under the influence of usage. There are two main reasons for this. First, the spectral composition of the sound will change when the product is used (e.g. by increasing the amount of pressure on a toothbrush). Differences in spectral compositions may evoke different experiences. Consequently, it will be more difficult to design the preferred sound experience. Secondly, user actions are not systematic and will generate random fluctuations that are harder to predict, and therefore more difficult to incorporate in the sound design. Three examples of product sounds will be given. In Figure 3.1 the spectrogram of an alarm clock sound is presented. In the spectrogram frequency as a function of time is represented with grey shading to indicate the intensity of the frequencies. It can be seen that a tone with duration of approximately 350 ms is repeated for four times (i.e. discrete events). The pitch of this sound is approximately 890 Hz, it can be seen that there is a strong emphasis for the sixth overtone (around 5300 Hz, darkest shading). In addition,

Figure 3.1 Spectrogram of an alarm clock sound. Time has been represented on the x-axis. Frequency has been represented on the y-axis. Intensity of overtones has been indicated with shades of gray.

The experience of product sounds

75

the ninth through the fourteenth overtones also have a relatively strong intensity. This indicates that the sound will be perceived as somewhat sharp, because the sensation of sharpness depends on the amount of energy in the higher frequencies. Consequently, the sound may be perceived as relatively unpleasant. This is, of course, a desirable experiential property for an alarm sound if one needs to wake up. In Figure 3.2 the spectrogram of the sound of an electric toothbrush is presented. It can be seen that there are no discrete events; the sound events are continuous. It can also be seen that the sound contains noise, because hardly any frequency bands can be distinguished. Nevertheless, in Figure 3.2 some darker frequency bands can still be distinguished, caused by the periodicity of the engine. The user involvement is reflected in this figure, especially if one considers the dark band in the spectrogram around 2500 Hz. This band fluctuates over time due to the pressure the user applies to the brush. With an increase of pressure the frequency decreases, whereas if the pressure decreases the frequency increases. One can hear this in the perceived pitch of the sound. This demonstrates that there is a certain bandwidth in which the spectral content of the sound fluctuates. Consequently, the experience of the sound may be dependent on the amount and the strength of these fluctuations. In a pilot study, we investigated the effect of pressure on the preference and experience of a toothbrush sound. In this study, the pressure was systematically varied by putting weights on the head of the toothbrush. Thirty participants were asked to indicate which sound they preferred the most using a paired comparison paradigm. Preference was negatively associated with the amount of pressure (r
.92). In other words, if the pressure increases then the preference decreases. Zampini, Guest and Spence (2003) have investigated the effect of a toothbrush sound on the experience of the vibrotactile stimulation on teeth. They have found that if the loudness level or the intensity of the higher frequencies decreases the ‘tactile’ feeling of roughness and unpleasantness of the toothbrush also decreases.

Figure 3.2 Spectrogram of the sound of an electric toothbrush. Time has been represented on the x-axis. Frequency has been represented on the y-axis. Intensity of overtones is indicated with shades of gray.

76

Product Experience

Figure 3.3 Spectrogram of the sound of a vacuum cleaner. Time has been represented on the x-axis. Frequency has been represented on the y-axis. Intensity of overtones has been indicated with shades of gray.

In Figure 3.3 the spectrogram of the sound of a vacuum cleaner is shown. It can be seen that there is even less structure in the frequency domain. In addition, the energy of the sound appears to be more concentrated in the lower frequency range, whereas for the toothbrush the lower frequencies do not contain that much energy and the higher frequencies are relatively high in energy (note that the sounds have all been normalized for intensity to the same sound pressure level). In Figure 3.3 vertical lines have been drawn to identify the areas in the spectrogram in which changes occur. In these areas, the dark bands in the spectrogram are interrupted and the amount of energy for the higher frequencies increases (this area becomes darker). This change results from the blocking of the nozzle of the vacuum cleaner when the hose is moved back and forth. In conclusion, consequential product sounds are often noisy. This noise (random structure) complicates the design and predictability of these sounds. The spectral– temporal composition of the sound is not only dependent on how the product has been constructed, but in addition is dependent on the usage actions (e.g. pressure). The structural aspects of intentional sounds (e.g. feedback or alarm sounds) can be designed, and thus the experience of these sounds is largely under control (not withstanding that contextual aspects may also influence these sounds). In addition, the improved technology nowadays allows the generation of high quality (CD standard) sounds. However, in many cases the design of the sounds has not been explicitly based on the type of interaction the designer wanted. In comparison with consequential sounds there is so much design freedom (e.g. timbre, pitch, inter-onset intervals) that it becomes difficult to make choices. From ecological acoustics (e.g. Gaver, 1993a,b) it is known that sounds often convey a meaning. That is, one can hear the type of source, the location of the sound, the context in which the sound is present, and whether the source of the sound is dangerous. Therefore, it is important that sounds in user interfaces are designed to convey a specific meaning. Alarm sounds in intensive care units are a ‘good’ example of improper – or not well-thought-through – sound designs.

The experience of product sounds

77

Figure 3.4 Two spectrograms of alarm sounds of a device that controls a patient’s breathing from the same manufacturer.

The use of alarm sounds in intensive care units generates many problems (Edworthy, Loxley and Dennis, 1991; Edworthy and Meredith, 1994; Welch, 1999; Yan Xiao and Seagull, 2003). Manufacturers often implement alarm sounds because they want to reduce the probability of liability law-suits (Edworthy and Adams, 1996). Because alarms go off unnecessarily and too often, staff in intensive care units often turn the alarms off. Because they are occupied with coping with their annoyance, they forget to attend to the actual problem (e.g. Block, Nuutinen and Ballast, 1999; Edworthy and Adams, 1996; Momtahan, Hetu and Tanssley, 1993). The large number of sounds, the lack of conveying a well-defined meaning (e.g. type of device, the level of urgency), and the masking effects by other alarms and noise-producing sources make alarm sounds very difficult to remember and to recognize (Edworthy and Adams, 1996; Momtahan, Hetu and Tanssley, 1993; Welch, 1999). Often alarms that need to convey a certain meaning (e.g. cardiac arrest) may differ between devices. This finding is not only true for devices of different manufacturers, but also for devices from the same manufacturer. For example, we recorded the sound of a device that controls the breathing of a patient. If the pump stops then this alarm sounds. In Figure 3.4 spectrograms of sounds are shown that stem from two versions of this device from the same manufacturer. It can be readily seen that the sounds differ in their spectral composition. In addition, the spectrogram depicted on the left consists of the same tones that are repeated. The tones have a duration of 250 ms followed by a pause of 250 ms. The spectrogram depicted on the right consists of five tones. The arrows indicate at what point the new tone starts. It can be seen that there is a small overlap of the tones (the next tone starts while the previous tone is still sounding). Furthermore, the temporal structure of the second sound is more rhythmic than of the first sound. This rhythm can be characterized as: short-short-longshort-long. After the last tone has sounded, a pause starts in which the decay of the last sound can still be heard. After 3 seconds this rhythm is repeated again. It can be concluded that the left and the right sound do not contain any similarity. Consequently, staff of the intensive care unit will have to remember two different sounds that correspond to the same cause. This example demonstrates that a lot can still be improved in the consistent design of alarm sounds.

5. PROCESS OF AUDITORY PERCEPTION Several theories and models exist that describe the process of auditory perception or specific aspects in this process (e.g. Bregman, 1994; Handel, 1991; McAdams, 1993; Yost, 2007;

78

Product Experience

Fastl and Zwicker, 2007). I have been developing a conceptual framework for the process of auditory perception that guided my thinking and the set up of research projects. In this conceptual framework, I make a distinction between experiential aspects of sounds that stem from a bottom-up process and experiential aspects of sounds that stem from top-down processing (or ‘knowledge-rule based aspects’). This distinction is coarse, and I am aware of the many subtleties that have been reported in the literature concerning the interaction of bottom-up and top-down processing of sound. However, the distinction is of importance in the experience of product sounds because it allows pinpointing certain effects as a result of bottom-up processing which are salient and unaffected by prior knowledge. Figure 3.5 represents a flowchart of the conceptual framework. On top a sound has been symbolically represented in the temporal domain. The sound will be modified by

FIGURE 3.5 Conceptual framework of auditory perception.

The experience of product sounds

79

the outer and middle ear, but the main processing occurs in the inner ear. The middle ear functions as a half-open organ pipe with a resonance frequency between 2 kHz and 5 kHz. This enhances our sensitivity for this frequency range. The sound is processed in the inner ear by the cochlear mechanics and neural activity (see, e.g. von Békésy and Rosenblith, 1951; Yost, 2006). The inner ear (with the basilar membrane as an important constituent) can be considered as a low pass filter. The higher frequencies are mapped at the start of the cochlea, whereas the lower frequencies are mapped at the end of the cochlea. Thus, each region in the inner ear is sensitive for a certain frequency. The inner ear decomposes a sound into its constituent frequencies. In a way, this resembles the mathematical technique of the Fourier transform in which a signal in the temporal domain is transformed into the frequency domain (as a result the constituent frequencies can be determined). The spatial mapping of frequency is projected on the brain through the auditory nerve. This is called a tonotopic representation of the sound. Because of the cochlear composition (fluids and organ of Corti cause integration of single resonances) there are areas of resonance. Critical bands are areas in which the summation of loudness and masking effects take place (see Fletcher, 1940; Fastl and Zwicker, 2007). For example, frequencies within a critical band contribute less to the perceived loudness than frequencies outside a critical band. Furthermore, the understanding of the masking of tones (a tone with a certain intensity can be made inaudible by another tone, noise-band, or noise) made it possible to develop coding techniques like mp3. In addition to loudness, sharpness, harmonicity, and roughness can be explained by our understanding of the functioning of the inner ear. These measures are sensations that contribute to the sensory pleasantness of a sound. Many of the psychoacoustical measures (see for an overview, Allen, 1999; Fastl and Zwicker, 2007; Hirsh and Watson, 1996; Moore, 2003; Yost, 2007) have been implemented in algorithms for sharpness, harmonicity, loudness, roughness, and tonalness. Aures (1985) and Fastl and Zwicker (2007, p. 245) proposed models of sensory pleasantness on the basis of these values. The models differ in the weighing of the factors. An increase in the value of loudness, sharpness, and roughness all result in a decrease of sensory pleasantness. An increase of tonalness (named tonality by Fastl and Zwicker, 2007, but this is to my opinion too much related to the musical concept of tonality and causes confusion) results in an increase of sensory pleasantness. Tonalness can best be described as the amount of harmonic structure in a sound in relation to the amount of noise. Because psychoacoustic experiments demand a large amount of experimental control, the sounds employed to determine these effects are often synthesized (e.g. frequency modulation, amplitude modulation, noise bands). As mentioned before, psychoacoustical measures are often used because these measures can be implemented in algorithms (see Keiper, 1999). These implemented psychoacoustics measures will be referred to as psychoacoustic metrics. Lim (2001) has used psychoacoustic metrics to predict the quality of the sound of power windows. Västfjäll et al. (2002) have found that these metrics do not predict whether a sound is related to the pleasantness and activation dimensions of emotional experience. In a graduation project (Bouter, 2006) we have investigated if the psychoacoustic metrics of roughness, loudness, sharpness, and harmonicity (an implementation for tonalness) are associated with participant responses on self-report scales of roughness, loudness, sharpness, and harmonicity for product sounds. The metrics of loudness, sharpness and harmonicity were highly correlated with the constructs (r
.82, r
.97, r
 .89, respectively), whereas the correlation between the roughness metric and the corresponding construct was not that high (r
.47).

80

Product Experience

These auditory sensations are important in our understanding of the experience of product sounds because they are very difficult to suppress. In other words, if these factors are perceptually salient their effect is hard to change by an attribution of meaning. Top-down influences (the next step in the model) may influence but will not completely override these primary sensations. Thus, a proper sound design should always control for these primary sensations. Consequently, if these primary sensations are under control then top-down influences may contribute more to the experience of the sound. This will eventually lead to a richer sound experience. In the last stage of the conceptual framework, information from memory influences the processing of a sound. This information may consist of the attribution of meaning, but it can also consist of more abstract structural aspects that describe a sound (e.g. the increase and decrease pattern in pitch). For example, if one of the alarm sounds of Figure 3.4 is coded in memory, then several aspects are represented. First, the timbre of the sound (audition-specific image) should be coded. Secondly, the temporal structure (the rhythmic pattern) will be represented. Thirdly, the name of the medical device that produces the sound is represented. Fourthly, the type of medical failure and the level of urgency associated with the sound will be represented (attribution of meaning). Fifthly, the context in which the sound is heard (intensive care unit) is important. When the sound is heard and encoded on a subsequent occasion, these aspects (if there is recognition) will be activated and will be combined with the sensory sense of urgency (caused by the tempo, the repetition) that will evoke the necessity for a staff member of the intensive care unit to act. Note that the combination of top-down and bottom-up sensations that the sound makes evokes a need to act. If somebody who is not a staff member hears the sound but is not present in the intensive care unit, a sense of urgency may be induced by the rhythmic nature of the sound, but no knowledge is available concerning the actual failure. Therefore, the need to act will not be induced. Note that there is a difference between evaluating a sound and experiencing an emotion while listening to the sound. Nevertheless, Västfjäll et al. (2002) have demonstrated that affective reactions and affective evaluations are associated. Thus, although this distinction is important from a theoretical point of view, it may be difficult to measure experimentally. Possibly the context and the attribution of meaning are important to induce a difference in experience and in evaluation. However, these aspects have not been controlled in the study of Västfjäll et al. (2002). Thus far, the sensation-driven and more cognitively induced experiences have been discussed. However, is it possible to enhance our understanding of how these aspects interact and what type of emotions they evoke? In addition, which aspects in product sounds contribute to the sensations and which contribute to the cognitive aspects? In other words, can we gain more insight into where in the process of auditory perception certain perceptions or emotions occur? Several studies have shed light on these issues. For example, for the recognition of the rhythmic structure a mental frame of reference needs to be induced (see e.g. Povel and Essens, 1982), suggesting that it involves topdown knowledge. On the other hand, the perception of the tempo will be more bottomup driven. Many studies have investigated affective reactions to acoustic stimuli using rating scales (Bradley and Lang, 2000; Edworthy, Hellier and Hards, 1995; Meunier, Habault and Canevet, 2001; Solomon, 1959; Von Bismarck, 1974a,b). Rating scales have been successfully employed in many studies to obtain emotional responses and convey the intensity of the evoked emotion. However, they do not convey where the evaluation took place in the perceptual process. Thus, these scales cannot be used to differentiate between basic and more cognitive emotions. Basic emotions are the outcome of a simple appraisal.

The experience of product sounds

81

They are needed for survival purposes and are often directly related to an action (e.g. Izard, 1977; Plutchik, 1980). For example, the emotion ‘fear’ that is the outcome of an appraised ‘physical threat’ will result in the action to flee. More complex, or evolved, appraisals evoke ‘cognitive emotions’ (see Ortony, Clore and Collins, 1988). An example of a cognitive emotion is inspiration, which requires an appraised ‘mental illumination’ (see Desmet, 2002). This knowledge is essential if one wants to design a product sound, because it will determine which features in the sound to manipulate. For example, to evoke basic emotions manipulating aspects like, for example, roughness or sharpness will be important, whereas for the more cognitive emotions attribution of meaning will be more important. It is hypothesized that the distinction between basic and cognitive emotions results from a difference in processing. The more cognitive emotions will need more processing time because more information (e.g. structural aspects, association of meaning) from memory has to be accessed. It is hypothesized that the basic emotions will rely more on the sensations discussed before. If the two emotion types are measured using verbal selfreports, the meaning of these emotion words needs to be accessed in memory. In addition, the bottom-up processing of the sounds (i.e. the sensations) will be independent of the type of emotion. Thus, if a sound is evaluated on a basic emotion, then long-term memory is only accessed once. However, in order to evaluate the more cognitive emotions, longterm memory in the conceptual framework needs to be accessed twice: Once to access the meaning of the words, and once to access the structural aspects, associations of meaning of the sounds. Consequently, the processing time will take longer. Van Egmond, Desmet and Van der Helm (2004) have tested this hypothesis by using different sounds and letting listeners evaluate these on ‘basic’ and more ‘cognitive’ emotions. Rating values and response times have been collected in this study. Responses to the ‘basic’ emotions were significantly faster than responses to the ‘cognitive’ emotions. This study shows that response times can be successfully used to differentiate between emotion types.

6. DESIGNING THE EXPERIENCE OF CONSEQUENTIAL PRODUCT SOUNDS Just as one designs visual concepts for products and tests the experience of these concepts, there is a need to design auditory concepts that a ‘future’ product will produce. Several studies have investigated perceptual attributes for sound design and have demonstrated the usefulness of sound design for product sounds (e.g. Bowen and Lyon, 2003; Collins, 2001; Jordan and Engelen, 1999; Louwers, 1993; Lyon, 2000, 2003). Consequential sounds are a result of how an appliance has been constructed. Thus, the sound is dependent on, for example, the type of engine, the type of material, and the gears. Therefore, if a designer wants to create a consequential sound in the conceptual phase it will be a difficult task, because the sound needs to be realistic without the product being there. For example, if one wants to design the sound of an electric toothbrush one needs to model or to mimic the new product. This means that knowledge of the construction, the engine, the gears and the material has to be implemented in order to create a new sound. Gaver (1986) has suggested that sounds in user interfaces (i.e. intentional sounds) should afford certain actions. Therefore, these sounds should be caricatures of natural sounds (named auditory icons) because this will facilitate conveying the meaning of a sound. Gaver (1993a,b) elaborated on these ideas more extensively. He proposes not only involving the perceptual and spectral aspects in our understanding of environmental sounds, but also understanding the physical processes and mechanisms that cause the

82

Product Experience

sounds to sound as they sound. Furthermore, he has identified that the type of interaction of an object with the environment is important in our understanding of the produced sound. Therefore, several interaction types have been discerned (scratching, scraping, bouncing, etc.). Lyon (2000) has modeled several vibrating systems and has related them to perceptual judgment in order to obtain predictions on the sound quality of products. The question still remains: How realistic are these synthesized sounds? In a number of studies and projects for industry we have proposed a design method for producing sounds on the basis of recordings and manipulations of recorded parts of product sounds, and we proposed to make this part of the design process (Özcan and Van Egmond, 2006; Van Balken, 2001; Van Egmond, 2006). This methodology consists of four stages. In the first stage the sounds are recorded. Secondly, the sounds are analyzed (spectral-temporal, psychoacoustical, and perceptually). Thirdly, a realistic sound design concept is proposed based on preferred experience, and the sounds are designed. In the fourth stage, the sounds will be tested with respect to how successful the design is in relation to the required sound experience. These stages can be recursively applied if the sound experience is not as desired. In the next paragraphs the different stages will be discussed.

6.1. Recording One of the most important aspects in the recording of sound is the choice and the placement of a microphone, or the use of even more than one microphone. In music recordings, knowledge is available on how to record different instruments. This has resulted in standards of which microphones to use and where to place them (see Dickreiter, 1984). However, this knowledge is not easily applicable to the recording of product sounds. Although one can learn from the way musical instruments are recorded (e.g. where to place a microphone for a wind instrument or where to put the microphone for a relatively large instrument), the placement and the choice of microphones for the recording of product sounds is often a combination of common sense and trial-and-error. In addition, the sound level needs to be measured in order to know the loudness of sound. A standard calibrated sound level meter is appropriate to measure the sound before recording it (e.g. Bruel and Kjaer Type 2260). As mentioned before, usage actions may influence the sound of an appliance. Therefore, the sound should be recorded with, and without, user involvement. In addition, the separate sound-producing parts of the appliance will be recorded separately. This allows an analysis of the separate sound sources that will enable one to determine which aspect (or part) contributes to a certain wanted or unwanted experience. Other candidates for sources (replacement) are also recorded. These sources should offer the same functionality but achieve this functionality in another way. For example, one could use a boiler to warm water, but also a heating element. These will all be used in the design phase to generate sound concepts.

6.2. Analysis In the second stage, several methods of analysis will be conducted on the sounds using implemented algorithms, in addition to an analysis by the investigator’s or designer’s ear. The spectral–temporal composition of the sound will be determined and will be related to the sound-producing parts. This process will be exemplified with the analysis and design of a coffeemaker sound (Van Balken, 2001). In Figure 3.6 the amplitude of the signal in the time domain (upper graph) and the spectrogram (lower graph) of the recording of a coffeemaker have been depicted. The amplitude of the time signal has been normalized. It can be seen that the entire process after the coffeemaker has been switched on takes

The experience of product sounds

83

Figure 3.6 The sound of a coffeemaker represented in a time–amplitude and in a time–frequency/intensity domain (Van Balken, 2001).

approximately 100 seconds. In the graph specific events and sound producing parts have been indicated. Because the valve ticking is a very characteristic sound, this part has been magnified in the amplitude-time graph (the small click can thus be distinguished). After approximately 75 seconds the water has reached the correct temperature (approximately 95ºC) and a decay of the amplitude can be seen. Five seconds later, the pump starts and forces the water through the valve and the coffee pad. The pump sound is relatively high in intensity and will therefore evoke a sense of unpleasantness. From 85 seconds through 100 seconds the coffee flows into the cup, after which the pump stops. An event that is noticeable is the squeak that occurs approximately 2 seconds after the pump starts. It can be seen in the spectrogram (bottom graph) that this squeak contains many high frequencies of a relatively high intensity. This means that the sound will be experienced as very sharp and unpleasant. In Table 3.1 the sound events are represented as a function of the sound producing parts and use stages. It can be seen that there are nine distinctive sound events, and that most events are independent of user actions (only the opening of the lid is user dependent). In addition, most of the events occur during the brewing process. This brewing process is also the period in which multiple sources contribute to the sound (see also Figure 3.6). The next step in the analysis phase will be to determine the psychoacoustical metrics (e.g. roughness, sharpness, loudness) previously described for each of the identified events. Because sound varies over time and many different events occur, it is important

84

Product Experience

TABLE 3.1 Characteristic cofeemaker sounds with related product parts and use stages Sound

Sound producing product parts

Use stage

Squeaking Sucking Rustling Ticking Whirring Squeaking Rustling Coffee flow Rustling

Metal opening system Air escaping valve for the reservoir Boiler Valves between boiler and spout Pump Air driven through the hole in pad holder Brew chamber Spout Boiler

Opening lid Placing the reservoir Heating Heating Brewing Brewing Brewing Brewing Post-brewing

Figure 3.7 Top figure: Aggregated pleasantness ratings as a function of time (in seconds). Bottom figure: The sound scenario of Figure 3.6 (Van Balken, 2001).

to know which of the events results in a – for example – unpleasant experience. In order to relate sound events to a judgment of pleasantness, an online measurement task is used. In this task a listener is asked to indicate the experience of the sound (e.g. how pleasant) while listening to it. A slider device consisting of a linear potentiometer is employed to determine the rating value and the time on which the rating occurred. A computer samples the responses on the slider with a certain frequency that depends on the needed temporal resolution. In this way the rating data can be mapped onto the sound. With this device fluctuations in people’s judgment can be tracked, and the boundaries can be determined at which a judgment changes. In Figure 3.7 an example of this type of response has been represented. In the bottom part of this figure an abstract representation of the sound events as they can be seen in Figure 3.6 is given (note that we shortened the duration of the first part of the heating process in the recording). In the top figure the aggregated number of times (over listeners) people scored in one of the five categories (pleasant, fairly pleasant, neutral, fairly unpleasant, unpleasant) has been presented as a function of time. It can be readily observed by comparing the top and bottom graph that the heating process has been judged overall fairly pleasant, whereas as soon as the pump starts the judgment becomes unpleasant. Note that because the squeak is very short in relation to people’s behavioral reaction of moving

The experience of product sounds

85

Figure 3.8 The sound events of a redesigned sound of a coffeemaker (Van Balken, 2001).

the slider, this event cannot be found in the pleasantness ratings. However, additional interviews have indicated that this has been perceived as a very unpleasant event. In the next stage the results of the first two stages will be used to come up with a different sound scenario and a concept for a sound design.

6.3. Concept and sound design phase After the problems with the original sound have been determined, the sound needs to be improved. In this phase several scenarios and sound design concepts are created that should evoke the desired experience. The experience will be determined by the usecues one wants to convey (e.g. when does something start or stop) and the experiential aspects (sometimes based on a company’s brand values). In Figure 3.8 a new sound scenario for the coffeemaker is presented based on the experience of quality, warmness, coziness, and hominess. The sound events have been presented as a function of time. It can be seen that the duration is shorter than the duration of the scenario presented at the bottom of Figure 3.7 of the original sound. It is envisioned that this should evoke a sense of quality because devices are considered to be of higher quality if they are faster than similar devices (comparable to the acceleration rate of cars). In addition, the heating starts and then stops and starts again. This introduces a kind of wave-like feeling in the sound that can be considered cozier because it resembles the breaking waves on a sea-shore. The heating also continues when the pump starts, removing the small silent period in the original sound just before the pump starts. This masks the start of the pump and creates a more natural flow. In the original sound the silence before the brewing and the pump start possibly evokes a suspenseful experience that enhances the negative effect induced by the pump. Because the pump and the squeak were the main contributors to the most unpleasant reactions to the sound (see Figure 3.7), the squeak has been removed and the pump runs more regularly and starts up more gently. The new sound is synthesized using recordings of sound-producing parts, other parts than those present in the current appliance, sound filtering techniques to simulate different damping or resonant aspects, and some physical modeling algorithms (for example, to simulate air flow). It is essential at this stage to synthesize a realistic sound concept of which the constituent parts can be used in the construction of a new appliance.

6.4. Evaluation In the evaluation phase, listeners will judge the sounds in two ways. First, the sounds are evaluated using the aforementioned online measuring technique. Because this technique takes a long time, only a small selection of attributes will be used. Secondly, after

86

Product Experience

a listener has completely listened to a sound, it is evaluated by rating a larger list of attributes. In this way, the holistic experience of the sounds is measured. The attributes will reflect the desired experience. Together these evaluations will determine if the new sound design concepts fulfill the required experience. If not, then new concepts will be developed and the cycle starts all over again.

7. CONCLUSION This chapter presents an overview of the existing literature that is relevant for the understanding of the experience of product sounds and their design. It also contains a positioning to clarify what ‘good’ sound design can mean in the experience of industrial products. However, several topics have not been tackled. For example, the design of intentional sounds has not been discussed at full length, although the design of these sounds offers designers a lot of freedom. In fact, it should be possible to design these sounds in such a way that they would evoke exactly the preferred experience. In this way, the visual and auditory design could complement or strengthen the experience. Furthermore, in Section 6 the redesign of a sound has been discussed. In other words, the sound of an existing product has been improved to evoke a preferred experience. However, in the conceptual phase of the design process for entirely new products it is much harder to include sound design, because there is a lack of tools and methods. Just like sketching tools for visual design, a tool for sketching a sound would help designers in this conceptual phase. This type of tool would allow the design of a sound on the basis of the desired product experience. In this way, sound design would be an integral part in the development process for new products. Although most people are confronted with the sound of products throughout their daily life, the research into the auditory processing of these sounds, the experience they evoke and how they can be changed is limited. The challenge lies in that one has to try to integrate knowledge from several disciplines (psychology, acoustics, psychophysics, music) and to disclose this in such a way that designers can use it. The current chapter started with the question ‘Whether to be silent’ is the best to develop sounds for products. I hope it has been demonstrated in that there is more to the experience of product sounds than silence.

ACKNOWLEDGMENTS Spectrograms have been made with Praat (Boersma and Weenink, 2006). I would like to thank my colleagues at the Faculty of Industrial Design Engineering at the Delft University of Technology and those in the field of auditory perception for their discussions. Since 2003 I have worked with Elif Özcan and Rolf den Otter, this chapter would not have been possible without their contributions and discussions on this topic.

REFERENCES Allen, J. B. (1999). Psychoacoustics. In: J. G. Webster (Ed.) Wiley encyclopedia of electrical and electronics engineering, pp. 422–437. New York: John Wiley. Aures, W. (1985). Berechnungsverfahren für den sensorischen Wohlklang. Acta Acustica, 59, 130–141. Ballas, J. A. (1993). Common factors in the identification of an assortment of brief everyday sounds. Journal of Experimental Psychology – Human Perception and Performance, 19, 250–267. Ballas, J. A. (2002). What is that sound? Some implications for sound design. Proceedings of Journées Design Sonore. www.sfa.asso.fr/DS2002/procs/ballas.pdf.

The experience of product sounds

87

Ballas, J. A. and Howard, J. H. (1987). Interpreting the language of environmental sounds. Environment And Behavior, 19, 91–114. Benko, U., Petrovcic, J., Juricic, D., Tavcar, J. and Rejec, J. (2005). An approach to fault diagnosis of vacuum cleaner motors based on sound analysis. Mechanical Systems And Signal Processing, 19, 427–445. Benko, U., Petrovcic, J., Juricic, D., Tavcar, J., Rejec, J. and Stefanovska, A. (2004). Fault diagnosis of a vacuum cleaner motor by means of sound analysis. Journal Of Sound And Vibration, 276, 781–806. Björk, E. A. (1985). The perceived quality of natural sounds. Acustica, 57, 185–188. Blauert, J. and Jekosch, U. (1997). Sound-quality evaluation – a multi-layered problem. Acustica, 83, 747–753. Block, F. E., Nuutinen, L. and Ballast, B. (1999). Optimization of alarms: a study on alarm limits, alarm sounds, and false alarms, intended to reduce annoyance. Journal of Clinical Monitoring and Computing, 15, 75–83. Boersma, P. and Weenink, D. (2006). Praat: doing phonetics by computer (Version 4.5.08) [Computer program]. Retrieved December 20, 2006, from http://www.praat.org/ Bornstein, R. F. (1989). Exposure and effect: Overview and meta-analysis of research, 1968–1987. Psychological Bulletin, 106, 265–289. Bouter, P. (2006). The association between the psychoacoustic measurement and subjective measurement of loudness, roughness, sharpness, and noisiness of product sounds. Internal Report, Delft University of Technology. Bowen, D. L. and Lyon, R. H. (2003). Mapping perceptual attributes of sound to product design choices. Noise Control Engineering Journal, 51, 271–279. Bradley, M. M. and Lang, P. J. (2000). Affective reactions to acoustic stimuli. Psychophysiology, 37, 204–215. Brandl, F. K., Biermayer, W. and Pfluger, M. (1999). Objective assessment of vehicle interior noise quality. Sound and Vibration, 33, 20–28. Bregman, A. S. (1994). Auditory scene analysis: the perceptual organization of sound. Boston: The MIT Press. Carello, C., Anderson, K. L. and Kunkler-Peck, A. J. (1998). Perception of object length by sound. Psychological Science, 9, 211–214. Chambrin, M. C. (2001). Alarms in the intensive care unit: how can the number of false alarms be reduced? Critical Care, 5, 184–188. Collins, M. (2001). The sound and the future. The engine design of the Harley-Davidson. I.D. magazine, 52–57. Desmet, P. M. A. (2002). Designing emotions. Doctoral Dissertation, Delft University of Technology, Delft, The Netherlands. De Wit, M. G. M., Verbunt, J. P. M. and Vael, J. E. M. (1993). From noise and vibration to functional sound. Philips Research Bulletin on Sands, 1–4. Dickreiter, M. (1984). Mikrofon-Aufnahmetechnik. Stuttgart: Hirzel Verlag. Edworthy, J. and Meredith, C. S. (1994). Cognitive-psychology and the design of alarm sounds. Medical Engineering and Physics, 16(6), 445–449. Edworthy, J. and Adams, A. (1996). Warning design: a research prospective. London: Taylor and Francis. Edworthy, J. and Hards, R. (1999). Learning auditory warnings: the effects of sound type verbal labeling and imagery on the identification of alarm sounds. International Journal of Industrial Ergonomics, 24, 603–618. Edworthy, J., Hellier, E. and Hards, R. (1995). The semantic associations of acoustic parameters commonly used in the design of auditory information and warning signals. Ergonomics, 38, 2341–2361. Edworthy, J., Loxley, S. and Dennis, I. (1991). Improving auditory warning design – relationship between warning sound parameters and perceived urgency. Human Factors, 33, 205–231. Edworthy, J. and Stanton, N. (1995). A user-centered approach to the design and evaluation of auditory warning signals.1. Methodology. Ergonomics, 38, 2262–2280. Fastl, H. and Zwicker, E. (2007). Psychoacoustics. Facts and models (3rd Ed.) Berlin: Springer Verlag. Fletcher, H. (1940). Auditory patterns. Review of Modern Physics, 12, 47–66. Fletcher, H. (1962). Quality of piano tones. Journal of the Acoustical Society of America, 34, 749–761. Friesdorf, W., Buss, B. and Gobel, M. (1999). Monitoring alarms – the key to patients’ safety in the ICU? Intensive Care Medicine, 25, 1350–1352. Gaver, W. W. (1986). Auditory Icons: Using sound in computer interfaces. Human–Computer Interaction, 2, 167–177. Gaver, W. W. (1993a). How do we hear in the world – Explorations in ecological acoustics. Ecological Psychology, 5, 285–313. Gaver, W. W. (1993b). What in the world do we hear. 1. An ecological approach to auditory event perception. Ecological Psychology, 5, 1–29. Giordano, B. L. and McAdams, S. (2006). Material identification of real impact sounds: Effects of size variation in steel, glass, wood, and Plexiglas plates. Journal of the Acoustical Society of America, 119, 1171–1181. Grassi, M. (2005). Do we hear size or sound? Balls dropped on plates. Perception and Psychophysics, 67, 274–284.

88

Product Experience

Handel, S. (1991). Listening (2nd Ed.) Boston: The MIT Press. Hermes D. J. (1998) Auditory material perception. IPO Annual Progress Report 33, 95–102. Hirsh, I. J. and Watson, C. S. (1996). Auditory psychophysics and perception. Annual Review of Psychology, 47, 461–484. Houben, M. M. J., Kohlrausch, A. and Hermes, D. J. (2004). Perception of the size and speed of rolling balls by sound. Speech Communication, 43, 331–345. Houben, M. M. J., Kohlrausch, A. and Hermes, D. J. (2005). The contribution of spectral and temporal information to the auditory perception of the size and speed of rolling balls. Acta Acustica United with Acustica, 91, 1007–1015. Ih, J. G., Lim, D. H., Shin, S. H. and Park, Y. (2003). Experimental design and assessment of product sound quality: application to a vacuum cleaner. Noise Control Engineering Journal, 51, 244–252. Izard, C. E. (1977). Human emotions. New York: Plenum. Jekosch, U. (2004). Basic concepts and terms of ‘quality’, reconsidered in the context of product-sound quality. Acta Acustica United with Acustica, 90, 999–1006. Jordan, P. W. and Engelen, H. (1999). Sound design for consumer products. Paper presented at the Proceedings of Stockholm, Hey Listen! Stockholm. Keiper, W. (1997). Sound quality evaluation in the product cycle. Acustica, 83, 784–788. Keiper, W. (1999). Psychoacoustics in industry: Needs and benefit. Acustica, 85, 665–666. Kidd, G. R. and Watson, C. S. (2003). The perceptual dimensionality of environmental sounds. Noise Control Engineering Journal, 51, 216–231. Kunkler-Peck, A. J. and Turvey, M. T. (2000). Hearing shape. Journal of Experimental Psychology – Human Perception and Performance, 26, 279–294. Lim, T. C. (2001). Correlations between deficiencies in power window systems influencing sound quality and some psychoacoustic metrics. Applied Acoustics, 62, 1025–1047. Lindeman, H. E. and Stekelenburg, M. (1981). Lawaai (Noise). Stafleu’s Wetenschapelijke Uitgeversmaatschappij. Louwers, M. T. (1993). Introductory research on the effects of operational sounds on the quality perception of consumer products. IPO Report 932, Eindhoven. Lyon, R. H. (2000). Designing for product sound quality. New York: Dekker. Lyon, R. H. (2003). Product sound quality – from perception to design. Sound and Vibration, 37, 18–23. Marcell, M. E., Borella, D., Greene, M., Kerr, E. and Rogers, S. (2000). Confrontation naming of environmental sounds. Journal of Clinical and Experimental Neuropsychology, 22, 830–864. McAdams, S. (1993). Recognition of sound sources and events. In: S. McAdams and E. Bigand (Eds.) Thinking in sound: the cognitive psychology of human audition. Oxford: Oxford University Press. Meunier, S., Habault, D. and Canevet, G. (2001). Auditory evaluation of sound signals radiated by a vibrating surface. Journal of Sound and Vibration, 247, 897–915. Momtahan, K., Hetu, R. and Tanssley, B. (1993). Audibility and identification of auditory alarms in the operating-room and intensive-care unit. Ergonomics, 36, 1159–1176. Moore, B. C. J. (2003). An introduction into the psychology of hearing (5th Ed.) Amsterdam: Academic Press. Ortony, A., Clore, G. L. and Collins A. (1988). The cognitive structure of emotions. Cambridge: Cambridge University Press. Özcan, E. and Van Egmond, R. (2006). Product sound design and application: overview. In: M. Karlsson (Ed.) Design and Emotion 2006 Conference Proceedings, pp. 1–19. Göteborg: Chalmers University. Özcan, E. and Van Egmond, R. (2007). Memory for product sounds: The effect of label type. Acta Psychologica, 126, 196–215. Park, C. W., Lee, S. I. and Lee, S. J. (2004). Effect of suction nozzle modification on the performance and aeroacoustic noise of a vacuum cleaner. Ksme International Journal, 18, 1648–1660. Parncutt, R. H. (1989). Harmony: A Psychoacoustical Approach. Berlin: Springer Verlag, Plutchik, R. (1980). Emotion: A psychoevolutionary synthesis. New York: Harper and Row. Povel, D. J. and Essens, P. (1985). Perception of temporal patterns. Music Perception, 2, 411–440. Rittmueller, S. P., Mann, J. A. and Holger, D. K. (1997). Iterative design method to reduce the exhaust and venting noise of a vacuum cleaner motor with minimum use of absorptive foam. Noise Control Engineering Journal, 45, 63–68. Solomon, L. N. (1959). Search for physical correlates to psychological dimensions of sounds. Journal of the Acoustical Society of America, 31, 492–497. Solsona, J. F., Altaba, C., Maull, E., Rodriguez, L., Bosque, C. and Mulero, A. (2001). Are auditory warnings in the intensive care unit properly adjusted? Journal of Advanced Nursing, 35, 402–406. Van Balken, J. (2001). Designing coffeemaker sounds. Exploring the experience of Senseo’s sound. Unpublished MSc Thesis, Delft University of Technology.

The experience of product sounds

89

Van Egmond, R. (2004). Emotional experience of frequency modulated sounds: Implications for the design of alarm sounds. In: D. de Waard, K. A. Brookhuis and C. M. Weikert (Eds.) Human factors in design, pp. 345–356. Maastricht, The Netherlands: Shaker Publishing. Van Egmond, R. (2006). Designing an emotional experience for product sounds. In: M. Karlsson (Ed.) Design and Emotion 2006 Conference Proceedings, pp. 1–19. Göteborg: Chalmers University. Van Egmond, R., Desmet, P., Van Der Helm, A. (2004). Basic and cognitive emotions in sound perception. In: A. Kurtgözü (Ed.) Proceedings of the Fourth International Conference on Design and Emotion, pp. 1–11. Ankara: METU. Västfjäll, D., Gulbol, M. A., Kleiner, M. and Garling, T. (2002). Affective evaluations of and reactions to exterior and interior vehicle auditory quality. Journal of Sound and Vibration, 255, 501–518. Von Békésy, G. and Rosenblith, W. A. (1951). The mechanical properties of the inner ear. In: S. S. Steven (Ed.) The handbook of experimental psychology. New York: John Wiley. Von Bismarck, G. (1974a). Sharpness as an attribute of the timbre of steady sounds. Acustica, 30, 159–172. Von Bismarck, G. (1974b). Timbre of steady sounds: A factorial investigation of its verbal attributes. Acustica, 30, 146–159. Welch, J. (1999). Auditory alarms in intensive care. In: N. A. Stanton and J. Edworthy (Eds.) Human factors in auditory warnings. Aldershot: Ashgate, 319–342. Yan Xiao and Seagull, F. J. (2003). An analysis of problems with auditory alarms: Defining the roles of alarms in process monitoring tasks. In: E. Haas and J. Edworthy (Eds.) The ergonomics of sound: Selections from Human Factors and Ergonomics Society Annual Meetings, 1985–2000. Santa Monica: Human Factors and Ergonomics Society, 43–47. Yost, W. A. (2007). Fundamentals of Hearing: An Introduction (5th Ed.). New York: Academic Press. Zajonc, R. B. (1968). Attitudinal effects of mere exposure. Journal of Personality and Social Psychology Monographs, 9, 1–27. Zampini, M., Guest, S. and Spence, C. (2003). The role of auditory cues in modulating the perception of electric toothbrushes. Journal of Dental Research, 82(11), 929–932.

4

TASTE, SMELL AND CHEMESTHESIS IN PRODUCT EXPERIENCE ARMAND V. CARDELLO Natick Soldier Center, MA, USA

PAUL M. WISE Monell Chemical Senses Center, PA, USA

1. INTRODUCTION 1.1. Importance of taste, smell and chemesthesis to product experience The oldest human senses are those that respond to chemicals in the environment. A primitive amoeba responds to a drop of vinegar placed in a Petri dish by moving away from it. Anthropomorphically, we could say that the amoeba ‘experiences’ this unpleasant ‘product’ and behaves in accordance with its negative consequences. Among mammals, the chemical senses play no less of an important role. In humans, the senses of taste, smell, and chemesthesis have become secondary to those of vision and audition, but they still play a pre-eminent role in how we experience a wide range of everyday objects, including all foods and beverages, as well as many important products in the perfume, healthcare and personal product markets. In spite of the importance of the chemical senses to human experience, they have not received the scientific attention that the ‘higher’ senses have received. However, exciting discoveries during the past decade have produced greater interest in the chemical senses. Among these discoveries have been the identification of receptor proteins that encode odorant qualities, and the discovery of basic tastes beyond those we were taught as children. These and other scientific findings have opened the doors to further discoveries that will enable a better understanding of how we experience products using the chemical senses and how we might design products that better meet the utilitarian and aesthetic needs of today’s user. Product Experience Copyright © 2008 Elsevier Ltd.

91

92

Product Experience

1.2. Chemical senses as integrated perceptual systems Scientists working in the areas of physiology, biochemistry, and sensory psychology have traditionally treated sensory systems as discrete, passive, linear systems. Although this view is still predominant in the area, from the perspective of ‘product experience,’ J. J. Gibson’s analysis of human sensory systems is worth considering. Gibson (1966) viewed the human senses as ‘perceptual systems’, i.e. active interrelated systems designed to extract information from the environment for a purpose. In the case of smell, he described this purpose as ‘detecting things at a distance by means of their odors’. In the case of taste, the purpose was ‘the control of ingestion … standing guard over eating, selecting certain things and rejecting others.’ Taken together, the primary purpose ascribed by Gibson to the chemical senses was appetitive or ‘food getting’ (Gibson, 1966, p. 138). Gibson’s notion that the human senses are information gathering systems has relevance to the study of product experience. When we encounter products, whether they are food products on a supermarket shelf or in a restaurant, perfumes in a department store, automobiles in a showroom, or apparel in a clothing store, we are engaged in the process of information gathering for a purpose. The purpose is decision-making related to purchase, consumption or use of the product. However, obtaining sensory information about a product through the senses is only part of the story. Sensory information almost always occurs within a context of other information. Food items in a restaurant occur within a context that includes the price of the item, the appearance and cleanliness of the restaurant, and the perceived nutritive value of the food. Perfumes and automobiles are perceived within a context of marketing images and sales offers, while clothing and certain personal care items are perceived within gender-based, social, and perhaps moral or religious contexts. Thus, the experience of a product must take into account not only its sensory attributes, but the situational, cognitive, social, and cultural lenses through which that information must pass before a judgment can be made regarding its goodness, pleasantness, desirability for purchase, consumption or use. In this chapter we examine the role of three sensory systems that signal the presence of chemicals in the environment, and that influence our product experiences. Taste (gustation) is the sensory system that detects chemicals dissolved in the liquids we drink and the foods we eat. Smell (olfaction) detects air-born chemicals emitted by natural, living, and synthetic products. Chemesthesis detects chemical irritants via the same skin-sense nerves that give us sensitivity to touch, temperature, and pain. These three sensory systems have different specialties, but work together to contribute to our overall experience of the products we use. We will begin by outlining the anatomical, physiological, and biochemical mechanisms that underlie chemosensory experience. This will be followed by a discussion of fundamental sensory and perceptual phenomena that operate when products are experienced through these senses. From there, we will examine how chemosensory product experiences are quantified for practical applications in industry. Lastly, we will examine the roles of non-sensory information, beliefs, expectations and product context on experience, concluding with a discussion of how age, gender, social, and cultural variables can influence chemosensory product experience.

2. TASTE 2.1. The experience of taste Taste is the sensory experience that results from stimulation of chemoreceptors located on the tongue, palate, pharynx, larynx, and other areas of the oral cavity. The human

Taste, smell and chemesthesis

93

experience of taste includes at least five distinct qualities: Bitter, sour, sweet, salty and umami taste, all of which play a significant role in product experience and in helping us to decide what potential foods and/or beverages we should ingest or reject. In addition to these qualities of taste experience, there are two other dimensions of sensory experience that apply to all sensory systems. These are the intensity or magnitude of the taste, smell, etc. and its affective (like/dislike) dimension. These three elements of sensory experience will recur throughout our discussions in this chapter. As noted in the Introduction, the chemical senses evolved as important survival mechanisms, signaling stimuli in the environment that should be avoided and those that are safe to approach and/or consume. The sensory experience of ‘bitterness’ signals the potential presence of toxic substances and triggers rejection. Sourness signals the presence of acids and can be an indicator of spoiled food or unripe fruit, which may also lead to product rejection. Sweetness, on the other hand, indicates the presence of calories needed for survival and triggers ingestion. Saltiness indicates the presence of sodium and other ions needed for nervous function and fluid homeostasis and can also trigger ingestive responses. Lastly, umami taste (savoriness), which is exemplified by the taste of monosodium glutamate (MSG), chicken broth or aged cheddar cheese, suggests the presence of amino acids, the building blocks of life, and leads to ingestion. All of the taste qualities that lead to ingestion also trigger physiological or ‘cephalic phase’ responses that prepare the body to process food, including production of stomach acid and the release of insulin (see Mattes, 1987).

2.2. The mechanisms of taste experience Oral anatomy Although taste receptors are located throughout the oral cavity, many are clustered on taste papillae located on the dorsal surface of the tongue. The most plentiful of these are fungiform papillae, which are mushroom-shaped structures distributed over the anterior tongue. Foliate papillae consist of folds located on the margins of the tongue, while circumvallate papillae are large, circular structures forming a chevron near the back of the tongue. A fourth type of papilla, filiform, is widely distributed over the dorsal surface, but houses no taste buds. Taste buds are rosebud shaped groups of cells (Smith and Margolskee, 2001). Each bud contains about 100 cells, some of which are taste receptor cells. Tastants (molecules of products in solution) pass through a small pore at the top of the taste bud and contact finger-like projections from the taste receptor cells. These projections (microvilli) contain receptor-proteins. Receptors for sweet, savory (umami), and bitter (see below) are probably expressed in different subsets of taste receptor cells, although a given cell may express multiple types of bitter receptors (Zhao et al., 2003; Mueller et al., 2005; Breslin and Huang, 2006). Accordingly, a given taste receptor cell may respond to a number of different compounds, but those compounds will tend to signal the same taste quality (though this picture may change when sour and salty receptors are identified). Receptor mechanisms for experiencing taste Over the last decade, researchers have discovered receptor-proteins for sweet, umami (savory), and bitter tastes (Lindemann, 2001; Mombaerts, 2004; Breslin and Huang, 2006). When these proteins ‘recognize’ particular chemicals, they initiate a cascade of chemical reactions in the taste receptor cell that eventually causes the cell to respond. Receptors for salty and sour tastes have been harder to pinpoint (Lindemann, 2001;

94

Product Experience

Breslin and Huang, 2006). Candidate receptors include ion channels that allow ions to pass through cell membranes. The search for sour and salty receptors continues, and may lead to greater understanding soon (see Liu and Simon, 2001; Huang et al., 2006). Individual differences in the genes that encode receptor-proteins can lead to differences in taste experience (see Breslin and Huang, 2006). Many readers will remember classroom demonstrations using the bitter substance propylthiouracil (PROP). Those who express one form of a particular bitter receptor-protein can detect PROP at much lower concentrations than those who express other forms (see Bufe et al., 2005). The former individuals prove more sensitive to certain other bitter chemicals, including compounds present in a number of healthful vegetables, and are less likely to consume such vegetables (Bell and Tepper, 2006; Dinehart et al., 2006; but also see Yackinous and Guinard, 2002). Other studies suggest that PROP-sensitive people experience greater bitterness and burn from alcoholic beverages and that this sensory response, in turn, reduces consumption (Duffy et al., 2004; Lanier et al., 2005). Still other studies suggest that sensitive children prefer higher levels of sugar in their foods, perhaps to mask the more intense bitter sensations that they experience (Mennella et al., 2005). Differences in sweet preference are also related to genetic variation in the receptor-proteins for sweet taste (McDaniel and Reed, 2004; Li et al., 2005). In short, some aspects of everyday product experience can be predicted by variation in the genes that encode taste receptor-proteins. Knowledge of variations in genes may suggest specific product formulations targeted toward particular phenotypes. For example, as technology progresses, a consumer might someday scan a card with information about his/her genes for taste and smell receptors and receive a cup of coffee with just the right balance of taste and aroma compounds to optimize his/her personal flavor experience. Processing taste receptor information into taste experience The first relay station for taste signals coming from receptors in the oral cavity lies in the brain stem (Smith-Swintosky et al., 1991; Smith and Shepherd, 1999). Connections to other areas in the hindbrain mediate very basic motor and visceral responses connected to acceptance or rejection, including characteristic facial expressions in response to bitter and sweet tastes (see Mattes, 1987) and certain cephalic phase responses, like salivation and increased production of stomach acid. In primate models, and thus perhaps in humans, projections from the brainstem go to primary taste areas in the cortex via the thalamus (Small, 2006). Research suggests that the primary taste cortex plays a role in coding taste intensity (Smith-Swintosky et al., 1991). Next, connections pass to other nearby areas (Baylis et al., 1995; Carmichael and Price, 1995). Responses in the amygdala seem to be driven by interactions among intensity, pleasantness, and familiarity (Small et al., 1997, 2003). Thus, the amygdala might also play an important role in representing stimulus salience (Small, 2006). Responses in the orbitofrontal cortex (OFC) do not seem closely tied to intensity, but different patterns of activation emerge for pleasant versus unpleasant tastes (Small et al., 2003). As will be discussed elsewhere in this chapter, the OFC seems to represent reward value, and probably plays a key role in decisions about chemosensory stimuli. The neural basis of perceived taste quality remains unclear, but the experience of quality probably begins in higher (cortical) taste areas. In these areas, quality is represented in patterns of activity across neurons, and perhaps in timing of responses (Hallock and Di Lorenzo, 2006; Lemon and Smith, 2006). The primary taste cortex and the OFC also receive input from other sensory modalities, including somatosensation and smell (Rolls and Baylis, 1994; Cerf-Ducastel et al., 2001; Savic et al., 2002). In a brain imaging study, Small and colleagues (1997) found

Taste, smell and chemesthesis

95

that both odors and tastes (presented separately) caused activation in primary taste cortex and the OFC. However, when subjects received some taste-odor pairs simultaneously, total activation exceeded the sum of the activations from either stimulus alone. This suggests a special representation in these areas for flavor (the combined effect of taste and smell). Interestingly, both physiological and behavioral studies suggest that effective combinations between tastes and smells only occur for congruent pairs, e.g., vanilla odor plus sweet taste, but not vanilla odor plus salty taste (Small et al., 1997, 2004; Dalton et al., 2000). Learning plays an important role in what sensations yield a combined effect (Diamond et al., 2005).

3. TASTE: BASIC PHENOMENA OF TASTE EXPERIENCE 3.1. Taste adaptation To understand product experience, one must go beyond genetics and physiology to look at the fundamental perceptual phenomena that underlie taste experience. One such phenomenon that occurs in all sensory systems is adaptation, i.e. the sensory experience grows weaker during sustained stimulation. Figure 4.1 shows a typical adaptation curve for taste. When a taste stimulus comes into contact with receptors, the minimum concentration of that stimulus that can be detected (its absolute threshold) is elevated. With continued stimulation this threshold concentration continues to increase until it asymptotes after about one minute. When the stimulus is removed, the threshold concentration begins to fall dramatically, returning to its baseline level after about 30 seconds. This phenomenon is the reason why your first taste of a sour ball candy can be unbearably sour, but as you continue to suck on it, it becomes less sour. Taste ‘adaptation’ also occurs when eating one product reduces the perceived taste intensity of a different product that has the same or similar taste, such as tasting a bite of a Granny Smith apple after having the sour ball candy in your mouth. This latter

FIGURE 4.1 A typical taste adaptation curve (from Matlin and Foley, 1992).

96

Product Experience

phenomenon is known as cross-adaptation. When products that have different tastes are eaten one after another, an increase in the perceived intensity of the second product may occur. One of the most well studied examples of this cross-enhancement of taste experience reveals itself in the bitter taste of distilled water. The bitterness that results from drinking distilled water is due to the fact that, normally, the salt in saliva adapts taste receptors in the oral cavity. When distilled water is consumed, the adapting saliva is removed, producing a characteristic bitter ‘water taste’ (Bartoshuk et al., 1964). In fact, distilled water can take on a variety of different tastes depending upon the nature of the pre-adapting taste (McBurney and Schick, 1971). Thus, our taste product experiences depend not only on our genetic make-up, physiology, and the product being consumed, but also on other products that the user has recently consumed.

3.2. Repetitive taste experiences and liking A similar, but different, phenomenon to taste adaptation occurs with our affective (liking) experiences following repeated exposure to a food or beverage. Cabanac (1971) showed that the pleasantness of a stimulus increases as a function of how much the stimulus satisfies a biological need. Thus, liking for a food will increase the hungrier is the individual. He termed this effect ‘allesthesia’. However, when a food is consumed repeatedly over time, its pleasantness declines relative to foods that have not been eaten (Rolls et al., 1981, 1988). This change in the affective experience of a food or beverage after repeated exposure is known as ‘sensory specific satiety’. In simple terms, it is the reason why, if you eat the same meal for several days in a row, you start to desire other foods. It is the basis for our need for variety in taste experiences. Sensory specific satiety was first viewed to be related to physiological satiation. However, since the phenomenon can occur in response to the taste of foods and beverages that are not consumed (Murphy, 1982; Drewnowski et al., 1982) it seems clear that this is a perceptual effect. Although there is evidence that some sensory attributes and, therefore, some foods/beverages are less susceptible to sensory specific satiety (Drewnowski et al., 1982; Johnson and Vickers, 1991), the phenomenon is a powerful one that shapes our daily experience of foods, how much we like them, and whether and how much we eat of them (Swithers and Hall, 1994; Hetherington and Rolls, 1996).

3.3. Combined taste experiences Most foods and beverages are comprised of a complex combination of ingredients. Each of these contributes their own unique taste to the product. So what happens to our taste experiences when these separate ingredients are combined? Most commonly, the perceived intensity of the mixed tastes will be less than the total of the perceived intensities of the unmixed tastes (Lawless, 1987; Kroeze, 1990). This phenomenon is known as ‘taste suppression’. A number of studies, including those that have shown that the suppression occurs even when different tastes are presented to different sides of the tongue, support the fact that suppression occurs in the brain and is not due to adaptation of taste receptors in the mouth (Lawless, 1979; Kroeze and Bartoshuk, 1985). However, the suppression that occurs when different tastes are combined does not always affect each taste equally. For example, if you mix certain bitter tastants with salty ones, the bitterness is suppressed, but the salty experience typically remains unchanged (Breslin and Beauchamp, 1995). Similarly, if you mix certain bitter compounds with sour ones, you can get enhancement of the taste. Breslin (1996), in a review of the interactions that occur between bitter, salty and sour tastes, concluded that: (1) salty and sour tastes

Taste, smell and chemesthesis

97

suppress each other at high concentrations, but enhance each other at lower concentrations; (2) bitter and sour tastes either suppress or enhance each other, depending on concentration and other factors; and (3) salty and bitter tastes usually suppress the bitterness, but leave the salty taste unaffected. In many food formulations, ingredients are combined that have the same taste quality. Even here interactions can occur to alter the experienced taste intensity of the product. Thus, combining MSG and 5⬘ ribonucleotides can produce enhancement of the umami taste (Rifkin and Bartoshuk, 1980). Similarly, combining high intensity sweeteners will produce enhancement of their sweetness (Ayya and Lawless, 1992; Schiffman et al., 1995). Combining ingredients with either the same or different tastes can produce a variety of interactions, depending on the taste compounds that are mixed, their absolute and relative concentrations, and other factors. The art of the food product developer is to be aware of these mutually interactive effects and to compensate for their impact in order to optimize the sensory experience of the product for the consumer. But can the quality (rather than the intensity) of taste experience change when two or more tastes are combined? On this point most research supports the notion that the components of any taste mixture are still perceived separately and that taste is, therefore, an ‘analytic’ sense. However, the data are not singular in this regard. Not all taste experience comes from foods. One common ‘taste interaction’ that many people experience is how toothpaste ‘changes’ the taste of foods after brushing. At first glance, one might think that the ‘taste’ of the toothpaste is somehow responsible. However, brushing changes a variety of factors that control the general milieu of the mouth (e.g. amount and composition of saliva, presence of minute food particles, micro flora, etc.), so the change in the taste of foods may be due to multiple factors. Further, it must be remembered that much of what we experience as ‘taste’ is really smell. On this point, a recent study of the effects of toothpaste on the flavor of foods (Allison and Chambers, 2005) showed that toothpaste did not affect the perceived taste experience of food, rather it changed the perceived olfactory attributes of the food. For the flavor of orange juice, this effect was seen for up to an hour after brushing.

3.4. Modifying taste experiences Can other sensory properties of a product influence our experience of its taste, and is it possible that our taste experiences can alter perception of other sensory properties? The answer to both of these questions is ‘yes’. Perhaps the most ubiquitous interactions with taste experiences are those that occur with odors. Research by Murphy, e.g. Murphy and Cain (1980), has shown that the overall intensity of taste-odor mixtures presented orally are perceived to be a simple additive function of the components separately. However, when samples of the odorants in solution are presented alone to the mouth, subjects perceive the experience to be one of taste, not odor. The effect, which they termed ‘taste-referred olfaction’, is illusory, because pinching the nose to block retronasal passage of the volatile compounds eliminates the effect. Similar effects have been demonstrated by others with some evidence showing that the more qualitatively similar the tastes and odors, the more likely is the effect (Frank and Byram, 1988; Dalton et al., 2000; Delwiche and Heffelfinger, 2005). Figure 4.2 is taken from research by Labbe et al. (2006), in which the investigators added either cacao or vanilla flavorants (odorants) to a cocoa beverage. As can be seen, the addition of a cocoa odor (top panel) increased both the odor intensity and the bitter taste of the beverage, while depressing its sweetness. Adding vanilla odor (bottom panel) increased aroma and also elevated sweetness at the highest concentration. Thus, the odor

98

Product Experience

FIGURE 4.2 The impact of the addition of three different concentrations of cocoa (top panel) or vanilla flavorings (bottom panel) on taste and olfactory attributes (and body) of cocoa beverage. The stars indicate significant differences at the levels specified above them (from Labbe et al., 2006).

imparted by flavorants in food can differentially affect our experiences of the product’s taste qualities. In commercial practice, the phenomenon of taste-referred olfaction is now being used to ‘flavor’ bottled water by simply embedding odorants into the bottle caps (ScentSational Technologies, LLC, Jenkintown, PA), and quite surprisingly, even imagined odors can produce illusory taste experiences (Algom et al., 1993; Djordjevic et al., 2004). In the latter study, the investigators measured the accuracy of detecting sucrose in water versus plain water during simultaneous presentation of real or imagined ham and strawberry odors. Figure 4.3 shows the results of this study. The accuracy (% correct) of detecting sweetness in the forced choice trials was enhanced in the presence of actual strawberry odor and depressed in the presence of ham odor. Nearly identical effects were found when subjects merely imagined these odors. Chemesthesis, the tactile feel of chemical irritants, can also alter our taste experiences of food products. Several researchers have shown that a rinse of either piperine or capsaicin (black or red pepper solutions) reduces the perceived intensity of sweet, sour, salty and bitter tastes (Lawless and Stevens, 1984; Lawless et al., 1985) Similarly, Gilmore and Green (1993) examined the effect of a capsaicin rinse on the perceived saltiness, sourness, and irritation of high concentration NaCl and citric acid. Their results showed a significant desensitization effect of the capsaicin on both the experienced taste intensity and irritation of these compounds. Thus, it seems that chemesthesis has broad

99

Taste, smell and chemesthesis

100

Detection of sucrose (percentage)

90

80 Strawberry Ham No odor 70

60

50 Perception

Imagery

FIGURE 4.3 Accuracy of detecting sucrose as a function of real (perception) and imagined (imagery) strawberry odor, ham odor, or no odor (redrawn from Djordjevic et al., 2004).

interactive effects with basic tastes and that cross-adaptation can occur between the irritant effects of capsaicin and other taste-related oral irritants. As far as taste experience affecting other perceived aspects of a product, research by Christensen (1980) showed that increasing the sweet taste of solutions resulted in an increase in the experience of their thickness or viscosity. One explanation often given for this ‘cross-modal’ interaction is that, through learning, people come to associate high sugar levels with greater thickness in beverages, e.g. tea sweetened with honey. However, in today’s marketplace, the prevalence of high intensity sweeteners that impart little viscosity to a solution diminishes the likelihood of such a learned association. The taste of certain foods can also alter our experience of somesthetic sensations. A common example of this is how the sweetness of sugar can reduce the experience of burning that results from eating black or red pepper spices (Stevens and Lawless, 1986; Nasrawi and Pangborn, 1989). (See Section 6 for a broader discussion of chemesthetic effects.)

3.5. Innate and early taste experiences and preference How much of our experiences and product preferences is learned and how much is innate? This is a complex question because functioning taste receptors appear in the human fetus at around 14 weeks. Thus, by birth, infants have been exposed to a wide variety of taste compounds present in the amniotic fluid. However, the work cited earlier on the characteristic facial expressions of infants in response to bitter and sweet compounds provides support to the notion that some aspects of our adult taste experiences may be hard-wired. Other research also supports this view. Desor et al. (1973, 1975) measured the amount of solution that young infants consumed and found a strong ‘innate’ preference for sweetness, but little difference in their intake of sour, salty, and bitter solutions. Desor et al. (1975) also demonstrated that adding citric acid to a sweet

100

Product Experience

solution suppressed the infants intake of the mixture, suggesting that the sour experience produced by the citric acid was innately unpleasant. In a review of the area, Cowart and Beauchamp (1990) concluded that there is good evidence of an innate preference for sweet taste and reasonable evidence supporting an innate rejection of sour tastes. While it seems that many everyday products like sour milk, bitter coffee, and sweet pastry may produce pleasurable or unpleasurable experiences as a result of factors present at birth, many other product experiences are modifiable at an early age. For example, Beauchamp et al. (1986) found that infants, who are relatively insensitive to salty tastes at birth, develop a preference for it within the first four months of life. Cowart and Beauchamp (1990) have argued that this is the result of developmental changes, not learning. Other researchers have shown that exposure to certain tastes early in life can alter our taste experiences and preferences later in life (Birch and Marlin, 1982; Harris and Booth, 1987; Beauchamp and Cowart, 1993). The reader is referred to Ganchrow (1995) for a more detailed analysis of the development of human taste experience and preference.

3.6. The effects of learning on taste experience While the perceptual and hedonic natures of many taste experiences are innate, others are learned. An example of this is the development of taste aversions. Researchers have shown that a single pairing of food with the experience of illness is enough to create a lasting aversion to the food (Garb and Stunkard, 1974; Bernstein and Webster, 1980; Pelchat and Rozin, 1982). In many cases this aversion develops within minutes, although the delay between eating the suspect food and illness can be as long as 12 hours. In one survey of college students, 57% reported to have at least one learned food aversion (Midkiff and Bernstein, 1985). Not unexpectedly for that sample population, many of these aversions were to the taste of an alcoholic beverage that had been over-consumed, resulting in alcohol-induced illness. Bartoshuk and Wolfe (1990) have suggested that the aversions that form to foods result from an association with the food’s odor, not its taste. This argument makes sense from a survival perspective, because to be of future advantage to the organism, the aversion to illness-producing foods should occur as early as possible upon encountering the food. By all accounts, taste aversion learning is a primitive form of learning (Schafe and Bernstein, 1996) that can powerfully shape our product experiences. An opposite ‘medicine effect’ in which taste experiences associated with recuperation from illness become preferred has been shown in animals, but it has been difficult to demonstrate in humans. The reader is referred to Chambers and Bernstein (1995) and Schafe and Bernstein (1996) for reviews of this area. In the case of learned taste preferences, mere exposure to novel or disliked tastes (and odors) can increase one’s liking and intake of them (Cain and Johnson, 1978; Davis and Porter, 1991; Stein et al., 2003). This form of simple repetitive exposure to tastes and smells can serve as an important strategy for overcoming neophobia (the reluctance to try new foods or other products). This is why parents, usually unwittingly, continue to offer their children foods that they dislike, e.g. vegetables, in an attempt to ‘overcome’ their disliking with time and repeated exposure. The influence that repeated exposure can have on our product experiences has been powerfully demonstrated in research by Rozin and others on the development of liking for chili pepper. In one such study, Rozin and Schiller (1980) demonstrated that simple repeated exposure to increasing levels of chili in food is all that is needed to reduce the aversion to its burning sensation and to create a preference for it. Other studies have shown that a neutral or disliked taste paired with a pleasant taste experience will increase the preference for the neutral flavor (Zellner et al., 1983; Breslin

Taste, smell and chemesthesis

101

et al., 1990; Capaldi, 1996). This flavor–flavor conditioning occurs unknowingly among coffee and tea drinkers who often initiate their coffee or tea drinking by adding liberal amounts of sugar or non-nutritive sweeteners to offset the bitterness of the coffee/tea. With time, the pairing of the pleasantness of the sweet taste with that of the taste of the coffee/ tea produces a greater liking for the taste of them alone, resulting in the gradual reduction of the added sweetener. The opposite effect of associating a non-preferred taste, e.g. bitter, with a preferred taste, e.g. sweet, has been demonstrated to reduce the liking for the preferred taste (Capaldi and Hunter, 1994). Still other research has shown that flavor preferences can be increased by associating a neutral flavor or food with nutritional consequences (calories) or with feelings of satiety (fullness) (Booth, 1981; Capaldi et al., 1987). Although some taste and hedonic experiences in the real world may be ‘innately’ determined on the basis of our genetic make-up, our needs to avoid toxic substances and to obtain calories have created the ability for our taste experiences to be modified to achieve survival goals. In Section 8 we will discuss how taste and other product experiences can also be modified by such factors as context, information and expectations. The important point to take away from the data discussed here is that our taste experiences of products result from a combination of innate, genetic, cognitive and learned factors. Product experience is only partly to be found in the product. An equally important part is to be found in the consumer and his/her past experiences.

4. SMELL 4.1. The experience of smell Smell is the sensory experience that results from stimulation of receptors in the olfactory epithelium of the nose by airborne molecules. When these molecules interact with olfactory receptors, experiences such as ‘grassy’, ‘woody’ or ‘musky’ result. Odors in the environment warn us of dangers both near, like spoiled food, and remote, like the smell of smoke from a distant fire. Odors also alert us to more pleasant prospects, such as breakfast cooking downstairs, and they play an important role in social communication.

4.2. The mechanisms of olfactory experience Nasal anatomy Information gathering in the olfactory system begins with the passive (normal breathing) or active (sniffing) arrival of airborne molecules. Upon passage into the nares, odorant molecules interact with specialized olfactory receptor-neurons located in the olfactory epithelium, a small patch of receptor tissue located high in the nasal cavity (Rawson and Yee, 2006). Sniffing is more effective for stimulating smell than natural breathing because sniffs create eddy-currents that drive odorized air up to this epithelium. In fact, some authors believe that the act of sniffing is an important unit of olfactory processing and experience, with each sniff constituting a ‘snapshot’ of the olfactory information in the environment (Kepecs et al., 2006; Mainland and Sobel, 2006). While sniffing causes the passage of inhaled air through the nares (orthonasal), chewing of food causes air from the back of the mouth to be passed upward into the nasal chamber in the reverse direction, through the nasopharynx (retronasal). When one perceives the ‘flavor’ of a food in the mouth, it is its retronasal smell that is the major contributing factor to the experience. When a cold produces nasal congestion, it interferes with the flow of air both through the nares and

102

Product Experience

retronasally, which is why our flavor experience of foods eaten when we have a head cold is so diminished. Receptors for olfactory experience Olfactory receptor neurons extend hair-like cilia into the mucous layer that covers the olfactory epithelium (Lledo et al., 2005; Rawson and Yee, 2006). Like taste receptor cells, these cilia contain specialized receptor-proteins that recognize odor molecules. Recent research has identified many of the genes that encode receptor-proteins (Glusman et al., 2001). Humans have approximately 350 potentially functional genes, which implies that we may have as many functional receptor-proteins (Rawson and Yee, 2006). Research suggests that each type of receptor-protein is expressed by many olfactory receptor neurons, but that each individual receptor neuron expresses only a single type of receptor-protein (Buck, 1996). A given receptor-protein, and therefore the olfactory receptor neuron that expresses that protein, will respond to multiple odorants, and a given odorant will stimulate multiple receptor neurons (Hildebrand and Shepherd, 1997). The overall pattern, or mosaic, of stimulated receptor neurons signals the identity of the molecule at this level of the olfactory system. Recent research on odor receptors provides new insights into consumer olfactory experience. It is clear that odor perception differs substantially among people with a ‘normal’ sense of smell. For example, the compound androstenone, which is found in products like celery, pork, and truffles, is experienced quite differently among individuals. It is described as being either urinous, musky, or floral by different people. Yet, upwards of 50% of the population cannot even smell it (Labows and Wysocki, 1984). Wysocki and Beauchamp (1984) have shown a strong genetic basis for this difference and have demonstrated the involvement of genetic, developmental, and experiential factors (Wysocki and Beauchamp, 1991). Researchers are now attempting to link differences in olfactory experiences to specific mutations in genes that encode receptors. Those who develop consumer olfactory products, like perfume, know that a single formulation will rarely please all consumers. That we differ in our mosaic of smell receptors helps explain why we each seem to live in our own world of scents. As with taste experience, it may soon prove possible to infer much about the olfactory experiences and preferences of an individual by knowing the genes that he or she expresses. Receptors for pheromones Pheromones are chemical signals released by one individual that influence the physiology or behavior of another individual of the same species (see Wysocki and Preti, 2004). Many animals can sense pheromones both with the olfactory system and with a specialized chemosensory organ called the VNO, or vomeronasal organ (see Buck, 2000). Humans lack a functional VNO, but we may express some receptors in our olfactory epithelium very similar to receptors found in the VNO of other animals (Rodriguez et al., 2000). Further, members of a newly-discovered family of receptors respond to volatile amines, including stress-related amines in urine and amines that differ in concentration between males and females (Liberles and Buck, 2006). Genes for some of these receptors can be found in human olfactory epithelium. In short, between regular olfactory receptors and other receptors that may exist, humans have adequate capabilities for sensing pheromones. It is clear that humans respond to pheromones (Wysocki and Preti, 2004), although not with the automatic, stereotyped behavior of moths that fly miles upwind to mate. A magic love potion that irresistibly draws the opposite sex seems unlikely. However, pheromones may influence mood and physiology. Male underarm extract (odor masked) placed on the upper lip makes women ‘less tense’ and ‘more relaxed’ relative to the

Taste, smell and chemesthesis

103

masking vehicle alone (Preti et al., 2003). Underarm extract from a female donor can influence release of hormones to either accelerate or delay ovulation, depending on the stage of the donor (Stern and McClintock, 1998; Shinohara et al., 2001). This finding helps explain menstrual synchrony, i.e., the tendency for women who live together to fall into register over time. In addition, underarm extract from male donors can make women more regular and accelerates release of hormones associated with ovulation (Cutler et al., 1986; Preti et al., 2003). More research is needed, but data suggest that a properly formulated olfactory product could alter both our mood and our physiology. Processing olfactory receptor information into smell experiences Signals from the olfactory receptor neurons pass to the brain for further processing (Rolls, 2001; Lledo et al., 2005; Gottfried, 2006). First, signals are sharpened in specialized structures called the olfactory bulbs. Then, signals diverge into several pathways (Price, 1973). The direct cortical targets of the signals are situated in the primary olfactory cortex. Activity in parts of the primary olfactory cortex is modulated by sniffing, again reflecting its importance in olfactory processing (Sobel et al., 1998). Other work suggests that primary olfactory cortex plays a role in olfactory learning and memory, and may be involved in separating odors of interest from steady (background) odors (Gottfried et al., 2003; Kadohisa and Wilson, 2006). Interestingly, there are direct or nearly direct connections to other evolutionarily old parts of the brain, including the hypothalamus. The hypothalamus regulates hormonal release via the pituitary gland and plays a key role in the regulation of hunger, thirst, sexual behavior, and other basic functions, The hypothalamus may play a roll in the olfactory circuits that help regulate these functions, and may modulate sensory response based on both internal and external signals (Rolls, 2001). There are also direct connections to the amygdala, which responds to interactions between intensity and pleasantness (Anderson et al., 2003). Thus, similar to conclusions regarding taste processing (see p. 94), olfactory activity in the amygdala seems to play a role in representing stimulus salience. Interestingly, personally meaningful odors are described with strong emotional language and are associated with relatively strong activation in the amygdala (Herz et al., 2004a). Contrary to popular perception, studies suggest that odor evoked memories are no more accurate than those evoked by auditory, visual, or tactile cues, but they do entail stronger emotional reactions (Herz, 1997). Thus, people may confuse emotional vividness with accuracy. From the primary cortical areas, odor signals pass to higher brain areas, either directly or through the thalamus. The orbitofrontal cortex (OFC) is one important target (see Rolls, 2001; Gottfried, 2006). As with taste, the OFC combines input from multiple sensory modalities to make evaluative ‘judgments’. Also like taste, odors that people find pleasant evoke different patterns of activity in the OFC than do odors they find unpleasant (Anderson et al., 2003). Other studies suggest that some cells in the OFC respond according to the reward-value of a stimulus (reviewed in Rolls, 2001). In other words, a given cell might respond differently to a particular odor, depending on whether or not the odor was associated with a pleasant reward. This result suggests that evaluative judgments about odors are subject to learning. Other work has shown that, relative to a vanilla odor control, OFC activation to banana odor declines after subjects have eaten bananas to satiety (O’Doherty et al., 2000). This effect is not simple adaptation, since both neural response at lower levels of processing and rated intensity might change very little. Rather, this is a case of sensory specific satiety and reflects a change in the rewardvalue of the stimulus.

104

Product Experience

5. SMELL: BASIC PHENOMENA OF EXPERIENCE 5.1. Olfactory adaptation Continued exposure to the odor of a product decreases our ability to detect it, reduces its perceived intensity, and increases our reaction time to its presence (see Cometto-Muñiz and Cain, 1995; Lawless, 1997). A typical pattern consists of rapid loss of odor intensity within 5–10 seconds, and a more gradual decline thereafter, reaching a new steady state within a minute or two. However, the rate and degree to which odors adapt varies. Intense odors are more resistant to adaptation than weak ones (Cain, 1974). Further, unpleasant odors adapt more rapidly than pleasant ones (Jacob et al., 2003). The time course of adaptation could also depend on physical properties of the molecules, related to diffusion into the mucus and eventual clearance (Dalton, 2000). Adaptation is strongest for odors identical to the adapting odor (self-adaptation), but exposure to one odor may reduce the perceived experience of another odor to some lesser degree (cross-adaptation). One might expect similar smelling odors to stimulate overlapping sets of physiological mechanisms and thus cross-adapt effectively, and dissimilar odors to cross-adapt less effectively. Some data are consistent with this notion (e.g. Todrank et al., 1991; Cain and Polak, 1992). However, molecular structure, apart from the perceived quality, seems to play a roll. For example, 3-methyl-2-hexenoic acid (3M2H), a sweaty-smelling compound important in human underarm odor, can be crossadapted by the fruity-smelling ethyl ester of 3M2H (Pierce et al., 1995). Other research suggests it may even be possible to cross-adapt an odor using an odorless molecule with similar structure (Pierce et al., 1996). Wysocki et al. (2003) identified common fragrance ingredients that cross-adapt human underarm odor. This work showed that for males, many odors cross-adapt underarm odor, but far fewer cross-adapt underarm odor for females. Thus, properties of the observer, as well as properties of the molecule, can play a roll in efficacy of cross-adaptation. Adaptation probably occurs at various levels of the olfactory system (Cometto-Muñiz and Cain, 1995; Dalton, 2000). Receptor cells adapt, but neither the magnitude nor the duration of peripheral adaptation fully accounts for human olfactory responses (Zufall and Leinders-Zufall, 1997). Further, long-term reductions in sensitivity observed with repeated exposure on the job (Dalton et al., 2003) or in the home (Dalton and Wysocki, 1996) may have quite different underlying causes than short-term adaptation seen while sampling multiple perfumes at the department store. At some level, cognitive influences play a role. We may cease to pay attention to an odor that is always present, a general phenomenon many researchers call habituation (Lawless, 1997). Interestingly, if subjects believe an odor is safe and beneficial, rated intensity will wane more over time than if subjects believe the same odor is potentially harmful (Dalton, 1996). Thus, cognitive and emotional reactions to odors may modulate their experienced intensity (see Distel and Hudson, 2001, for a discussion of these influences on perceived odor intensity).

5.2. Combined olfactory experiences The odors of most consumer products come from mixtures of odorants. Products that counteract malodors often seek to mask offending smells by creating a mixture of the masker and the offending odor. Understanding the relationship between the experience of unmixed components and the experience of their mixtures will aid in product design, optimization, and quality control.

Taste, smell and chemesthesis

105

Most research on odor mixtures has been limited to two-component mixtures. The following general rules have emerged with respect to odor strength: (1) a mixture of two odorants will generally smell less intense than the sum of the intensities of the unmixed components, but stronger than the weaker of the two components; (2) the components, when they can be individually perceived, will smell weaker than when unmixed; and (3) the stronger of the two mixture-components will reduce the intensity of the weaker component more than the weaker will reduce the stronger (Laing et al., 1984; Cain et al., 1995; Lawless, 1997). Less work has been done on more complex mixtures, but some research suggests that beyond three or four components, adding additional odors has little impact on overall intensity (Moskowitz and Barbe, 1977). In addition, interactions depend to some extent on the overall intensity of the odors. Studies have found more complete summation when the unmixed components are relatively weak (Laing et al., 1984; Cain et al., 1995). With weak odors, even synergy has been observed (Engen, 1982; Laska et al., 1990). This general trend of concentration-dependence in mixture interactions also holds true at threshold levels (Cometto-Muñiz et al., 2005). Like taste mixtures, simple mixtures of odorants can be analyzed into their components. However, while taste qualities can be discerned in five to six component mixtures (Marshall et al., 2006), complex odor mixtures often form holistic sensations, and even expert perfumers fail to correctly identify all the constituents of a mixture that consists of more than about three to five chemicals (Livermore and Laing, 1996). This phenomenon is commonly assumed to support the fact that olfaction is a synthetic sense (Burgard and Kuznicki, 1990). According to most sensory work, the perceived qualities of mixtures generally fall between those of their unmixed components, with no radically new notes emerging, though some research leaves the possibility of emergent qualities open (Moskowitz and Barbe, 1977; Jinks and Laing, 2001; Zou and Buck, 2006). In addition, some odors tend to have an impact on the quality of mixtures disproportionate to their individual perceived intensities. For example, woody odors tend to dominate fruity ones, consistent with the laments of wine makers who find that tannins have a large impact on flavor (Atanasova et al., 2005). Unfortunately, science cannot precisely predict how a mixture will smell based on the properties of the molecules that comprise it. There is no substitute for the flavorist and perfumer, who create flavors and scents using a combination of personal experience and experimentation. However, systematic studies of how molecular properties influence mixture-interactions could eventually lead to perceptual models that greatly streamline the process of formulating novel flavors and aromas.

5.3. Modification of olfactory experiences Olfactory experiences can often be modified by taste. Von Sydow et al. (1974) showed that the experience of the intensity of the pleasant odors of fruit juice increases, while the experience of unpleasant odors decreases, in response to added sugar. McBride and Johnson (1987) found enhancement of lemon flavor experience with increasing levels of sucrose and citric acid in lemon juices, while Perng and McDaniel (1989) showed that sucrose enhanced the fruit flavor intensity of blackberry juice. However, in contrast to McBride and Johnson, the latter authors found that increasing the sourness of the juice lowered the experience of its fruit flavor. Other investigators have shown that aspartame enhances and prolongs the fruitiness of fruit-flavored solutions more than sucrose (e.g. Bonnans and Noble, 1993). A number of additional studies have also shown the strong interactions of taste and smell experience in foods (Frank et al., 1993; Stevenson et al., 1995, 1999). When odorant compounds are mixed with irritant compounds, mutual suppression occurs, much like with mixtures of odors. Cain and Murphy (1980) demonstrated this

106

Product Experience

effect by mixing the nasal irritant, carbon dioxide, with concentrations of amyl butyrate. The result was a mutual suppressive interaction, i.e. increasing concentrations of carbon dioxide resulted in decreasing judgments of the odor of amyl butyrate and vice versa. Moreover, by presenting the odorant compound to one nostril and the irritant compound to the other, it was possible to show that the locus of interaction occurred centrally in the nervous system (Cain and Murphy, 1980).

5.4. Innate versus learned odor experiences and preferences Some research has found that infants and young children have odor preferences that differ from those of adults, including a relative indifference to some odors that adults find offensive (Engen, 1982, 1988). These data suggest that odor preferences are learned over time. However, other research has found support for innate odor preferences. Among these latter studies are those showing that infants, like adults, display more negative facial responses to fishy and rotten egg odors than to the odors of vanilla, banana or butter (Steiner, 1977; see also Schmidt and Beauchamp, 1988) and that very young infants display preferences for certain biologically relevant odors, like the odor of breast milk (Porter et al., 1992). However, the fact that we begin learning odor preferences in the womb (Schaal et al., 1998; Winberg and Porter, 1998), e.g. infants of mothers who consume garlic are born with a preference for garlic odor (Mennella et al., 1995), makes it difficult to dissociate innate odor preferences from early learning. Regardless of whether there are innately preferred or rejected odors, learning and context certainly play important roles later in life. One line of evidence comes from intercultural differences in odor preferences (Schleidt et al., 1981; Wysocki and Gilbert, 1989; AyabeKanamuura et al., 1998). An odor that appeals to one culture does not necessarily appeal to another (see Section 9.3). Also consider a study sponsored by the US military that sought a universally offensive odor for non-lethal crowd control (Dilks et al., 1999). No single odor in the study was considered equally unpleasant across all ethnic groups. Another line of evidence comes from studies of odors paired with either positive or negative emotional experiences (Baeyens and Wrzesniewski, 1996; Herz et al., 2004b). The results suggest that, through associative learning, the experience of an odor takes on the emotional tone of the context in which the odor was originally experienced. As suggested elsewhere in this chapter, beliefs and expectations play a critical role. In one study, panelists rated a given odor as less pleasant if they believed it consisted of synthetic rather than natural materials (Herz, 2003), while in another, people expressed more liking for a mixture of fatty acids when the stimulus was labeled ‘parmesan cheese’ rather than ‘vomit’ (Herz and von Clef, 2001). Perfumers and flavorists know the benefits of selecting odors that have positive associations in the culture in which the product will be marketed. The above research suggests that the consumer product industry might actively influence the acceptability of new scents through proper advertising, packaging, and labeling that creates associations with positive images and concepts. Retailers can further help by ensuring that consumers experience new odors in an emotionally positive context. Such techniques might prove particularly effective in influencing the pleasantness of newly developed odors and scents.

6. CHEMESTHESIS 6.1. The experience of chemesthesis We feel chemicals when they stimulate nerve endings in our skin (see Doty and ComettoMuñiz, 2003; Wysocki and Wise, 2003). Chemical feel is called ‘chemesthesis,’ a

Taste, smell and chemesthesis

107

contraction of ‘chemical somesthesis’. When chemicals reach nerve endings in sufficient concentration, we experience sensations such as the ‘cooling’ of menthol, the ‘bite’ of carbonation, the ‘burn’ of distilled spirits, the ‘heat’ of chili peppers, and the ‘sting’ of smelling salts. Chemesthesis warns us against high chemical concentrations, and helps protect tissue through reactions such as tearing, increased mucus production, sneezing, and coughing. However, mild to moderate chemesthesis plays an important role in our everyday experience of foods, beverages, and many personal products.

6.2. Peripheral anatomy and receptor mechanisms for experiencing chemesthesis Mucous membranes, like those that cover the eyes, nose, and mouth, are more sensitive to chemicals because they offer easier access to nerve endings than the keratinized skin on the rest of the body. As with taste and smell, some irritants stimulate specific receptor proteins in nerve endings (Caterina and Julius, 2001; Peier et al., 2002; McKemy, 2005). For example, one protein responds to capsaicin, the active principle in hot peppers, whereas another responds to cooling compounds like menthol. Interestingly, these and other chemically sensitive receptors also respond to temperature changes, so the terms ‘hot pepper’ and ‘cool mint’ make perfect sense. In addition, some compounds may act through non-specific effects on cell membranes rather than on specific receptor proteins. The action of carbon dioxide (CO2), the gas that gives bite to carbonated beverages, depends on enzyme activity (see Tarun et al., 2003). CO2 diffuses into tissue, where enzymes facilitate formation of carbonic acid (the effective stimulus). Recent work has found individual differences in the enzyme genes expressed in the nose (Tarun et al., 2003) and sensory receptors for nicotine may also vary among individuals (Keiger et al. 2003). Furthermore, PROP tasters and PROP non-tasters (p. 94) differ in sensitivity to oral irritation, probably due to differences in nerve density in the tongue (Prescott and SwainCampbell, 2000; Duffy et al., 2004). As with smell and taste, we may eventually predict some chemesthetic product experiences and preferences based on gene make-up.

6.3. Processing peripheral nerve signals into chemesthetic experiences Nerve fibers that carry somatosensory information project first to the brain stem (for signals from the eyes, nose, and mouth) (Hendry et al., 1999). From there, signals pass to the thalamus, then to the primary somatosensory cortex, then to higher cortical areas. Data on central processing of chemesthesis are sparse. An imaging study using an irritating concentration of acetone found activation in the brainstem and somatosensory cortex (Savic et al., 2002). In addition, research measuring electrical potentials has found differences in patterns of activity across the scalp between pure odors and irritants (reviewed in Shusterman, 2002). However, little research bears on quality coding past the periphery.

6.4. Chemesthesis: Basic phenomena At both supra-threshold (Cometto-Muñiz and Hernádez, 1990) and threshold levels (Cometto-Muñiz et al., 2004), there is greater cooperation (better additivity) among mixture-components for nasal chemesthesis than for smell. Better cooperation might indicate that detection-mechanisms for chemesthesis are more broadly tuned (Cometto-Muñiz et al., 2004). Practically speaking, good cooperation means a mixture might irritate, even if none of the individual compounds irritate, having implications for the experience of multi-odorous indoor air and odor-containing personal products used in indoor spaces.

108

Product Experience

In terms of compounds that have both chemesthetic and olfactory components, Figure 4.4 shows data from an experiment in which ammonia was presented to subjects in two concentrations and for two durations. As can be seen, the perceived irritation (and thus, the total intensity) increased more dramatically as a function of both concentration and stimulus duration than did the perceived odor. Thus, when one inhales ammonia, its odor reaches a peak rapidly while irritation continues to grow over time, perhaps even suppressing its odor. Although strong chemesthesis is innately aversive, many individuals develop a preference for mild to moderate irritation with repeated exposure (see Rozin and Schiller, 1980; Stevenson and Yeomans, 1993). Also, people in warmer climates tend to consume more hot spices. Since compounds in many spice plants have anti-microbial activity, societies in which food spoilage poses greater problems might develop a taste for spices, in part, for practical reasons. Further, as with odor, emotional reactions will depend on beliefs about the stimulus. A pleasantly pungent fragrance will be received quite differently if subjects believe the aroma comes from a potentially dangerous chemical (see Dalton and Wysocki, 1996). Relative to taste and smell, the dynamics of chemesthesis are slow (Cain et al., 1986; Green, 1990; Prescott and Swain-Campbell, 2000). Depending on the chemical, concentration, method of presentation, and location of stimulation (mouth, nose, or eyes), irritation may build over many minutes and fade slowly once a peak is reached. Build-up, seen with steady presentation or with a sufficiently brief inter-stimulus interval (ISI), is often referred to as sensitization (Green, 1990; Prescott and Swain-Campbell, 2000). The waning of irritation, seen after some time with steady presentation or with a sufficiently long ISI, is often called desensitization (Green, 1990; Prescott and Swain-Campbell, 2000; Jacquot et al., 2005). However, the mechanisms underlying these phenomena are not completely understood.

FIGURE 4.4 Perceived odor, pungency (irritation) and total intensity for two concentrations of ammonia inhaled for two different durations (from Cometto-Muniz and Cain, 1984).

Taste, smell and chemesthesis

109

7. MEASURING CHEMOSENSORY PRODUCT EXPERIENCE 7.1. Chemosensory experiences: Consumers versus experts Quantifying the taste, smell and chemesthetic experiences produced by products is critical to many product development efforts. In many commodity areas, e.g. wine, tea, coffee, dairy products, and perfumes, expert ‘palates’ or ‘noses’ are used to characterize sensory properties, to create new formulations, and to delineate what constitutes a ‘good’ or ‘bad’ product. However, by virtue of their extensive experience and their acquired abilities to distinguish subtle product differences, the product experiences of experts are not the same as those of consumers. A reason commonly cited by product experts for not using consumers to judge product quality is that consumers differ from one another in their experiences and preferences. Yet, while this is true, it has been shown that the expert’s experience of wine and beer quality differs significantly from one expert to another (Langron et al., 1983; Guinard et al., 1999), and that even the vocabulary they use to describe these products differs dramatically among them (Sauvageot et al., 2006). In the perfume industry, ‘experts’ also disagree. Witness a leading perfumist’s review of Dark Chocolate perfume by Coty: ‘The name suggests a beguilingly sinful, high calorie confection. Instead, the fragrance has all the charm and mystery of a neon-lit shopping mall, and smells of cheap chocolates with a fruit heart. A case of pastel clad obesity’ (Turin, 2006). The notion that experts have special sensory abilities that enable them to detect product differences that consumers cannot is also often overstated. For example, it has been shown that while expert tea tasters can discriminate the experience produced from tasting tea in which the milk has been poured into the tea versus the tea having been poured into the milk, it has been shown that consumers have the same ability (Powers, 1988). Of the many purported abilities of experts, the one that most differentiates expert product experiences from those of consumers is their ability to identify and characterize differences in taste and olfactory product experiences using a well-defined vocabulary. This ability to provide a detailed description of product attributes, while beyond the capacity of most consumers, can often be taught using standard descriptive sensory methods, as discussed below.

7.2. Methods for quantifying chemosensory product experiences: Trained panels Although the use of experts is still common in some specialized product areas, most commercial products are now characterized by trained ‘descriptive’ panels. Descriptive panels are groups of individuals, screened to ensure lack of defects in taste or smell. These individuals, usually 10–20 in number, are trained in one of a variety of descriptive sensory methods that enable them to articulate and describe their product experiences using standard terminology and operational definitions. This process produces a product profile that identifies all of the taste, smell, and chemesthetic attributes of the product and the intensities of each. There are a wide variety of methods for conducting descriptive analysis. The oldest is the flavor profile method developed by Arthur D. Little Co. (Cairncross and Sjostrom, 1950). This method utilized a panel of six to eight individuals who received training in taste and smell physiology, sensory testing, and flavor vocabulary. Physical reference standards for specific taste and smell attributes, e.g. ‘salty’, ‘rancid’ or ‘piney’ were employed to be used during training and product evaluation. A panel leader was chosen

110

Product Experience

to lead panel discussions and to arrive at a consensus on the experienced aroma, taste, flavor, mouthfeel, and aftertaste attributes of the product, the order in which they were perceived, their intensity, and the ‘amplitude’ or overall quality of the product. Although the A. D. Little Flavor Profile method was the first popular descriptive method, it has largely been replaced by more contemporary methods that have improved upon various aspects of the methodology. Among these more contemporary approaches is Quantitative Descriptive Analysis (QDA) (Stone et al., 1974), a method that replaced consensus agreement with the use of quantitative, statistical analysis of panel data. The perceived intensity ratings were improved in QDA by using a linear graphic rating scale, resulting in a method with reportedly good quantitative reliability (Stone and Sidel, 2004). This technique also spurred the use of ‘spider web’ charts for displaying product attributes. Figure 4.5 shows a spider web chart of descriptive data for two personal care products (bar soap). The lengths of the spines reflect the maximum intensity for each attribute and the mean product ratings for that attribute are plotted along the spine. By overlaying the data for two or more products on a single spider web (as in Figure 4.5), the differences in the sensory profiles of the products are easily visualized. A third technique, known as Spectrum Descriptive Analysis, has also been popularized (Meilgaard et al., 1999). This latter technique adapts approaches from earlier methods, while utilizing ‘universal’ rating scales with physical reference anchors in order to quantify the product experiences of panelists. While each of these methods differ to some degree, comparisons of the methods for the same products typically produce similar results.

7.3. Methods for quantifying chemosensory product experiences: Consumers Quantifying sensory experience After training and extensive panel work, it can be argued that the product experiences of descriptive panelists are also unlike those of the average user. In an effort to avoid the use of either experts or trained panelists to describe taste and olfactory product

FIGURE 4.5 A typical spider web plot of descriptive data for two bar soaps (from Lawless and Heymann, 1998).

Taste, smell and chemesthesis

111

experience, the method known as ‘free-choice profiling’ was developed (Williams and Langron, 1984; Steenkamp and van Trijp, 1988). This approach relies on naive consumers who use their own vocabulary to describe their experience of the product or who select terminology from a pre-prepared list. Terminology may also be generated using the repertory grid method (e.g. Jaeger et al., 2005). The data are then analyzed using multivariate techniques in order to develop a user-based perceptual space of products, the dimensions of which can be defined in terms of the actual experiential vocabulary that consumers use. Other consumer-friendly techniques have also been developed. Free-sorting procedures, in which products are sorted into groups and a common vocabulary is used to describe the groups, have been used to analyze the perceptual product space for tastants (Yang and Lawless, 2005), odorants (Rawcliffe et al., 1992), and a variety of foods and beverages (Lawless et al., 1995; Popper and Heymann, 1996; Tang and Heymann, 2002). In a recent study comparing sorting by consumers and QDA trained panelists, the perceptual product space for the two groups was found to be quite similar. Moreover, the sorting procedure for trained panelists provided similar results to traditional QDA methods (Cartier et al., 2006). The authors concluded that ‘sorting combined with verbalization led to meaningful and consistent product sensory mapping, whatever the panelists’ level of training’ (Cartier et al., 2006, p. 562). Similar results also have been reported by Faye et al. (2006) for non-food products. Quantifying affective experience As noted in other sections, most chemosensory products produce a strong affective or ‘hedonic’ experience in the user. When an individual smells a new cologne or tastes a new chewing gum, there is an emotional experience of its pleasantness or unpleasantness. These hedonic experiences of products are the primary drivers of the consumer’s choice, purchase, consumption, and/or utilization of the product. The first studies of affective experience were undertaken by Beebe-Center in the 1930s, but the systematic study of food hedonics (from the Greek word for ‘pleasure’) and its measurement began after World War II when the US Army Quartermaster Food and Container Institute in Chicago began to study why soldiers liked and/or disliked certain foods. This research program ultimately produced the now well known, ‘ninepoint hedonic scale’ (Peryam and Pilgrim, 1957) (see left side of Figure 4.6). This simple method for quantifying the intensity of a person’s affective experience is now the most commonly used affective measure in the food, cosmetics, perfumery, and personal care products industries. In spite of its simplicity, the nine-point hedonic scale does have problems. Since it is a form of ‘category’ scale there is a tendency for people to avoid the extreme categories (Stevens and Galanter, 1957). In addition, the intervals between scale points are not perceptually equal (Jones and Thurstone, 1955; Moskowitz and Sidel, 1971), so the level of measurement it provides is, at best, ordered metric. Other methods that have been used over the years include magnitude estimation, linear graphic rating scales and, most recently, a Labeled Affective Magnitude (LAM) scale (Schutz and Cardello, 2001; Cardello and Schutz, 2004). The latter scale was developed to provide a simple-to-use, ratio-like scale of product liking experience that is modeled after other contemporary labeled magnitude scales (Green et al., 1993; Bartoshuk, 2000). As shown on the right side of Figure 4.6, this scale uses a vertical line with the end points labeled ‘greatest imaginable liking’ and ‘greatest imaginable disliking’. Between these end-point anchors the nine labels used in the nine-point hedonic scale are distributed along the liking/disliking dimension according to the perceived ratios in their semantic meaning, as determined

112

Product Experience

Scales for rating product liking experience Nine-point Hedonic Scalea

Labeled Affective Magnitude Scaleb,c Greatest imaginable liking

Like extremely Like very much Like moderately Like slightly Neither like nor dislike Dislike slightly Dislike moderately Dislike very much Dislike extremely

Like extremely Like very much Like moderately Like slightly Neither like nor dislike Dislike slightly Dislike moderately Dislike very much Dislike extremely Greatest imaginable disliking

a

Peryam and Pilgrim (1957)

bSchutz

and Cardello (2001) and Schutz (2004)

cCardello

FIGURE 4.6 Scales for rating product liking/disliking (affective) experiences.

using a ratio scaling procedure. Using this scale it is possible to estimate whether the liking experience for a particular product is twice, one-third, etc. as large as the liking experience for a different product (Schutz and Cardello, 2001). In addition, the scale has been shown to produce better sensitivity, equal reliability, and equal ease of use to the nine-point hedonic scale by consumers (Schutz and Cardello, 2001).

8. CONTEXT, INFORMATION AND EXPECTATIONS IN CHEMOSENSORY AND PRODUCT EXPERIENCE 8.1. Non-sensory influences on product experience The taste and olfactory experiences that arise from the intrinsic properties of a product do not constitute the sole determinants of product experience. There are a host of other ‘extrinsic’ factors, e.g. product names, packaging, labeling, pricing, context and promotional information that influence product experience through cognitive and psychological processes. In this section we will explore the effect of some of these factors.

8.2. The effects of context Both the sequential and situational contexts in which a product is experienced can significantly alter our experiences of it. Sequential effects are commonplace. When you are shopping for fragrances in a department store, you might first smell a perfume on one wrist and then its eau de toilette on your other wrist. You would likely conclude that the eau de toilette was quite weak. However, if you tried them in the reverse order, you would

Taste, smell and chemesthesis

113

likely feel differently about the strength of the eau de toilette, even if you waited a relatively long time between trials to ensure that no adaptation to the scent was occurring. In fact, in a recent study of sequential context effects, the perceived intensity of the odor of perfumery grade California Orange Oil was affected by presentation of either a weaker or stronger sample as much as 25 minutes before the target odorant was presented (Pol et al., 1998). A number of other studies using tastants and odorants have shown that the intensity of a target product presented within a context of products of either higher or lower taste/smell intensity or liking will be ‘contrasted’ with the intensity of the contextual products (Lawless, 1983; McBride, 1985; Schifferstein, 1995). These results are commonly explained in terms of a ‘perceptual’ adaptation level (Helson, 1964) or in terms of range-frequency theory (Parducci, 1974). Recently, Schifferstein has addressed this phenomenon in terms of the nature and characteristics of the contextual stimuli and their impact on the magnitude of these contrast effects (Schifferstein, 1995; Schifferstein and Frijters, 1992; Schifferstein and Oudejans, 1996). His results support the notion that the more similar the contextual and target stimuli, the larger will be the contrast effect. In addition to sequential context effects, the situational and environmental context in which products are encountered can alter our experiences of them. For example, the product experience that results from eating ice cream on a hot summer day is quite different from eating that same ice cream on a cold, winter day. Likewise, the experience of encountering a fishy odor in the hallway of your home is quite different from the experience of encountering that same odor in a seafood market. In the world of taste and smell, context can make all the difference. The number of different environmental, situational, and other contextual factors that can alter product experiences is quite large. The reader is referred to articles and book chapters by Rozin and Tuorila, 1993; Cardello, 1994, 2007; Deliza and MacFie, 1996; Schifferstein, 2001; Wansink et al., 2005; Jaeger, 2006 and Chapter 24 by Meiselman in this volume for discussions of the contextual influences on product experience. Perhaps the most common contextual or ‘framing’ influences in the commercial products industry are those involving product information. Whether written, verbal, or non-verbal, information influences consumer experience and behavior by altering the attitudes and beliefs that the consumer holds toward the product. In turn, these attitudes and beliefs can give rise to product expectations, i.e. psychological anticipations that the product will possess certain attributes or have certain anticipated effects. Expectations can arise from previous experiences with the product, with other products in the category, or from any one of a variety of information sources about the product. Two major types of expectations are those that relate to expected sensory experiences, e.g. a belief that a skin cream will have a floral scent, and those that relate to expected affective experiences, e.g. a belief that you will like the smell of a new automobile’s leather interior.

8.3. The effects of information and expectations Effects on olfactory product experiences Olfactory experiences seem to be especially prone to the influence of expectations. As early as 1899, Slosson performed a classroom demonstration in which he told students that an odorant was to be dispersed into the back of the classroom and that he wanted to see how long it took for the odor to reach the front of the room. He asked his students to raise their hand when they smelled the odor. Although no odor was actually dispersed, a majority of students reported smelling the odor, some even becoming upset by its unpleasant quality. Much more recently, O’Mahony (1978) performed a similar but more dramatic experiment. Using a British television broadcast of a documentary on

114

Product Experience

the chemical senses, viewers were informed that smells could be produced by transmitting sound waves of the same vibrational frequency as the frequency of vibration of the molecules of the odorant. The documentary then went on to ‘demonstrate’ the phenomenon by ‘broadcasting’ a specific frequency of sound alleged to correspond to a pleasant country odor. Viewers were asked to phone or write the station if and when they smelled the odor. Of 130 responses received, 114 viewers said that they experienced a smell when the sound was emitted. The most common smell experiences reported were ‘hay’ or ‘grass’. Using more traditional paradigms, Engen (1972) performed experiments in which an odorant was presented to subjects at regular intervals prior to the presentation of a ‘blank odorant’ in order to establish an expectation for the odor. The mere expectation that the odorant was to be presented resulted in an odor experience. In another experiment, he flowed odorants through a tube that was visible to the subjects. The odorants were either colored yellow or left transparent. A much higher rate of odor experience resulted on trials in which the odorant was colored yellow than when it was transparent. The color appeared to have induced an expectation that an odor was being presented and this expectation led to a greater likelihood of the olfactory experience. Dalton (1996) exposed subjects to the odorant ‘isobornyl acetate’ under three different information conditions. One information group was told that they would be exposed to a natural extract of balsam trees. A second group was told they would be exposed to an industrial solvent that could produce health and cognitive problems after longterm use. A control group was simply told that they would be exposed to a standard odorant approved for smell research. The subjects were exposed to the odorant over the course of 20 minutes and were asked to judge its intensity every minute. The results of this study are depicted in Figure 4.7. Although the positive information group (balsam trees) showed a typical adaptation curve to the odorant over time, the negative bias group (industrial solvent) first began to adapt, but then their experience of the odor began to increase in intensity, reaching strong levels by the end of the test. The neutral

50

Very strong Positive bias Negative bias Neutral bias

Perceived intensity

40

Strong 30

20 Moderate 10 Weak Barely detectable 0

5

10

15

20

Exposure duration (minutes)

FIGURE 4.7 Perceived odor intensity of isobornyl acetate for subjects in the positive, negative, and neutral bias conditions during a 20-minute exposure (from Dalton, 1996).

Taste, smell and chemesthesis

115

(control) group produced an intermediate level of adaptation. This study underscores the relevance of survival mechanisms and how our olfactory experiences can be shaped by perceived dangers. Another study showing the importance of information about the source of experienced odors was conducted by Distel and Hudson (2001). In this study odors of everyday products were experienced as more pleasant and more intense when subjects were told the name of the product than when the source of the odor was left unstated. Cognitive elements, perhaps related to the reduction in uncertainty about the odors, appear to have induced changes in the olfactory experiences of them. In a study on perfumes, Scharf and Volkmer (2000) examined the olfactory expectations produced by brand information. They tested two different perfumes at two different concentrations. Consumers first evaluated the fragrances in a blind test for their olfactory experiences of ‘intensity’, ‘sweetness’, ‘freshness’ and ‘floweriness’. In a second session, consumers evaluated images of four different package designs for the perfumes that differed in color (dark red vs. pastel green) and brand (‘Christian Dior elegance’ vs. ‘TIP woman’). Each design was evaluated for the olfactory experiences that would be expected of the product using the same attributes used in the blind test. Lastly, consumers rated combinations of the four fragrances and the four package images on the attributes. Results showed that the olfactory expectations created by the packages influenced the actual olfactory experiences of the perfumes. In some cases, the olfactory experience was altered to be more consistent with the expectation that was created by the packaging. In other cases, the experience was altered to become less like that which was expected. The former finding is commonly referred to as ‘assimilation’, i.e. the experience assimilates (incorporates) the expectation of it. The latter finding is referred to as ‘contrast’. As we shall see in our discussion of taste and liking expectations, ‘assimilation’ and ‘contrast’ are but two of several possible outcomes, yet they seem to form the basis of what appears to be an evolving cognitive model to explain how expectations can influence our product experiences. Effects on taste product experiences One early study of the effects of information on taste experience was conducted by Carlsmith and Aronson (1963). These researchers presented a series of sweet or bitter solutions to consumers who evaluated them for their intensity. Before presenting each, they gave a signal to indicate that either a bitter or sweet stimulus would be presented. In some cases the signal accurately cued the taste solution being presented, but in other cases the cue was misleading, e.g. a signal for a bitter solution was given, but a sweet solution was presented. Their results showed that sweet solutions that disconfirmed a taste expectation were rated less intense than sweet solutions that confirmed a taste expectancy. However, bitter solutions that disconfirmed an expectation were rated more intense. In both cases, disconfirmed expectations resulted in a more negative experience (less sweet or more bitter). They concluded that the data supported Cognitive Dissonance Theory (Festinger, 1957), a psychological theory that proposes that when cognitions (beliefs, values, attitudes, perceptions) or behaviors are inconsistent with one another, a negative psychological state of ‘dissonance’ occurs that makes the individual evaluate the stimulus more negatively. Olson and Dover (1976) performed a similar study on the taste of coffee. These researchers assigned consumers to an information or control condition. In the information condition, consumers were given written information that the coffee had ‘no bitterness’. After reading the information, consumers rated the strength of their belief that the coffee would be either ‘very bitter’, ‘fairly bitter’, ‘somewhat bitter’, or ‘not at all bitter’. Several days later, they returned to taste and evaluate the actual coffee using the same procedure to assess their perceptions of its bitterness. The consumers in the

116

Product Experience

Effect of expectations on product experience (coffee bitterness) 8 7

Control group E

Expectation group

6

Expectation level (E)

5 4 3 2

E

1 E

E

0 Not at all bitter Somewhat bitter Fairly bitter

Very bitter

Bitterness cateogries

FIGURE 4.8 The effects of product expectations on product experience (plot of the data from Olson and Dover, 1976). The bars indicate the mean ratings of the reported ‘belief strength’ of the bitterness of coffee. The arrows indicate the level of pre-trial expectation in the information condition.

control condition merely came to the taste test session and evaluated the coffee. Figure 4.8 shows the results of the study. The arrows indicate the expected belief strength of the information group before tasting the coffee. As can be seen, when compared to the control group, the information group’s experience of the bitterness of the tasted coffee moved in the direction of their expectations of it, supporting an ‘assimilation’ of the expectation into the product experience. Cardello and Sawyer (1992) conducted a study in which consumers were given one of four different types of information about pomegranate juice, i.e. that the juice was ‘very bitter’, had ‘average bitterness’, was ‘not bitter at all’, or no information. The consumers rated their expected experience of the bitterness and other sensory and hedonic attributes of the juice prior to tasting it. The results showed that the information established different expectations for the bitterness of the juice that were consistent with the information. However, the ratings of the bitterness experienced in the tasted product showed both assimilation and contrast effects, similar to the findings of Scharf and Volkmer (2000). Lastly, in a recent study examining the role of information about the area of origin of olive oils (Carporale et al., 2006), users of Italian olive oils were led to believe that a particular oil had been produced in one of three different olive oil producing regions of Italy. Expectations of the bitterness and pungency of the oil were found to differ depending upon the regional information provided about the oil. Moreover, these expectations were assimilated into the ratings of the actual experience of the bitterness and pungency of the oil. Effects on affective product experiences Over the years, a number of studies have also examined the effects of product information on expectations of the liking for foods. Among these have been studies that have examined the effects of nutritional or health content labeling of foods (Tuorila et al., 1994, 1998; Kahkonen and Tuorila, 1998; Kahkonen, et al., 1999; Wansink and Park, 2002). In these studies, product labels such as ‘low-fat’ or ‘contains soy’ lowered user expectations for both the taste and other desirable sensory attributes of the product and, in many cases, resulted in an assimilation of the lowered expectation into the actual liking

Taste, smell and chemesthesis

117

experience (Tuorila et al., 1994; Kahkonen et al., 1996, 1999). Other studies have examined the effects of information about the geographic origins of the product (Stefani et al., 2006; Caporale et al., 2006) or the methods of processing and preserving them (Siret and Issanchou, 2000; Cardello, 2003; Caporale and Monteleone, 2004; Iaccarino et al., 2006). In the majority of these studies product liking has been shown to assimilate the expectation. However, as with sensory expectations, contrast effects have also been observed. Researchers in the area of consumer expectations are now shifting to a hybrid ‘assimilation-contrast model’ to account for the disparate data, with factors related to the magnitude of the discrepancy between expectations and actual product features and the individual’s confidence in his/her sensory abilities determining whether assimilation or contrast effects will occur. The reader is referred to Cardello (2007) for a recent review of this general topic area and for a more detailed discussion of predictive models of the effects of expectations on product liking experiences. Brain mechanisms, expectations and product experience In recent years progress has been made in understanding the neural bases of cognitive effects on taste, smell and other sensory experiences. Much of this progress has been made through the use of fMRI technology that measures blood flow in various regions of the brain. McClure et al. (2004) measured fMRI brain activity in response to the taste of cola beverages (Coke and Pepsi). When a simple visual stimulus of a Coke can (versus a circle of light) was presented before the test stimulus, increased brain activity was observed in several brain areas. The authors concluded that these latter brain areas are involved in the process of ‘biasing perception based on prior affective behavior’. In a more recent study, remarkably similar to the early expectation studies on bitter taste, fMRI was used to investigate the neural mechanisms by which taste expectations can attenuate actual taste perception (Sarinopolous et al., 2006). These investigators provided misleading information to subjects that a taste solution would be mildly bitter. This false information produced neural activity in specific areas of the cortex, including the orbitofrontal cortex (OFC) that predicted decreases in other brain activity in response to the presentation of a very bitter taste solution. The authors concluded that the (neural) network processing aversive taste was suppressed by ‘expectancy-related brain activity’ in specific regions of the cortex, including the OFC. Since the OFC encodes the ‘reward value’ of olfactory and taste stimuli (Rolls, 2000; Hornack et al., 2004), it appears that mere expectations about products activate reward areas in the brain, enabling cognitive-based stimuli to alter, at a neural level, the hedonic experience of the products we taste, eat, smell and drink.

9. AGE, GENDER, CULTURAL AND SOCIAL FACTORS IN CHEMOSENSORY AND PRODUCT EXPERIENCE 9.1. The effects of age As life expectancy rises, the elderly comprise an increasingly larger group of consumers. With advanced age comes declines in sensitivity to taste, smell and chemesthesis (Murphy 1993; Hummel et al., 2003; Boyce and Shone, 2006) and concomitant changes in our experiences of products. Aging takes the largest toll on olfactory experience, with substantial losses in sensitivity to a wide range of odorants. Figure 4.9 shows data from a study by Murphy and Cain (in Murphy, 1986) showing the percent correct identification for a battery of 80 odorants as a function of age. As age increases, there is a steady and progressive decline in olfactory identification. In terms of sensory thresholds, some

118

Product Experience

90

80

Percent correct

70

60

50

40

30

20

10

r ⫽ ⫺0.75

20

30

40

50

60

70

Age (yrs)

FIGURE 4.9 Percent correct identification for a battery of 80 odorants as a function of age (from Murphy and Cain [unpublished] in Murphy, 1986).

evidence suggests that the minimum concentration that subjects can smell increases by two orders of magnitude between the 2nd and 8th decades of life (Wysocki et al., 2003). Declines in taste sensitivity are more modest than those for smell, with age-associated increases in thresholds ranging from 1.3- to 9-fold, depending upon the taste compound studied (Stevens and Cain, 1993; Mojet et al., 2001). Results vary, but in several studies sensitivity to bitter and salty stimuli tends to decrease more than sensitivity to other tastes (reviewed in Mojet et al., 2001). Figure 4.10 from Mojet et al. (2003) shows data on age and gender differences for the five basic tastes. Each taste quality is represented by two taste compounds. Age effects are largest for umami taste, with other differences varying by both taste compound and gender. What is clear, though, is that in all cases, taste responsiveness declines with age. For chemesthesis, declines in sensitivity are also modest, with an age-associated increase in threshold of less than 10-fold for one study (Wysocki et al., 2003), and no significant effect of age in another study (Stuck et al., 2006). Declining chemosensory function poses risks. Declining taste sensitivity could increase the risk of consuming bitter food contaminants and of chronically over-consuming sodium (Stevens and Cain, 1993). Declining olfactory function also might render product warning odors less effective. In one study, 45% of elderly subjects failed to detect the warning odor in natural gas at standard levels (Murphy et al., 2002). Another study found that, relative to normal controls, those with olfactory dysfunction were more likely to miss fires and gas leaks, and more likely to ingest spoiled foods or toxic substances (Santos et al., 2004). These results suggest that concentrations of warning odors in products could require adjustment for elderly consumers. In addition, reduced flavor perception may lead to reduced food intake, reduced dietary variety, and less healthful food choices (Murphy, 1993; Duffy et al., 1995; Rolls 1999). Research suggests that appropriate flavor enhancement can improve consumption and nutritional status among the elderly (Schiffman, 2000; Mathey et al., 2001), but not all manipulations of this kind are effective

Taste, smell and chemesthesis

119

FIGURE 4.10 Perceived intensity of ten taste compounds in water for young (solid lines) and elderly (dotted lines) males (triangles) and females (circles) (from Mojet et al., 2003).

(Koskinen et al., 2003; Essed et al., 2006). Thus, formulation of optimal taste and smell product experiences to improve consumption, nutrition, and safety for the elderly represents both an opportunity and a challenge for the consumer products industry.

9.2. The effects of gender When gender differences occur in simple laboratory tests of sensitivity to taste, smell, and chemesthesis, the differences tend to favor women (Dalton, 2000; Shusterman, 2002; Mojet et al., 2001, 2003; Stuck et al., 2006). However, differences in taste and smell experiences are usually small and often fail to reach statistical significance (Dalton, 2000; Mojet et al., 2001, 2003) (see Figure 4.10 as an example). Differences are somewhat more consistent for chemesthetic experience, but are still moderate and still often fail to

120

Product Experience

reach significance (Wise et al., 2003; Wysocki et al., 2003; Hummel et al., 2003; Stuck et al., 2006). Some evidence suggests that gender differences may increase with age (e.g. Wysocki and Gilbert, 1989; Mojet et al., 2001), but on the whole, individual differences within the sexes are much larger than average differences between them. Although simple tests of sensitivity reveal relatively little difference between men and women, some interesting higher-order effects occur, particularly in olfaction. For example, with repeated testing, women can become progressively more sensitive to some odors (Dalton et al., 2002). Over the course of 30 sessions, sensitivity can increase by several orders of magnitude. However, beyond an initial practice period, men fail to show this sensitization. Interestingly, neither pre-pubescent girls nor post-menopausal women show this sensitization, which suggests that the effect may depend on hormonal function. Fluctuations across the menstrual cycle also suggest that hormonal function might influence olfactory sensitivity in women, though the exact role that hormones play remains unclear (Pause et al., 1996; Navarrete-Palacios et al., 2003). Other studies suggest that women are more likely than men to develop intolerance to certain environmental odors (Kreutzer, 1999; Caress and Steinman, 2004; Johansson et al., 2005). Yet another study found that a pleasant odor could reduce rated pain intensity in women, but not in men (Marchand and Arsenault, 2002). Sex differences in emotional responses to odors appear to play a role here. Regardless, it is clear that chemosensory experiences do differ between men and women. Although it may not always make sense to create gender-specific product formulations, gender differences are worth considering in product development and evaluation.

9.3. The effects of cultural and social factors Cultural factors are, perhaps, the most powerful of factors that can control product experience. In fact, one set of authors has concluded ‘If one were interested in determining as much as possible about an adult’s food preferences and could only ask one question, the question should undoubtedly be: “What is your culture or ethnic group?” ’ (Rozin and Vollmecke, 1986). Insofar as odors are concerned, there are large intercultural differences in odor preferences (Schleidt et al., 1981; Wysocki and Gilbert, 1989; Ayabe-Kanamuura et al., 1998). For example, wintergreen is experienced as a ‘candy’ odor in the US (Cain and Johnson, 1978), but as a ‘medicine’ odor in the UK (Moncrieff, 1966). Also the study cited in Section 5.4 that sought a universally offensive odor for non-lethal crowd control (Dilks et al., 1999) found no single odor to be unpleasant across all ethnic groups. In a recent study on the pleasantness of ‘Japanese’, ‘international’, and ‘European’ odors as experienced by Japanese and German consumers, ‘Japanese’ odors were perceived as much less pleasant by the German consumers than by the Japanese consumers. The ‘international’ odors were generally perceived as pleasant by both groups, and the ‘European’ odors elicited mixed responses (Ayabe-Kanamura et al., 1998). For taste, ethnic, and cultural differences abound. Kimchi is the national food of Korea and is eaten ubiquitously there. However, its strong salty/sour taste is not well accepted in most European and North American countries. Nor are the super-tart umeboshi pickled plums that are eaten in Japan. These and other exotic taste and olfactory experiences that one encounters in Asian, Indian and other ethnic markets are product experiences that few Americans or Europeans fully appreciate. Thus, it appears that the food habits and olfactory experiences of different ethnic groups can influence future sensory and affective preferences for chemosensory stimuli and products. Even within the same country, differences in product experiences are commonplace. Take, for example, the preference for brown eggs over white eggs in the Northeastern

Taste, smell and chemesthesis

121

United States and the preference for ‘grits’ (coarsely ground, hulled grain) in the Southern US, preferences not seen elsewhere in the U.S. Rozin (1996) has argued that some product avoidances, e.g. those for bean curd or pork kidneys among Midwesterners in the US, are due to people not viewing them as food. Thus, cultural factors can even define what items are considered to be foods and which are not. Religious and moral taboos provide the most dramatic examples of cultural effects on product experience. The dietary avoidance of pork by Jews and Muslims, beef in India, and eggs among certain African tribes (Simoons, 1961) are powerful examples of religious and cultural beliefs influencing taste, smell, and product preferences. The moral and cultural differences that make horse, cat, and dog consumption unheard of in North America, but a common practice in certain other countries is another example. Interestingly, the peoples who disdain these and other food products for cultural or religious reasons often consider them ‘unclean’ or ‘disgusting’; yet those who are not part of that culture view them to be quite normal for consumption. Disgust has also been found to be a major reason for morally motivated vegetarians to avoid all meat products (Rozin et al., 1996). Socioeconomic factors, such as status in society and income level can also affect our product experiences. In an early study of smell preferences, Brill (1932) reported that among 43 common odors the most disliked was that of perspiration, which he said was due to its ‘association with people of the lower class’. Perfumes, on the other hand, are often associated with royalty and wealth, while incense is associated with religious piety and sacredness (Largey and Watson, 2006). In the case of taste, witness the expensive, gourmet and health food tastes of today’s urban socialites that makes drinking a non-specialty shop coffee an experience to be avoided. And compare that to the oftenreported consumption of back alley restaurant waste by the homeless in our society. Clearly, the different socioeconomic situations in which these individuals find themselves drive their product experiences, defining for them what is a good or acceptable product experience and what is a bad or unacceptable product experience. Lastly, when talking about product experiences in today’s marketplace, one must, of course, mention the important social influences of role models. Whether it is a famous Hollywood actress pitching her own brand of perfume, a handsome actor drinking the latest introduction into the beer market, or the soccer superstar eating his favorite candy bar, role models are the essence of product advertising. A variety of role model effects on taste and smell have been reported in children, where elders, teachers, and peers have been used to increase liking for specific foods (see Rozin, 1988, for a review). It has also been shown that father–son and mother–daughter food preferences are more similar than father–daughter or mother–son preferences (Pliner, 1983). However while taste preferences begin early in life, systematic data on the role of parental influences in forming taste preferences in children is weak (Rozin, 1996). In addition to experimental studies, there is a large case study literature in sociology and anthropology describing the variety of social and cultural differences in taste and smell. The reader is referred to recent anthologies and books by Korsmeyer, (1999, 2005), Watson and Caldwell (2005) and Drobnick (2006) for fascinating discussions of the role of culture in chemosensory experience.

10. CONCLUSION In this chapter, we have discussed the physiological, psychological, cognitive and sociocultural factors that affect the taste, smell, chemesthetic, and liking experiences of products in today’s market. Although our treatment of topics has relied heavily on the

122

Product Experience

understanding of the fundamental mechanisms of human taste and smell experience, we feel that only through a thorough understanding of these underlying principles can we begin to understand the complex problem of product experience. It is our opinion that the future of product development in the food, perfume, and personal care industries lies in a combination of breakthroughs in the genetics of taste, smell, and chemesthesis and in a better understanding of the psychological, cultural, and societal factors that form the bases of our everyday product experiences.

REFERENCES Algom, D., Marks, L. E. and Cain, W. (1993). Memory psychophysics for chemosensation: perceptual and mental mixtures of odor and taste. Chemical Senses, 18, 151–160. Allison, A. and Chambers, D. H. (2005). Effects of residual toothpaste flavor on flavor profiles of common foods and beverages. Journal of Sensory Studies, 20, 167–186. Anderson, A. K., Christoff, K., Stappen, I., et al. (2003). Dissociated neural representations of intensity and valence in human olfaction. Nature Neuroscience, 6(2), 196–202. Atanasova, B., Thomas-Danguin T., Chabanet, C., Langlois, D., Nicklaus, S. and Etiévant, P. (2005). Perceptual interactions in odour mixtures: Odour quality in binary mixtures of woody and fruity wine odorants. Chemical Senses, 30, 209–217. Ayabe-Kanamuura, S., Schicker, I., Laska, M., et al. (1998). Differences in perception of everyday odors: A Japanese–German cross-cultural study. Chemical Senses, 23, 31–38. Ayya, N. and Lawless, H. T. (1992). Quantitative and qualitative evaluation of high-intensity sweeteners and sweetener mixtures. Chemical Senses, 17, 245–259. Baeyens, F. and Wrzesniewksi, A. (1996). Toilet rooms, body massages, and smells: Two field studies on human evaluative odor conditioning. Current Psychology: Developmental, Learning, Personality, Social, 15, 77–96. Bartoshuk, L. M. (2000). Comparing sensory experiences across individuals: Recent psychophysical advances illuminate genetic variation in taste perception. Chemical Senses, 25, 447–460. Bartoshuk, L. M., McBurney, D. H. and Pfaffmann, C. (1964). Taste of sodium chloride solutions after adaptation to sodium chloride: Implications for the ‘water taste’. Science, 143(3609), 967–968. Bartoshuk, L. M. and Wolfe, J. M. (1990). Conditioned taste aversions in humans: Are they olfactory aversions? Chemical Senses, 15, 551. Baylis, L. L., Rolls, E. T. and Baylis, G. C. (1995). Afferent connections of the caudolateral orbitofrontal cortex taste area of the primate. Neuroscience, 64, 801–812. Beauchamp, G. K. and Cowart, B. J. (1993). Development of salt taste responses in human newborns, infants and children. In: NIH Publication No. 93-3483, 3. Development, Growth and Senescence in the Chemical Senses, pp. 61–68. Washington, DC: U.S. Department of Health and Human Services. Beauchamp, G. K., Cowart, B. J. and Moran, M. (1986). Developmental changes in salt acceptability in human infants. Developmental Psychology, 19, 75–83. Bell, K. I. and Tepper, B. J. (2006). Short-term vegetable intake by young children classified by 6-n-propylthiouracil bitter-taste phenotype. American Journal of Clinical Nutrition, 84(1), 245–251. Bernstein, I. L. and Webster, M. M. (1980). Learned taste aversions in humans. Physiology and Behavior, 25, 363–366. Birch, L. L. and Marlin, D. W. (1982). I don’t like it; I never tried it: Effects of exposure on two-year old children’s food preferences. Appetite, 3, 353–360. Bonnans, S. and Noble, A. C. (1993). Effect of sweetener type and of sweetener and acid levels on temporal perception of sweetness, sourness and fruitiness. Chemical Senses, 18(3), 273–283. Booth, D. A. (1981). The physiology of appetite. British Medical Bulletin, 37, 135–140. Boyce, J. M. and Shone, G. R. (2006). Effects of ageing on smell and taste. Postgraduate Medical Journal, 82(966), 239–241. Breslin, P. A. S. (1996). Interactions among salty, sour and bitter compounds. Trends in Food Science and Technology, 7, 390–399. Breslin, P. A. S., Davidson, T. L. and Grill, H. J. (1990). Conditioned reversal of reactions to normally avoided tastes. Physiology and Behavior, 47, 535–538. Breslin, P. A. S. and Beauchamp, G. K. (1995). Suppression of bitterness by sodium: variation among bitter taste stimuli. Chemical Senses, 20, 609–623.

Taste, smell and chemesthesis

123

Breslin, P. A. and Huang, L. Q. (2006). Human taste: Peripheral anatomy, taste transduction, and coding. In: T. Hummel and A. Welge-Lüssen (Eds.) Taste and smell: An update (Advances in Oto-rhino-laryngology) Vol. 63, pp. 152–190. Basel, Germany: Karger. Brill, A. A. (1932). The sense of smell in the neuroses and psychoses. Psychoanalytic Quarterly, 1 (Spring), 7–42. Buck, L. B. (1996). Information coding in the vertebrate olfactory system. Annual Review of Neuroscience, 19, 517–544. Buck, L. B. (2000). The molecular architecture of odor and pheromone sensing in mammals. Cell, 100, 611–618. Bufe, B., Breslin, P. A., Kuhn, C., et al. (2005). The molecular basis of individual differences in phenylthiocarbamide and propylthiouracil bitterness perception. Current Biology, 15, 322–327. Burgard, D. R. and Kuznicki, J. T. (1990). Chemometrics: chemical and sensory data. Boca Raton, FL: CRC Press. Cabanac, M. (1971). Physiological role of pleasure. Science, 173, 1103–1107. Cain, W. S. (1974). Perception of odor intensity and the time-course of olfactory adaptation. ASHRAE Transactions, 80, 53–75. Cain, W. S. and Johnson, F. (1978). Lability of odor pleasantness: influence of mere exposure. Perception, 7, 459–465. Cain, W. S. and Murphy, C. (1980). Interaction between chemoreceptive modalities of odour and irritation. Nature, 284, 255–257. Cain, W. S. and Polak, E. H. (1992). Olfactory adaptation as an aspect of odor similarity. Chemical Senses, 17, 481–491. Cain, W. S., Schiet, F. T., Olsson, M. J. and De Wijk, R. A. (1995). Comparison of models of odor interaction. Chemical Senses, 20, 625–637. Cain, W. S., See, L. C. and Tosun, T. (1986). Irritation and odor from formaldehyde: Chamber studies. In: IAQ ’86: Managing the Indoor Air for Health and Energy Conservation, pp. 126–137. The American Society of Heating and Refrigerating and Air-Conditioning Engineers, Atlanta. Cairncross, S. E. and Sjostrom, L. B. (1950). Flavor profiles – a new approach to flavor problems. Food Technology, 4, 308–311. Capaldi, E. D. (1996). Conditioned food preferences. In: E. D. Capaldi (Ed.) Why we eat what we eat: the psychology of eating, pp. 53–80. Washington, DC: American Psychological Association. Capaldi, E. D., Campbell, D. H., Sheffer, J. D. and Bradford, J. P. (1987). Conditioned flavor preferences based on delayed caloric consequences. Journal of Experimental Psychology: Animal Behavior Processes, 13, 150–155. Capaldi, E. D. and Hunter, M. J. (1994). Taste and odor in conditioned flavor preference learning. Animal Learning and Behavior, 22, 355–365. Caporale, G. and Monteleone, E. (2004). Influence of information about manufacturing process on beer acceptability. Food Quality and Preference, 15, 271–278. Caporale, G., Policastro, S., Carlucci, A. and Monteleone, E. (2006). Consumer expectations for sensory properties in virgin olive oils. Food Quality and Preference, 17, 116–12. Cardello, A. V. (1994). Consumer expectations and their role in food acceptance. In: H. J. MacFie and D. M. H. Thomson (Eds.) Measurement of food preferences, pp. 253–297. London: Blackie Academic. Cardello, A. V. (2003). Consumer concerns and expectations about novel food processing technologies: effects on product liking. Appetite, 40, 217–233. Cardello, A. V. (2007). Measuring consumer expectations to improve product development. In: H. MacFie (Ed.) Consumer-led Food Product Development. Cambridge: Woodhead, 223–261. Cardello, A. V. and Sawyer, F. M. (1992). Effects of disconfirmed consumer expectations on food acceptability. Journal of Sensory Studies, 7, 253–277. Cardello, A. V. and Schutz, H. G. (2004). Numerical scale-point locations for constructing the LAM (labeled affective magnitude) scale. Journal of Sensory Studies 19, 341–346. Caress, S. M. and Steinemann, A. C. (2004). Prevalence of multiple chemical sensitivities: A population-based study in the southeastern United States. American Journal of Public Health, 94(5), 746–747. Carlsmith, J. M. and Aronson E. (1963). Some hedonic consequences of the confirmation and disconfirmation of expectancies. Journal of Abnormal and Social Psychology, 66, 151–156. Carmichael, S. T. and Price, J. L. (1995). Limbic connections of the orbital and medial prefrontal cortex in macaque monkeys. Journal of Comparative Neurology, 363, 615–641. Caterina, M. J. and Julius, D. (2001). The vanilloid receptor: A molecular gateway to the pain pathway. Annual Reviews in Neuroscience, 24, 487–517. Cartier, R., Rytz, A., Lecomte, A., Poblete, F., Krystlik, J., Belin, E. and Martin, N. (2006). Sorting procedure as an alternative to quantitative descriptive analysis to obtain a product sensory map. Food Quality and Preference, 17, 562–571. Cerf-Ducastel, B., Van de Moortele, P. F., MacLeod, P., Le Bihan, D. and Faurion, A. (2001). Interaction of gustatory and lingual somatosensory perceptions at the cortical level in the human: a functional magnetic resonance imaging study. Chemical Senses, 26, 371–383.

124

Product Experience

Chambers, K. C. and Bernstein, I. L. (1995). Conditioned flavor aversions. In: R. L. Doty (Ed.) Handbook of olfaction and gestation, pp. 745–773. New York: Dekker. Christensen, C. M. (1980). Effects of taste quality and intensity on oral perception of viscosity. Perception and Psychophysics, 28(4), 315–320. Cometto-Muñiz, J. E. and Cain, W. S. (1984). Temporal integration of pungency. Chemical Senses, 8, 315–327. Cometto-Muñiz, J. E. and Cain, W. S. (1995). Olfactory adaptation. In: R. L. Doty (Ed.) Handbook of olfaction and gestation, pp. 257–282. New York: Marcel Dekker. Cometto-Muñiz, J. E., Cain, W. S. and Abraham, M. H. (2004). Chemosensory additivity in trigeminal chemoreception as reflected by detection of mixtures. Experimental Brain Research, 158, 196–206. Cometto-Muñiz, J. E., Cain, W. S. and Abraham, M. H. (2005). Odor detection of single chemicals and binary mixtures. Behavioural Brain Research, 156, 115–123. Cometto-Muñiz, J. E. and Hernandez, S. M. (1990). Odorous and pungent attributes of mixed and unmixed odorants. Perception and Psychophysics, 47, 391–399. Cowart, B. J. and Beauchamp, G. K. (1990). Early development of taste perception. In: R. L. McBride and H. J. H. MacFie (Eds.) Psychological Basis of Sensory Evaluation, pp. 1–17. London: Elsevier Applied Science. Cutler, W. B., Preti, G., Krieger, A. M., et al. (1986). Human axillary secretions influence women’s menstrual cycles: The role of donor extract from men. Hormones and Behavior, 20, 463–473. Dalton, P. (1996). Odor perception and beliefs about risk. Chemical Senses, 21, 447–458. Dalton, P. (2000). Psychophysical and behavioral characteristics of olfactory adaptation. Chemical Senses, 25, 487–492. Dalton, P., Cowart, B., Dilks, D., et al. (2003). Olfactory function in workers exposed to styrene in the reinforced-plastics industry. American Journal of Industrial Medicine, 44(1), 1–11. Dalton, P., Doolittle, N. and Breslin, P. A. (2002). Gender-specific induction of enhanced sensitivity to odors. Nature Neuroscience, 5(3), 199–200. Dalton, P., Doolittle, N., Nagata, H. and Breslin, P. A. (2000). The merging of the senses: Integration of subthreshold taste and smell. Nature Neuroscience, 3, 431–432. Dalton, P. and Wysocki, C. J. (1996). The nature and duration of adaptation following long-term odor exposure. Perception and Psychophysics, 58(5), 781–792. Davis, L. B. and Porter, R. H. (1991). Persistent effects of early odor exposure on human neonates. Chemical Senses, 16(2), 169–174. Deliza R. and MacFie, H. J. H. (1996). The generation of sensory expectations by external cues and its effect on sensory perception. Journal of Sensory Studies, 11, 103–128. Delwiche, J. F. and Heffelfinger, A.L. (2005). Cross-modal additivity of taste and smell. Journal of Sensory Studies, 20, 512–525. Desor, J. A., Maller, O. and Andrews, K. (1975). Ingestive responses of human newborns to salts, sour and bitter stimuli. Journal of Comparative and Physiological Psychology, 89, 966–970. Desor, J. A., Maller, O. and Turner, R. E. (1973). Taste in acceptance of sugars by human infants. Journal of Comparative and Physiological Psychology, 84, 496–501. Diamond, J., Breslin, P. A. S., Doolittle, N., Nagata, H. and Dalton, P. (2005). Flavor processing: Perceptual and cognitive factors in multi-modal integration. Chemical Senses, 30 (Suppl. 1), i232–i233. Dilks, D. D., Dalton, P. and Beauchamp, G. K. (1999). Cross-cultural variation in responses to malodors. Chemical Senses, 24, 599. Dinehart, M. E., Hayes, J. E., Bartoshuk, L. M., Lanier, S. L. and Duffy, V. B. (2006). Bitter taste markers explain variability in vegetable sweetness, bitterness, and intake. Physiology and Behavior, 87(2), 304–313. Distel, H. and Hudson, R. (2001). Judgement of odor intensity is influenced by subjects’ knowledge of the odor source. Chemical Senses, 26, 247–251. Djordjevic, J., Zatorre, R. J. and Jones-Gotman, M. (2004). Effects of perceived and imagined odors on taste detection. Chemical Senses, 29, 199–208. Doty, R. L. and Cometto-Muñiz, J. E. (2003). Trigeminal chemosensation. In: R. L. Doty (Ed.) Handbook of olfaction and gustation (2nd Ed.), pp. 981–999. New York: Marcel Decker. Drewnoski, A., Grinker, J. A. and Hirsh, J. (1982). Obesity and flavor perception: Multidimensional scaling of soft drinks. Appetite, 3, 361–368. Drobnick, J. (2006). The smell culture reader. New York: Oxford. Duffy, V. B., Backstrand, J. R. and Ferris, A. M. (1995). Olfactory dysfunction and related nutritional risk in free-living, elderly women. Journal of the American Dietetic Association, 95(8), 879–884. Duffy, V. B., Peterson, J. M. and Bartoshuk, L. M. (2004). Associations between taste genetics, oral sensation and alcohol intake. Physiology and Behavior, 82(2–3), 435–445. Engen, T. (1972). The effect of expectation on judgments of odor. Acta Psychologica, 35, 450–458.

Taste, smell and chemesthesis

125

Engen, T. (1982). The perception of odors. New York: Academic. Engen, T. (1988) The acquisition of odour hedonics. In: S. Van Toller and G. H. Dodd (Eds.) Perfumery: The psychology and biology of fragrance, pp. 79–90. London: Chapman and Hall. Essed, N. H., Van Staveren, W. A., Kok, F. J. and De Graaf, C. (2007). No effect of 16 weeks flavor enhancement on dietary intake and nutritional status of nursing home elderly. Appetite, 48, 29–36. Faye, P., Bremaud, D., Teillet, E., Courcoux, P., Giboreau, A. and Nicod, H. (2006). An alternative to external preference mapping based on consumer perceptive mapping. Food Quality and Preference, 17, 604–614. Festinger, L. (1957). A theory of cognitive dissonance. Evanston, IL: Row and Peterson. Frank, R. A. (2002). Response context affects judgments of flavor components in foods and beverages. Food Quality and Preference, 14, 139–145. Frank, R. A. and Byram, J. (1988). Taste–smell interactions are tastant and odorant dependent. Chemical Senses, 13, 445–455. Frank, R. A., Van der Klaauw, N. J. and Schifferstein, H. N. J. (1993). Both perceptual and conceptual factors influence taste–odor and taste–taste interactions. Perception and Psychophysics, 54, 343–350. Ganchrow, J. R. (1995). Ontogeny of human taste perception. In: R. L. Doty (Ed.) Handbook of olfaction and gustation, pp. 715–772. New York: Dekker. Garb, J. L. and Stunkard, A. (1974). Taste aversions in man. American Journal of Psychiatry, 131, 1204–1207. Gibson, J. J. (1966). The senses considered as perceptual systems. Boston: Houghton Mifflin. Gilmore, M. M. and Green, B. G. (1993). Sensory irritation and taste produced by NaCl and citric acid: effects of capsaicin desensitization. Chemical Senses, 18(3), 257–272. Glusman, G., Yanai, I., Rubin, I. and Lancet, D. (2001). The complete human olfactory subgenome. Genome Research, 11, 685–702. Gottfried, J. A. (2006). Smell: Central nervous processing. In: T. Hummel and A. Welge-Lüssen (Eds.) Taste and smell: An update (Advances in Oto-rhino-laryngology), pp. 44–69. Basel, Germany: Karger. Gottfried, J. A., O’Doherty, J. and Dolan, R. J. (2003). Encoding predictive reward value in human amygdala and orbitofrontal cortex. Science, 301, 1104–1107. Green, B. G. (1990). Effects of thermal, mechanical, and chemical stimulation on the perception of oral irritation. In: Green, B. G., Mason, J. R. and Kare, M. R. (Eds.) Chemical senses, Vol. 2, Irritation, pp. 171–175. New York: Marcel Dekker. Green, B. G., Shaffer, G. S. and Gilmore, M. M. (1993). Derivation and evaluation of a semantic scale of oral sensation magnitude with apparent ratio properties. Chemical Senses, 18, 683–702. Guinard, J., Yip, D., Cubero, E. and Mazzucchelli, R. (1999). Quality ratings by experts, and relation with descriptive analysis ratings: a case study with beer. Food Quality and Preference, 10, 59–67. Hallock, R. M. and Di Lorenzo, P. M. (2006). Temporal coding in the gustatory system. Neuroscience and Biobehavioral Reviews, 30(8), 1145–1160. Harris, G. and Booth, D. A. (1987). Infants’ preference for salt in food: Its dependence upon recent dietary experience. Journal of Reproductive and Infant Psychology, 5, 97–104. Helson, H. (1964). Adaptation-level theory: An experimental and systematic approach to behavior. New York: Harper. Hendry, S. H. C., Hsiao, S. S. and Bushnell, M. C. (1999). Somatic sensation. In: M. J. Zigmond, F. E. Bloom, S.C., et al. (Eds.) Fundamental neuroscience, pp. 761–789. San Diego: Academic Press. Herz, R. S. (1997). Emotion experienced during encoding enhances odor retrieval cue effectiveness. American Journal of Psychology, 110, 489–505. Herz, R. S. (2003). The effect of verbal context in olfactory perception. Journal of Experimental Psychology: General, 132, 595–606. Herz, R. S., Eliassen J., Beland S. and Souza T. (2004a). Neuroimaging evidence for the emotional potency of odor-evoked memory. Neuropsychologia, 42, 371–378. Herz, R. S., Beland, S. and Hellerstein, M. (2004b). Changing odor hedonic perception through emotional associations in humans. International Journal of Comparative Psychology, 17, 315–339. Herz, R. S. and Von Clef, J. (2001). The influence of verbal labeling on the perception of odors: Evidence for olfactory illusions? Perception, 30, 381–391. Hetherington, M. M. and Rolls, B. J. (1996). Sensory-specific satiety: theoretical frameworks and central characteristics. In: E. Capaldi (Ed.) Why we eat what we eat: The psychology of eating, pp. 267–290. Washington, DC: American Psychological Association. Hildebrand, J. G. and Shepherd, G. M. (1997). Mechanisms of olfactory discrimination: Converging evidence for common principles across phyla. Annual Review of Neuroscience, 20, 595–631. Hornak J., O’Doherty J., Bramham J., Rolls E.T., et al. (2004). Reward-related reversal learning after surgical excisions in orbito-frontal or dorsolateral prefrontal cortex in humans. Journal of Cognitive Neuroscience, 16(3), 463–478.

126

Product Experience

Huang, A. L., Chen, X., Hoon, M. A., et al. (2006). The cells and logic for mammalian sour taste detection. Nature, 442, 934–938. Hummel, T., Futschik, T., Frasnelli, J. and Huttenbrink, K. B. (2003). Effects of olfactory function, age, and gender on trigeminally mediated sensations: A study based on the lateralization of chemosensory stimuli. Toxicology Letters, 140–141, 273–280. Iaccarino T., DiMonaco R., Mincione A., Cavella S. and Masi P. (2006). Influence of information on origin and technology on the consumer response: The case of soppressata salami. Food Quality and Preference, 17, 76–84. Jacob, T. J., Fraser, C., Wang, L., Walker, V. and O’Connor, S. (2003). Psychophysical evaluation of responses to pleasant and mal-odour stimulation in human subjects; adaptation, dose response and gender differences. International Journal of Psychophysiology, 48, 67–80. Jacquot, L., Monnin, J., Lucarz A. and Brand, G. (2005). Trigeminal sensitization and desensitization in the nasal cavity: a study of cross interactions. Rhinology, 43, 93–98. Jaeger, S. R. (2006). Non-sensory factors in sensory science research. Food Quality and Preference, 17, 132–144. Jaeger, S. R., Rossiter, K. L. and Lau, K. (2005). Consumer perceptions of novel fruit and familiar fruit: A repertory of grid application. Journal of the Science of Food and Agriculture, 85(3), 480–488. Jinks, A. and Laing, D. G. (2001). The analysis of odor mixtures by humans: evidence for a configurational process. Physiology Behavior, 72, 51–63. Johansson, A., Bramerson, A., Millqvist, E., Nordin, S. and Bende, M. (2005). Prevalence and risk factors for self-reported odour intolerance: the Skovde population-based study. International Archives of Occupational and Environmental Health, 78(7), 559–564. Johnson, J. L. and Vickers, Z. (1991). Sensory-specific satiety for selected bread products. Journal of Sensory Studies, 6(2), 65–79. Jones, L. V. and Thurstone, L. L. (1955). The psychophysics of semantics: an experimental investigation. Journal of Applied Psychology, 39(1), 31–36. Kadohisa, M. and Wilson, D. A. (2006). Olfactory cortical adaptation facilitates detection of odors against background. Journal of Neurophysiology, 95(3), 1888–1896. Kahkonen P., Hakanpaa P. and Tourila H. (1999). The effects of information related to fat content and taste on consumer responses to a reduced-fat frankfurter and a reduced-fat chocolate bar. Journal of Sensory Studies, 14, 35–46. Kahkonen P. and Tuorila H. (1998). Effect of reduced-fat information on expected and actual hedonic and sensory ratings of sausage. Appetite, 30, 13–23. Kahkonen P., Tuorila H. and Rita, H. (1996). How information enhances the acceptability of low-fat spread. Food Quality Preference, 7, 87–94. Keiger, C. J., Case, L. D., Kendal-Reed, M., et al. (2003). Nicotinic cholinergic receptor expression in the human nasal mucosa. Annals of Otology, Rhinology and Laryngology, 112(1), 77–84. Kepecs, A., Uchida, N. and Mainen, Z. F. (2006). The sniff as a unit of olfactory processing. Chemical Senses, 31, 167–179. Korsmeyer, C. (1999). Making sense of taste, food and philosophy. Ithaca: Cornell University Press. Korsmeyer, C. (2005). The taste culture reader, experiencing food and drink. Oxford: Berg. Koskinen, S., Kalviainen, N. and Tuorila, H. (2003). Flavor enhancement as a tool for increasing pleasantness and intake of a snack product among the elderly. Appetite, 41(1), 87–96. Kreutzer, R., Neutra, R. R., and Lashuay, N. (1999). Prevalence of people reporting sensitivities to chemicals in a population-based survey. American Journal of Epidemiology, 150, 1–12. Kroeze, J. H. A. (1990). The perception of complex taste stimuli. In: R. L. McBride and H. J. H. MacFie (Eds.) Psychological Basis of Sensory Evaluation, pp. 41–68. London: Elsevier Applied Science. Kroeze, J. H. A. and Bartoshuk, L. M. (1985). Bitterness suppression as revealed by split-tongue taste stimulation in humans. Physiology and Behavior, 35, 779–783. Labbe, D., Damevin, L., Vaccher, C., Morgenegg, C. and Martin, N. (2006). Modulation of perceived taste by olfaction in familiar and unfamiliar beverages. Food Quality and Preference, 17, 582–589. Labows, J. N. and Wysocki, C. J. (1984). Individual differences in odor perception. Perfumer and Flavorist, 9, 21–26. Laing, D. G., Panhuber, H., Wilcox, M. E. and Pittman, E. A. (1984). Quality and intensity of binary odor mixtures. Physiology and Behavior, 33, 309–319. Langron, S. P., Noble, A. C. and Williams, A. A. (1983). Measurement and Control. In: R. K. Atkin and A. A. Williams (Eds.) Sensory quality in foods and beverages: Definition, p. 325. Chichester: Ellis Horwood. Lanier, S. A., Hayes, J. E. and Duffy, V. B. (2005). Sweet and bitter tastes of alcoholic beverages mediate alcohol intake in of-age undergraduates. Physiology and Behavior, 83(5), 821–831. Largey, G. and Watson, R. (2006). The sociology of odors. In: J. Drobnick (Ed.) The smell culture reader, pp. 29–40. Oxford: Berg.

Taste, smell and chemesthesis

127

Laska, M., Hudson, R. and Distel, H. (1990). Olfactory sensitivity to biologically relevant odors may exceed the sum of component thresholds. Chemoecology, 1, 139–141. Lawless, H. T. (1979). Evidence for neural inhibition in bittersweet taste mixtures. Journal of Comparative and Physiological Psychology, 93, 538–547. Lawless, H. T. (1983). Contextual effects in category ratings. Journal of Testing and Evaluation, 11, 346–349. Lawless, H. T. (1987). Gustatory psychophysics. In: T. E. Finger and W. L. Silver (Eds.) Neurobiology of taste and smell, pp. 401–420. New York: John Wiley. Lawless, H. T. (1997). Olfactory psychophysics. In: G. Beauchamp and L. Bartoshuk (Eds.) Tasting and smelling, pp. 125–168. San Diego, CA: Academic Press. Lawless, H. T. and Heymann, H. (1998). Sensory evaluation of food: Principles and practices. New York: Chapman and Hall. Lawless, H. T., Rozin, P. and Shenker, J. (1985). Effects of oral capsaicin on gustatory, olfactory and irritant sensations and flavor identification in humans who regularly or rarely consume chilli pepper. Chemical Senses, 10, 579–589. Lawless, H. T. and Stevens, D. A. (1984). Effects of oral chemical irritation on taste. Physiology and Behavior, 32, 995–998. Lawless, H. T., Sheng, N. and Knoops, S. S. C. P. (1995). Multidimensional scaling of sorting data applied to cheese perception. Food Quality and Preference, 6, 91–98. Lemon, C. H. and Smith, D. V. (2006). Influence of response variability on the coding performance of central gustatory neurons. Journal of Neuroscience, 26(28), 7433–7443. Li, X., Li, W., Wang, H., et al. (2005). Pseudogenization of a sweet-receptor gene accounts for cats’ indifference toward sugar. PLoS Genetics, 1, 27–35. Liberles, S. D. and Buck, L. B. (2006). A second class of chemosensory receptors in the olfactory epithelium. Nature, 442(7103), 645–650. Lindemann, B. (2001). Receptors and transduction in taste. Nature, 413, 219–225. Liu, L. and Simon, S. A. (2001). Acidic stimuli activate two distinct pathways in taste receptor cells from rat fungiform papillae. Brain Research, 923, 58–70. Livermore, A. and Laing, D. G. (1996). Influence of training and experience on the perception of multicomponent odor mixtures. Journal of Experimental Psychology: Human Perception and Performance, 22, 267–277. Lledo, P. M., Gheusi, G. and Vincent, J. D. (2005). Information processing in the mammalian olfactory system. Physiological Reviews, 85, 281–317. Mainland, J. and Sobel, N. (2006). The sniff is part of the olfactory percept. Chemical Senses, 31, 181–196. Marchand, S. and Arsenault, P. (2002). Odors modulate pain perception: A gender-specific effect. Physiology and Behavior, 76(2), 251–256. Marshall, K., Laing, D.G., Jinks, A. L. and Hutchinson, I. (2006). The capacity of humans to identify components in complex odor-taste mixtures. Chemical Senses, 31, 539–545. Mathey, M. F., Siebelink, E., De Graaf, C. and Van Staveren, W. A. (2001). Flavor enhancement of food improves dietary intake and nutritional status of elderly nursing home residents. Journals of Gerontology Series A: Biological Sciences and Medical Sciences, 56(4), M200–M205. Matlin, M. W. and Foley, H. J. (1992). Sensation and perception. Boston: Allyn and Bacon. Mattes, R. D. (1987). Sensory influences on food intake and utilization in humans. Human Nutrition: Applied Nutrition, 41A, 77–95. McBride, R. L. (1985). Stimulus range influences intensity and hedonic ratings of flavour. Appetite, 6, 125–131. McBride, R. L. and Johnson, R. L. (1987). Perception of sugar-acid mixtures in lemon juice drink. International Journal of Food Science and Technology, 22, 299–408. McBurney, D. H. and Shick, T. R. (1971). Taste and water taste of twenty-six compounds for man. Perception and Psychophysics, 10, 249–252. McClure, S. M., Li, J., Tomli, D., Cypert, K. S., Montagu, L. M. and Montague, P. R. (2004). Neural correlates of behavioral preference for culturally familiar drinks. Neuron, 44, 379–387. McDaniel, A. H. and Reed, D. R. (2004). The human sweet tooth and its relationship to obesity. In: N. Moustaid-Moussa and C. Berdanier (Eds.) Genomics and proteomics in nutrition, pp. 49–67. New York: Marcel Dekker. McKemy, D. D. (2005). How cold is it? TRPM8 and TRPA1 in the molecular logic of cold sensation. Molecular Pain, 1, 1–7. Meilgaard, M., Civille, G. V. and Carr, B. T. (1999). Sensory evaluation techniques. Boca Raton, Fl: CRC Press. Mennella, J. A., Johnson, A. and Beauchamp, G. K. (1995). Garlic ingestion by pregnant women alters the odor of amniotic fluid. Chemical Senses, 20, 207–209. Mennella, J. A., Pepino, M. Y. and Reed, D. R. (2005). Genetic and environmental determinants of bitter perception and sweet preferences. Pediatrics, 115(2), e216–e222.

128

Product Experience

Midkiff, E. E. and Bernstein, I. L. (1985). Targets of learned food aversions in humans. Physiology and Behavior, 34, 839–841. Mojet, J., Christ-Hazelhof, E. and Heidema, J. (2001). Taste perception with age: Generic or specific losses in threshold sensitivity to the five basic tastes? Chemical Senses, 26, 845–860. Mojet, J., Heidema, J. and Christ-Hazelhof, E. (2003). Taste perception with age: Generic or specific losses in supra-threshold intensities to the five taste qualities? Chemical Senses, 28, 397–413. Mombaerts, P. (2004). Genes and ligands for odorant, vomeronasal, and taste receptors. Nature Reviews: Neuroscience, 5, 263–278. Moncreiff, R. W. (1966). Odour preferences. New York: Wiley. Moskowitz, H. R. and Barbe, C. D. (1977). Profiling of odor components and their mixtures. Sensory Processes, 1, 212–226. Moskowitz, H. R. and Sidel, J. L. (1971). Magnitude and hedonic scales of food acceptability. Journal of Food Science, 36, 677–680. Mueller, K. L., Hoon, M. A., Erlenbach, I., et al. (2005). The receptors and coding logic for bitter taste. Nature, 434, 225–229. Murphy, C. (1982). Effects of exposure and context on hedonics of olfactory–taste mixtures. Sensory Processes, 1, 212–226. Murphy, C. (1986). Taste and smell in the elderly. In: H. L. Meiselman and R. S. Rivlin (Eds.) Clinical measurement of taste and smell, pp. 343–375. New York: Macmillan. Murphy, C. (1993). Nutrition and chemosensory perception in the elderly. Critical Reviews in Food Science and Nutrition, 33(1), 3–15. Murphy, C. and Cain, W. S. (1980). Taste and olfaction: Independence versus interaction. Physiology Behavior, 24, 601–605. Murphy, C., Schubert, C. R., Cruickshanks, K. J., et al. (2002). Prevalence of olfactory impairment in older adults. Journal of the American Medical Association, 288, 2307–2312. Nasrawi, C.W. and Pangborn, R.M. (1989). The influence of tastants on oral irritation by capsaicin. Journal of Sensory Studies, 3, 287–294. Navarrete-Palacios, E., Hudson, R., Reyes-Guerrero, G. and Guevara-Guzman, R. (2003). Lower olfactory threshold during the ovulatory phase of the menstrual cycle. Biological Psychology, 63(3), 269–279. O’Doherty, J., Rolls, E. T., Francis, S., et al. (2000). Sensory-specific satiety-related olfactory activation of the human orbitofrontal cortex. Neuroreport, 11, 399–403. Olson J.C. and Dover P. (1976). Effects of expectation creation and disconfirmation on belief elements of cognitive structure. In: B. B. Anderson (Ed.) Advances in Consumer Research, 3, 168–175. Association for Consumer Research. O’Mahoney, M. (1978). Smell illusions and suggestion: reports of smells contingent on tones played on radio and television. Chemical Senses and Flavor, 3, 183–189. Parducci, A. (1974). Contextual effects: a range-frequency analysis. In: E. C. Carterette and M. P. Friedman (Eds.) Handbook of perception, psychophysical judgment and measurement, pp. 127–141. New York: Academic. Pause, B. M., Sojka, B., Krauel, K., Fehm-Wolfsdorf, G. and Ferstl, R. (1996). Olfactory information processing during the course of the menstrual cycle. Biological Psychology, 44(1), 31–54. Peier, A. M., Moqrich, A., Hergarden, A. C., Reeve, A. J., Andersson, D. A., Story, G. M., et al. (2002). A TRP channel that senses cold stimuli and menthol. Cell, 108, 705–715. Pelchat, M. L. and Rozin, P. (1982). The special role of nausea in the acquisition of food dislikes by humans. Appetite, 3, 341–351. Perng, C. M. and McDaniel, M. R. (1989). Optimization of a blackberry juice drink using response surface methodology. Presented at the 49th Annual Meeting of the Institute of Food Technologists. Chicago, IL. Peryam, D. R. and Pilgrim, F. J. (1957). Hedonic scale method of measuring food preferences. Food Technology, 11, 9–14. Pierce, J. D., Jr., Wysocki, C. J., Aronov, E. V., Webb, J. B. and Boden, R. M. (1996). The role of perceptual and structural similarity in cross-adaptation. Chemical Senses, 21, 223–237. Pierce, J. D., Jr., Zeng, X. N., Aronov, E. V., Preti, G. and Wysocki, C. J. (1995). Cross-adaptation of sweatysmelling 3-methyl-2-hexenoic acid by a structurally-similar, pleasant-smelling odorant. Chemical Senses, 20, 401–411. Pliner, P. (1983). Family resemblance in food preferences. Journal of Nutrition Education, 15, 137–140. Pol, H. E. H., Hijman, R., Baare, W. F. C. and van Ree, J. M. (1998). Effects of context on judgements of odor intensities in humans. Chemical Senses 23, 131–135. Popper, R. and Heymann, H. (1996). Analyzing differences among products and panelists by multidimensional scaling. In: T. Naes and E. Risvik (Eds.) Multivariate analysis of data in sensory science, pp. 159–184. Amsterdam: Elsevier.

Taste, smell and chemesthesis

129

Porter, R. H., Makin, J. W., Davis, L. B. and Christensen, K. M. (1992). Breast-fed infants respond to olfactory cues from their own mother and unfamiliar lactating females. Infant Behavior and Development, 15, 85–93. Powers, J. J. (1988). Mathematics of a lady tasting tea revisited. Journal of Sensory Studies, 3, 151–158. Prescott, J. and Swain-Campbell, N. (2000). Responses to repeated oral irritation by capsaicin, cinnamaldehyde and ethanol in PROP tasters and non-tasters. Chemical Senses, 25, 239–246. Preti, G., Wysocki, C. J., Barnhart, K. T., Sondheimer, S. J. and Leyden, J. J. (2003). Male axillary extracts contain pheromones that affect pulsatile secretion of luteinizing hormone and mood in women recipients. Biology of Reproduction, 68, 2107–2113. Price, J. L. (1973). An autoradiographic study of complementary laminar patterns of termination of afferent fibers to the olfactory cortex. Journal of Comparative Neurology, 150, 87–108. Rawcliffe, T., Howgate, P. and Geelhoed, E. N. (1992). Patterns of odours similarity among carbonyls and their mixture. Chemical Senses, 17, 119–125. Rawson, N. E. and Yee, K. K. (2006). Transduction and coding. Advances in Oto-Rhino-Laryngology, 63, 23–43. Rifkin, B. and Bartoshuk, L. M. (1980). Taste synergism between monosodium glutamate and disodium 5⬘-guaylate’. Physiology and Behavior, 24, 1169–1172. Rodriguez, I., Greer, C. A., Mok, M. Y. and Mombaerts, P. (2000). A putative pheromone receptor gene expressed in human olfactory mucosa. Nature Genetics, 26, 18–19. Rolls, B. J. (1999). Do chemosensory changes influence food intake in the elderly? Physiology and Behavior, 66(2), 193–197. Rolls, E. T. (2000). The orbitofrontal cortex and reward. Cerebral Cortex, 10, 284–294. Rolls, E. T. (2001). The rules of formation of the olfactory representations found in the orbitofrontal cortex olfactory areas in primates. Chemical Senses, 26, 595–604. Rolls, E. T. and Baylis, L. L. (1994). Gustatory, olfactory, and visual convergence within the primate orbitofrontal cortex. Journal of Neuroscience, 14, 5437–5452. Rolls, B. J., Hetherington, M. and Burley, V. J. (1988). The specificity of satiety: the influence of foods of different macronutrient content on the development of satiety. Physiology and Behavior, 43, 145–153. Rolls, B. J., Rolls, E. T., Rowe, E. A. and Sweeney, K. (1981). Sensory specific satiety in man. Physiology and Behavior, 27, 137–142. Rozin, P. (1988). Social learning about foods by humans. In: T. Zental and B. G. Galef, Jr. (Eds.) Social learning: A comparative approach, pp. 165–187. Hillsdale, NJ: Erlbaum. Rozin, P. (1996). Sociocultural influences on human food selection. In: E. Capaldi (Ed.) Why we eat what we eat: The psychology of eating, pp. 233–263. Washington, DC: American Psychological Association. Rozin, P., Markwith, M. and Stoess, C. (1996). Moralization and becoming a vegetarian: the transformation of preferences into values and the recruitment of disgust. Psychological Science, 8, 67–73. Rozin, P. and Schiller, D. (1980). The nature and acquisition of a preference for chili pepper by humans. Motivation and Emotion, 4, 77–100. Rozin, P. and Tuorila, H. (1993). Simultaneous and temporal contextual influences on food acceptance. Food Quality and Preference, 4, 11–20. Rozin, P. and Vollmecke, T. A. (1986). Food likes and dislikes. Annual Review of Nutrition, 6, 433–456. Santos, D. V., Reiter, E. R., DiNardo, L. J. and Costanzo, R. M. (2004). Hazardous events associated with impaired olfactory function. Archives of Otolaryngology – Head and Neck Surgery, 130, 317–319. Sarinopoulos, I., Dixon, G. E., Short, S. J., Davidson, R. J. and Nitschke, J. B. (2006). Brain mechanisms of expectation associated with insula and amygdala response to aversive taste: implications for placebo. Brain, Behavior, and Immunity, 20, 120–132. Sauvageot, F., Urdapilleta, I. and Peyron, D. (2006). Within and between variations of texts elicited from nine experts in wines. Food Quality and Preference, 17(6), 429–444. Savic, I., Gulyas, B. and Berglund, H. (2002). Odorant differentiated pattern of cerebral activation: comparison of acetone and vanillin. Human Brain Mapping, 17, 17–27. Schaal, B., Marlier, L. and Soussignan, R. (1998). Olfactory function in the human fetus: Evidence from selective neonatal responsiveness to the odor of amniotic fluid. Behavioral Neuroscience, 112, 1438–1449. Schafe, G. E. and Bernstein, I. L. (1996). Taste aversion learning. In: E. Capaldi (Ed.) Why we eat what we eat: the psychology of eating, pp. 31–52. Washington, D.C.: American Psychological Association. Scharf, A. and Volkmer, H. P. (2000). The impact of olfactory product expectations on the olfactory product experience. Food Quality and Preference, 11, 497–504. Schifferstein, H. N. J. (1995). Contextual shifts in hedonic judgments. Journal of Sensory Studies, 10, 381–392. Schifferstein, H. N. J. (2001). Effects of product beliefs on product perception and liking. In: L. Frewer, E. Risvik. and H. Schifferstein. Food, people and society: A European perspective of consumers’ food choices, London: Springer Verlag. Schifferstein, H. N. J. and Frijters, J. E. R. (1992). Contextual and sequential effects on judgments of sweetness intensity. Perception and Psychophysics, 52(3), 243–255.

130

Product Experience

Schifferstein, H. N. J. and Oudejans, I. M. (1996). Determinants of cumulative successive contrast in saltiness intensity judgments. Perception and Psychophysics, 58(5), 713–724. Schiffman, S.S. et al. (1995). Investigation of synergism in binary mixtures of sweeteners. Brain Resource Bulletin, 38, 105–120. Schiffman, S. S. (2000). Intensification of sensory properties of foods for the elderly. Journal of Nutrition, 130(Suppl. 4S), 927S–930S. Schleidt, M., Hold, B. and Attila, G. (1981). A cross-cultural study on the attitude towards personal odors. Journal of Chemical Ecology, 7, 19–31. Schmidt, H. J. and Beauchamp, G. K. (1988). Adult-like preferences and aversions in three-year-old children. Child Development, 59, 1136–1143. Schutz, H. G. and Cardello, A. V. (2001). A labeled affective magnitude (LAM) scale for assessing food liking/ disliking. Journal of Sensory Studies, 16, 117–159. Shinohara, K., Morofushi, M., Funabashi, T. and Kimura, F. (2001). Axillary pheromones modulate pulsatile LH secretion in humans. Neuroreport, 12, 893–895. Shusterman, D. (2002). Individual factors in nasal chemesthesis. Chemical Senses, 27, 551–564. Simoons, F. J. (1961). Eat not this flesh. Madison: University of Wisconsin Press. Siret, F. and Issanchou, S. (2000). Traditional process: influence on sensory properties and on consumers’ expectation and liking. Application to pate’ de campagne. Food Quality and Preference, 11, 217–228. Slosson, E. E. (1899). A lecture experiment in hallucinations. Psychology Review, 6, 407–408. Small, D. M. (2006). Central gustatory processing in humans. In: T. Hummel and A. Welge-Lüssen (Eds.) Taste and smell: An update (Advances in Oto-rhino-laryngology), pp. 191–220. Basel, Germany: Karger. Small, D. M., Gregory, M. D., Mak, Y. E., et al. (2003). Dissociation of neural representation of intensity and affective valuation in human gustation. Neuron, 39, 701–711. Small, D. M., Jones-Gotman, M., Zatorre, R. J., Petrides, M. and Evans, A. C. (1997). Flavor processing: More than the sum of its parts. Neuroreport, 8, 3913–3917. Small, D. M., Voss, J., Mak, Y. E., Simmons, K., Parrish, T. and Gitelman, D. (2004). Experience-dependent neural integration of taste and smell in the human brain. Journal of Neurophysiology, 92, 1892–1903. Smith, D. V. and Margolskee, R. F. (2001). Making sense of taste. Scientific American, 284, 32–39. Smith, D. V. and Shepherd, G. M. (1999). Chemical senses: Taste and olfaction. In: M. J. Zigmond, F. E. Bloom, S. C. Landis, et al. (Eds.) Fundamentals of neuroscience, pp. 719–760. San Diego, CA: Academic Press. Smith-Swinto