Product Experience

PREFACE ‘With this collection of chapters,’ say Hendrik Schifferstein and Paul Hekkert, ‘we hope to lay down the basic...

Author: Hendrik N. J. Schifferstein | Paul Hekkert

328 downloads 3132 Views 6MB Size Report

This content was uploaded by our users and we assume good faith they have the permission to share this book. If you own the copyright to this book and it is wrongfully on our website, we offer a simple DMCA procedure to remove your content from our site. Start by pressing the button below!

Report copyright / DMCA form

DOWNLOAD PDF

PREFACE

‘With this collection of chapters,’ say Hendrik Schifferstein and Paul Hekkert, ‘we hope to lay down the basics for the field of product experience research’. With these words, Schifferstein and Hekkert set forth a daunting challenge, but a necessary and important endeavor. Although industrial design has long been concerned with products, the emphasis has been on the physical structures of the devices, the forms and shapes, the materials, and the manufacturing processes to be followed. These are all important, to be sure, but today the focus has shifted from objects to the experiences that result from interacting with them. To understand experience we need to go beyond shape and form, even beyond simple ergonomics. We need to understand how psychology, the social sciences, communication, and business shape a person’s experience. This book provides an important first step. Design is hot: It is the topic of the day. Moreover, design is now coming to be understood in the broader sense, not just in the styling that provides a pretty face to products. Design is now recognized as a process applicable to all phases of product development. Design is, moreover, a multidisciplinary endeavor, requiring knowledge of a wide range of topics. The result is that the field of product design is entering a renaissance period. It’s a new world. It’s a challenge. And, of course, challenges are opportunities. But where to get all this new knowledge? Product Experience provides a powerful start. Schifferstein and Hekkert have packed an incredible array of information into the 27 chapters of the book, introducing the reader to multiple perspectives of product experience. From the senses to shopping, the chapters cover a rich gamut of issues. Schifferstein and Hekkert set the stage well with their introduction that reviews the many chapters,

xix

xx

PREFACE

providing structure and guidance. Then comes the meat of the book, the 27 chapters by recognized authorities, followed by Schifferstein and Hekkert’s closing reflections, neatly tying things together. The result encompasses many of the varied components and perspectives involved in understanding how products shape experience. Product Experience, the book, provides an essential reference work for students, instructors, and practicing designers. How do people experience products? What is the nature of the interaction, that dynamic interplay between the perceptions, actions, and understanding of the person experiencing the product? For the answers, start with this book. Don Norman1 Palo Alto, California Evanston, Illinois

1

Donald A. Norman. www.jnd.org [email protected] Nielsen Norman group and Northwestern University

LIST OF CONTRIBUTORS

The chapter(s) contributed to are indicated by the numbers in parentheses. Katja Battarbee, IDEO, Palo Alto, CA 94301, USA (19) Christina Bodin Danielsson, School of Architecture and Built Environment, The Royal Institute of Technology, 10044 Stockholm, Sweden (26) Stella Boess, Department of Industrial Design, Faculty of Industrial Design Engineering, Delft University of Technology, 2628 CE Delft, The Netherlands (12) Ünal Ö. Boya, Marketing Department, Appalachian State University, Boone NC 28608, USA (25) Reinhart Butter, Department of Industrial, Interior and Visual Communication Design, Ohio State University, Columbus, Ohio 43210, USA (14) Armand V. Cardello, Office of the Technical Director, US Army Natick Soldier Center, Natick, MA 01760-5020, USA (4) John M. Carroll, School of Information Sciences and Technology, The Pennsylvania State University, University Park PA 16802, USA (21) John Clarkson, Department of Engineering, University of Cambridge, Cambridge CB2 1PZ, United Kingdom (6) Gerald C. Cupchik, Department of Psychology, University of Toronto at Scarborough, Department of Life Sciences, Scarborough, ON, Canada M1C 1A4 (9) Michiel P. de Looze, TNO, 2130 AS Hoofddorp, The Netherlands (18) Pieter M.A. Desmet, Department of Industrial Design, Faculty of Industrial Design Engineering, Delft University of Technology, 2628 CE Delft, The Netherlands (15)

xxi

xxii

List of contributors

Jörgen Eklund, Department of Mechanical Engineering, Division of Quality and Human Systems Engineering, Linköping University, SE-58183 Linköping, Sweden (20) Ann Marie Fiore, Department of Textiles and Clothing, Iowa State University, Ames IA 50011, USA (27) Lawrence L. Garber, Department of Business Administration, Elon University, Elon, NC 27244, USA (25) Marc Hassenzahl, Economic Psychology and Human–Computer Interaction, University of Koblenz-Landau, Universität Campus Landau, 76829 Landau, Germany (11) Paul Hekkert, Department of Industrial Design, Faculty of Industrial Design Engineering, Delft University of Technology, 2628 CE Delft, The Netherlands (10) Michelle C. Hilscher, Department of Psychology, University of Toronto at Scarborough, Department of Life Sciences, Scarborough, ON, Canada M1C 1A4 (9) Eva M. Hyatt, Marketing Department, Appalachian State University, Boone NC 28608, USA (25) Shigekazu Ishihara, Department of Kansei Ergonomics, Hiroshima International University, Hiroshima, Japan (20) Jeroen Jansz, Amsterdam School of Communications Research (ASCoR), University of Amsterdam, 1012 CX Amsterdam, The Netherlands (23) Heimrich Kanis, Department of Industrial Design, Faculty of Industrial Design Engineering, Delft University of Technology, 2628 CE Delft, The Netherlands (12) David V. Keyson, Department of Industrial Design, Faculty of Industrial Design Ingeneering, Delft University of Technology, 2628 CE Delft, The Netherlands (22) Ilpo Koskinen, University of Art and Design Helsinki, 00560 Helsinki, Finland (19) Klaus Krippendorff, The Annenberg School for Communication, University of Pennsylvania, Philadelphia, PA 19104-6220, USA (14) Helmut Leder, Faculty of Psychology, University of Vienna, 1010 Wien, Austria (10) Herbert L. Meiselman, US Army Natick Soldier Center, Natick MA 01760-5020, USA, (24) Helena M. Mentis, School of Information Sciences and Technology, The Pennsylvania State University, University Park PA 16802, USA (21) Ruth Mugge, Department of Product Innovation Management, Faculty of Industrial Design Engineering, Delft University of Technology, 2628 CE Delft, The Netherlands (17) Mitsuo Nagamachi, User Science Institute, Kyushu University, Fukuoka 815-8540 Japan (20) Harold T Nefs, School of Psychology, University of St. Andrews, St. Mary’s College, St. Andrews, KY16 9JP, United Kingdom (1) Marsha L. Richins, College of Business, University of Missouri, Columbia, MO 65211, USA (16) Axel Roesler, Division of Design, University of Washington, Seattle, WA 98195, USA (7, 8)

List of contributors

xxiii

Hendrik N.J. Schifferstein, Department of Industrial Design, Faculty of Industrial Design Engineering, Delft University of Technology, 2628 CE Delft, The Netherlands (2, 5, 17) Jan P.L. Schoormans, Department of Product Innovation Management, Faculty of Industrial Design Engineering, Delft University of Technology, 2628 CE Delft, The Netherlands (17) Simon Schütte, Department of Mechanical Engineering, Division of Quality and Human Systems Engineering, Linköping University, SE-58183 Linköping, Sweden (20) Marieke H. Sonneveld, Department of Industrial Design, Faculty of Industrial Design Engineering, Delft University of Technology, 2628 CE Delft, The Netherlands (2) Charles Spence, Department of Experimental Psychology, University of Oxford, Oxford, OX1 3UD, United Kingdom (5) Ed S. Tan, Amsterdam School of Communications Research (ASCoR), University of Amsterdam, 1012 CX Amsterdam, The Netherlands (23) René van Egmond, Department of Industrial Design Faculty of Industrial Design Engineering, Delft University of Technology, 2628 CE Delft, The Netherlands (3) Thomas J.L. van Rompay, Department of Marketing Communication and Consumer Psychology, Faculty of Behavioral Sciences, Twente University 7500 AE Enschede, the Netherlands (13) Peter Vink, TNO, 2130 AS Hoofddorp, the Netherlands (18) Paul M. Wise, Monell Chemical Senses Center, Philadelphia, PA 19104-3308, USA (4) David Woods, Institute for Ergonomics, Ohio State University, 210 Baker Systems, Columbus OH 43210, USA (7, 8)

INTRODUCING PRODUCT EXPERIENCE PAUL HEKKERT AND HENDRIK N.J. SCHIFFERSTEIN

People live in a world in which they are surrounded by designed artifacts and services, products that were created by (other) people to serve some purpose: To get from one place to another; to clean the house; to cook; to feed and protect oneself; to contact someone; to have fun; to retrieve information; and so on. As a result, we have cars and bicycles, tissues and vacuum cleaners, shopping malls and living rooms, mobile phones and weblogs, computer games and festivals, archive systems and SMS messaging, and to use these products we need to interact with them. Notwithstanding the fact that the way in which people interact with a product is clearly product-dependent, they always use their senses to perceive it, they use their motor system and their knowledge to operate or communicate with it, and during the interaction they process the information they perceive, they may experience one or more emotions, and they are likely to form an affective evaluation of the product. Thus, although the interaction may be product-specific, the processes that are activated during the interaction are similar over products. As a consequence, it should be possible to develop an overall theoretical framework that guides the study of how people experience products. In the current book, we bring together existing work from several different research areas, and make a first attempt to integrate this work, as the starting point for the development of an overall, encompassing framework of product experience.

Theoretical views on product experience Before we can develop a conceptual framework, we first need to define the concept that we are interested in. In this book, we define the field of ‘product experience’ as the research area that develops an understanding of people’s subjective experiences that result from interacting with products. By focusing on products we restrict ourselves to physical objects or non-physical designs that have a utilitarian function, thereby excluding works of art and other non-utilitarian artifacts. Building on our previous definitions Product Experience Copyright © 2008 Elsevier Ltd.

1

2

Product Experience

(Hekkert, 2006; Schifferstein and Cleiren, 2005), we define the subjective product experience as the awareness of the psychological effects elicited by the interaction with a product, including the degree to which all our senses are stimulated, the meanings and values we attach to the product, and the feelings and emotions that are elicited. It can be debated whether these psychological consequences are always affective in nature. According to Russell (2003), core affect – the combination of pleasure and arousal – is, to varying degrees, ‘involved in most psychological events’ [p. 148, emphasis added]. Not all authors in this book embrace our definition of product experience. Some hold a different view and explain in their chapters where their definition originates from, and why they think it fits their research purposes better. These different perspectives from various fields of scientific endeavor add to the richness of our understanding of the concept. Our definition of product experience is quite broad compared to the definitions adopted by other authors and, thereby, generally encompasses their definitions. Therefore, in our view, all these authors investigate some aspects or antecedents of ‘product experience’, even though their reference frame may be narrower than ours. For example, Carroll and Mentis (Chapter 21) give an overview of how research on user experiences that emerged in the field of human–computer interaction yielded new insights that enriched research on product usability in the area of human factors. Battarbee and Koskinen (Chapter 19) use a designer’s perspective on product experience and stress the importance of social interaction in the formation of experiences. Cupchik and Hilscher (Chapter 9) explicitly limit the term ‘experience’ to special life events. They state that ‘experience refers to uniquely meaningful life events with both cognitive and affective qualities’ [emphasis added]. The contrast between Cupchik and Hilscher’s definition and ours becomes explicit in the two ways in which the term ‘experience’ can be translated into German or Dutch. All experiences, including very common, day-to-day experiences, are referred to with the German word ‘Erfahrung’ (Dutch: ‘ervaring’), whereas experiences of special, memorable events are better captured by the word ‘Erlebnis’ (Dutch: ‘belevenis’). The Erlebnis definition is closest to the views found in business literature on the Experience Economy (e.g. Pine and Gilmore, 1998). According to Pine and Gilmore, ‘an experience occurs when a company intentionally uses services as the stage, and goods as props, to engage individual consumers in a way that creates a memorable event’ (p. 98). Pine and Gilmore suggest that producers and retailers need to create special experiences for their customers, beyond simply offering good products. Their notion of experience thus resembles the uniquely meaningful events Cupchik and Hilscher refer to. Such experiences can occasionally be bound to individual products, but will more likely arise from designed spaces, installations, and attractions typically encountered in exhibitions and amusement parks. The design of these kinds of overwhelming and compelling events is often denoted as ‘experience design’. Designers and marketers may wonder whether it is always necessary to create an ‘Erlebnis’, or whether it will be sufficient to deliberately create, alter or improve the ‘Erfahrung’ that people have when they interact with products. According to Schmitt (1999), for example, the ultimate goal of experiential marketing is to create a desirable, coherent, and consistent customer impression that enhances the brand image. For companies it may be worthwhile to create possibilities for potential consumers to explore their products in a specific context, as part of a branding strategy. When people repeatedly encounter a particular brand within the context of a pleasant experience, they are more likely to develop a favourable attitude towards this brand. In addition, characteristics of the atmosphere during the experience (e.g. modern, fresh, impressive) may become associated with the brand.

3

Introducing product experience

The approaches sketched above, by Pine and Gilmore and Schmitt, were developed from a marketing perspective, in which the purchase and consumption of the product plays a central role. This deviates from the user perspective that is employed in most chapters of the current book. Many everyday experiences involve people who simply use and enjoy products. These everyday experiences are central to the book: The desire when seeing the new Apple iPod; the comfortable feel of a screwdriver in your hand; the awareness of the friendly whistle of a teapot; the frustration over the poor quality of an online help system; the delicious smell of a freshly baked apple-pie; the relief of smoothly parking your car in a narrow space. Understanding such experiences will allow designers and companies to ‘design for experience’.

Human–product interaction As argued at the beginning of this chapter, a product experience or ‘Erfahrung’ always results from some interaction the user has with a product. This interaction is not necessarily restricted to instrumental or non-instrumental (e.g. playing with your pen) physical action, but may also consist of passive (often visual) perception, or even remembering or thinking of a product (e.g. anticipating using your new stereo) (Desmet and Hekkert, 2007). Furthermore, the experience does not only result from the interaction, but also accompanies and guides the interaction, and thus affects the interaction. In sum, experience and interaction are fully intertwined and in order to explore people’s experiences of products, we need to thoroughly understand the constituents or building blocks of human–product interaction. Figure 1 provides a model of human–product interaction. In trying to obtain an overview of the literature covering the field of product experience, we got the impression that most scholars tend to approach human–product interaction from one of three possible perspectives. These three perspectives form the basic structure of the present book: The human beings with their systems and skills (Part I); the interaction itself with its different components (Part II); or a product (domain) with its specific properties (Part III). Independent from their surroundings and social context, humans are biologically equipped with a number of systems that make it possible for them to interact with their environment: A motor system to act upon the environment; sensory systems to perceive changes in the environment; and a cognitive system to make sense of the environment and to plan actions. Products are part of this environment. The motor capacities are needed to explore products, interact with them, and operate them. Sensory systems allow people to perceive a product and assess what kind of product it is. They provide feedback on people’s actions. Furthermore, they ‘tell’ a person whether a sensation (visual, auditory, tactual, olfactory, or gustatory) is pleasurable or should be avoided. Cognitive

HUMAN Motor system

PRODUCT

Motor skills

Sensory systems

Sensitivity

Cognitive system

Cognitive skills

Instincts

Concerns

INTERACTION

FIGURE 1 Model of human–product interaction.

Sensory properties

Structural properties

Possibilities for behavior

Composition

Functionality

Labels

Materials Technology

4

Product Experience

capacities link perceived information to stored knowledge to interpret the incoming information; they elicit memories of previous usage and evoke associations with other products. Finally, people are born with a limited set of instincts, which make them explore the world to try to satisfy basic needs. Through interaction with an environment, all these human capacities gradually develop into skills, expertise, and concerns (such as goals, intentions, and preferences). Skills and concerns can only be defined in relation to an outside world: We have skills in something, sensitivity for something, a preference for something, and our goals and intentions are directed towards something. The chapters in Part I of this book start out from one or more of these systems or skills, and try to explain experiential effects (and limitations) of human–product interaction on the basis of the specific qualities of these human capacities or characteristics. Analogously, a product perceived in isolation has (only) a number of structural or formal properties, such as its size, weight, and shape. Physical products are made of materials with specific technical characteristics (e.g. chemical constitution, heat conductivity, elasticity). Furthermore, products have a composition relating the various constituent parts to the whole. In addition, more and more products make use of embedded technology (electronics, sensors, and other digital components) to operate them. Finally, all kinds of labels (e.g. brand name, usage information, price information) may be attached to the product. It is, however, in the interaction with people that products obtain their meaning: On the basis of what is perceived sensorially (e.g. softness, freshness, loudness) products reveal cues of how to use them, and they reveal their function. Perceived product properties will only be of interest to an individual if they are somehow instrumental in fulfilling needs; only in relation to people we can determine what behaviour a product allows for, and what its primary or secondary functions might be. Studies that start out from the product (Part III) typically focus on one particular (set of) products and on how their product- or domain-specific characteristics affect people’s experiences. Often they include case studies of, for instance, how people experience household equipment, office interiors, or computer games. Product experiences depend on the way in which a person interacts with a product. Although phenomenologically experienced as a whole, at least three major components can be distinguished in product experiences (Desmet and Hekkert, 2007; Hekkert, 2006). The aesthetic response is characterized by feelings of (dis)pleasure that are based on the sensory perception of the object; the object looks beautiful, feels pleasurable, or sounds nice. In addition, people try to understand how a product must be operated or which actions it affords, and people attribute all kinds of expressive, semantic, symbolic, or other connotative meanings to it. The interactions with a product can help a person to reach a goal or can obstruct him or her in attaining that goal, and thereby lead to various emotional responses. Together these components shape the overall product experience. The chapters in Part II focus on one of these components of the experience, or they study a specific experiential state typified by a term like (dis)comfort, engagement, quality, or attachment. Finally, the way an interaction unfolds depends on the context in which this interaction takes place. This context can vary from the physical circumstances literally surrounding the interaction, such as the lighting conditions under which a product is perceived or the qualities of the space in which a meal is consumed, to the activities or experiences that have preceded the interaction at stake, and to the broader cultural and social situation that determines how people interact with and experience products (see e.g. Chapters 14, 24, and 26). It is clear that driving the same car may be experienced

Introducing product experience

5

completely differently when on holiday, in a hurry for an important meeting, testing it in order to make a purchase decision, or with the love of your life sitting next to you.

Empirical approaches to studying product experiences Given that product experience is defined as the awareness of the psychological effects elicited by the interaction with a product, it is not surprising that most research in this area typically assesses people’s subjective reports of their experiences with products. In the majority of studies, products or product parts are varied or manipulated under naturalistic conditions, and the effect of the manipulation on the subjective reports is assessed. In experimental studies the manipulations are typically done in a more or less systematic way, in order to isolate various underlying factors. In case studies and design projects, on the other hand, the manipulations are determined by wishes, demands, and limitations given by the product and its usage context, company goals, and designer capabilities. These two approaches yield complementary outcomes that both contribute to our knowledge on product experiences. The subjective reports may consist of either qualitative (e.g. in-depth interviews, diaries) or quantitative (e.g. responses on rating scales, preference rank orders) data. The ensemble of approaches used in these studies assures the ecological validity of the outcomes. However, subjective responses can only be obtained through introspection, and cannot be assessed objectively. Therefore, in several chapters self-reports are related to more objective measures, such as physical characteristics or instrumental measures of products, measures of information processing or decision making, consumer choice data, or psychophysiological measures and brain imaging data (see Chapters 1, 3, 18 and 20). Adding more objective measures and analyzing the relationships with subjective measures enables an internal validity-check on the subjective reports.

An overview of contributing scientific disciplines Research on product experience is situated at the intersection of several scientific (sub)disciplines. Because product experience research focuses on subjective experiences of people, all product experience research falls within the discipline of psychology. However, due to its multi-faceted nature, experience research crosses borders between several sub-disciplines of psychology that are usually distinguished, such as psychology of perception, cognitive psychology, and psychology of emotion. Acknowledging these different sub-disciplines, many questions can be asked concerning the interaction between a person and a product. How do people use their senses in experiencing products? How do people understand how to use a product? Why are people attracted to some products and not to others? On what grounds do people perceive a product as smart, solid, stupid, or splendid? Which memories, associations, and emotions does a product evoke? Why do people develop a bond with a product? These are the kinds of questions that will be addressed in this book. Although the contents of some chapters are clearly linked to one of these subdisciplines of psychology (e.g. Chapters 1–5 to perception; Chapters 7, 8, and 12 to cognition; and Chapters 15 and 16 to emotion), other chapters build on all sub-disciplines, and cover an applied field of knowledge. Most of these applied disciplines have their tradition in the social and behavioural sciences, such as psychological aesthetics, human factors, marketing, and consumer science. Others, however, have their roots in the technical sciences, such as mechanical and material engineering, and human–computer interaction (HCI). Analogously to how all these disciplines together contribute to and define

6

Product Experience

FIGURE 2 Disciplines contributing to the field of product experience.

the multi-disciplinary field of industrial or product design, they also converge in this relatively new domain of research: Product experience (see Figure 2). We now discuss each of these contributing disciplines briefly, and we indicate which (often new) areas or fields of study within each discipline ‘move’ into the emerging domain of product experience. Philosophers and psychologists have extensively studied people’s responses to works of art. From the field of philosophical aesthetics, the work of John Dewey has been most influential to the domain of product experience. In his ‘Art as Experience’, Dewey (1934) analyzes people’s involvement with artworks from a phenomenological perspective (see Chapters 9, 13, and 19). Ever since psychology was founded as a field of science, psychologists have had a keen interest in aesthetic experiences and evaluations of ‘objects’, such as simple patterns, faces, paintings, and landscapes (see e.g. Berlyne, 1971; Fechner, 1876). In their studies they generally looked for the principles governing people’s perception and appreciation of these manifestations by applying more general theories of perception, motivation, cognition, and emotion. More recently, psychologists have discovered products as an interesting subject area to investigate these aesthetic or pleasure principles (e.g. Crozier, 1994). Most of this work fits perfectly in the domain of product experience (see Chapters 10 and 11). The discipline of ergonomics or human factors traditionally focuses on the usability of products (in itself already an experiential goal). For a long time, the discipline limited itself to the perceptual and cognitive processes involved in product understanding, and to the physical or motor skills and processes enabling (or limiting) product use. In the current book, the perceptual systems and the way they operate in interaction with products are covered in Chapters 1–6, while the cognitive abilities and skills and their

Introducing product experience

7

effects on the interaction with products are covered in Chapters 6–8; human motor abilities and skills affecting interaction are predominantly discussed in Chapter 6. One of the ways in which products can be made easy to use is by making it self-evident how a product should be operated (see Chapter 12). However, this perspective has been expanded. At the end of the twentieth century the ergonomic discipline increasingly began to focus on other subjective experiences arising from the use of products, including research on satisfaction, pleasure (e.g. Jordan and Green, 2002), and comfort and convenience (Vink, 2005, see Chapter 18). Mechanical and material engineering have developed from having a singular focus on the technical/physical properties of artifacts and their effects on the durability, reliability, production, and (technical) performance of products, to studying, quantifying, and modeling the relationship between such properties and sensorial and other subjective responses in terms of meaning and aesthetics (e.g. Ashby and Johnson, 2002). This movement is most prominently seen in Kansei Engineering, a branch of engineering that started in Japan in the 1970s (see Chapter 20 for an extensive account of this approach). Technology-driven research focuses on how products can be created with new technologies that may be beneficial to potential users. The majority of this research consists of studies on the application of digital or smart technologies in human–computer interaction. Designers are interested in exploring new functionalities and interaction possibilities that can be created with these new technologies (see e.g. Moggridge, 2007). Because these technologies are applied in more and more everyday products, the findings obtained in the field of human–computer interaction become increasingly relevant for the entire product spectrum. In addition, within the HCI field we also see a shift from usability research to user experience research, variously looking at experiences such as presence, fun, trust, or engagement (e.g. Blythe, Overbeeke, Monk, and Wright, 2003). Various issues of user-product interaction with respect to digital technology are discussed in Chapters 7 and 8. In addition, the experience of functional interfaces (Chapter 21), intelligent products (Chapter 22), and computer games (Chapter 23) is extensively treated in Part III of this volume. The field of marketing studies how products find their way to customers. Traditionally, the marketer can make use of the four instruments in the marketing mix to bring the product to market in a profitable way: product, price, promotion, and distribution (Kotler, 1984). Research on product experience will typically be focused on the subjective evaluation of a physical product or service. This may concern a first encounter with the product in a store (Chapters 25 and 27), or a repeated encounter during product usage (Chapters 16 and 17). Also in the field of consumer research, research attention has shifted from information processing approaches with a focus on utilitarian value and price, to the emotional experiences associated with product consumption (e.g. Holbrook and Hirschman, 1982). In summary, the field of product experience research encompasses research from each of these disciplines. However, to fully understand human product experience, we need to use approaches that allow us to build bridges between these various fields of expertise. In our final ‘reflections’ we will further elaborate this view on the future of the research domain. People’s experiences with products are by definition subjective; probably no two experiences are alike. Individual differences in terms of, for instance, gender, age, expertise, and (cultural) background may account for the variety in people’s experiences, and some of these differences are discussed in this book. Finally, this book is not only about understanding product experience: We want to understand this phenomenon in order to contribute to ‘design for experience’. Hence, many authors address the design implications of their theories and findings, and they provide directions or guidelines to

8

Product Experience

increase the likelihood that a particular designed interaction will lead to the intended experience. Given that there are many scientific areas relevant for the study of product experiences, it is no surprise that the authors who contributed to this book differ considerably with respect to their scientific background and expertise. To preserve the coherence in the book, we have stimulated all authors to link their views and research to other related areas, to combine theoretical overviews with empirical studies, to provide concise descriptions of the methods they used, and to describe how their results may be used in design practice. It has proven to be a hard task for many authors to leave the field of expertise in which they feel comfortable, to explore largely unknown, but nonetheless fascinating, neighboring areas. This additional effort has made this book a unique collection of chapters that try to build bridges, for example, from neurophysiology and traditional psychophysics to product semantics, from universal aesthetic principles to design applications, and from human physical capabilities to product and consumption emotions. With this collection of chapters we hope to lay down the basics for the field of product experience research.

REFERENCES Ashby, M. and Johnson, K. (2002). Materials and design: the art and science of material selection in product design. Oxford, UK: Butterworth-Heinemann. Berlyne, D. E. (1971). Aesthetics and psychobiology. New York: Appleton-Century-Crofts. Blythe, M. A., Overbeeke, K., Monk, A. F. and Wright, P. C. (Eds.) (2003). Funology: from usability to enjoyment. Dordrecht: Kluwer Academic Publishers. Crozier, W. R. (1994). Manufactured pleasures: psychological responses to design. Manchester: Manchester University Press. Desmet, P. M. A. and Hekkert, P. (2007). Framework of product experience. International Journal of Design, 1, 57–66. Dewey, J. (1934). Art as experience. New York: Berkley Publishing Group. Fechner, G. T. (1876). Vorschule der Ästhetik. Leipzig: Breitkopf und Härtel. Hekkert, P. (2006). Design aesthetics: principles of pleasure in design. Psychology Science, 48, 157–172. Holbrook, M. B. and Hirschman, E. C. (1982). The experiential aspects of consumption: consumer fantasies, feelings, and fun. Journal of Consumer Research, 9, 132–140. Jordan, P. W. and Green, W. S. (2002). Pleasure with products: beyond usability. London: Taylor and Francis. Kotler, P. (1984). Marketing management: analysis, planning and control. London: Prentice-Hall. Moggridge, B. (2007). Designing interactions. Cambridge, MA: MIT Press. Pine, B. J. and Gilmore, J. H. (1998). Welcome to the experience economy. Harvard Business Review, 76(July/ August), 97–105. Russell, J. A. (2003). Core affect and the psychological construction of emotion. Psychological Review, 110, 145–172. Schifferstein, H. N. J. and Cleiren, M. P. H. D. (2005). Capturing product experiences: a split-modality approach. Acta Psychologica, 118, 293–318. Schmitt, B. (1999). Experiental marketing. Journal of Marketing Management, 15, 53–67. Vink, P. (2005). Comfort and design – Principles and good practice. Boca Raton: CRC Press.

1

ON THE VISUAL APPEARANCE OF OBJECTS HAROLD T. NEFS University of St. Andrews, Scotland, UK

1. ON VISUAL APPEARANCE 1.1. Introduction When looking around in the world, humans have to ﬁgure out what it is they see, what the shapes of things are, what kind of materials they are made of, and what their potential uses or dangers are. In addition to meaning, aesthetic value or affordance, the visual system also extracts some more fundamental object properties. Examples include the shape, size, glossiness, and lightness of an object. Counter-intuitively maybe, these aspects of the percept are not objective qualities because they are constructed in and by the human mind. Some of them have obvious correlates in the physical world, but this is often not the case. For example, objects might look ﬂatter when they are made from a different material (e.g. Khang, Koenderink and Kappers, 2003; 2004). The position, that perceived object shape is determined exclusively by the objects’ physical shape, is therefore clearly untenable. Likewise, perceived material and perceived illumination characteristics are also not determined exclusively by their respective physical counterparts. The problem of what objects look like is certainly not a trivial problem. It determines whether people approach an object carefully or not, if and how they pick it up, etc. In this chapter we examine the difﬁculties that humans face when looking at the real world. In particular we will examine the perceived shape, material properties, and illumination of objects with special emphasis on the perceived shape. A question that has inspired and frustrated many scholars over the last few millennia is why objects look the way they do. In the ancient Greek tradition it was thought Product Experience Copyright © 2008 Elsevier Ltd.

11

12

Product Experience

that objects send out exact little copies of themselves to the eyes where they are mysteriously inspected by the mind (Howard, 2002). Aristotle, however, rejected the idea that objects emitted substance to the eyes, and he clearly had the idea of the rectilinear propagation of light. Although this seems quite obvious from our modern vantage point, for a very long time it was not properly understood how image formation in the eye was accomplished. In our modern understanding we know that an image of the world is projected onto the back of the eyes where it is further processed and transported to the brain through the optic nerve. The visual image is a two-dimensional projection of the threedimensional world on the retina. In spite of the fact that the depth dimension is lost in the process, humans report a clear impression of a three-dimensional world. Somehow the brain reconstructs the world from what is only available in a degenerate two-dimensional projection. The main problem for vision lies in the fact that for any visual image there are an inﬁnite number of possible objects that give rise to the same image. This ambiguity not only lies in the fact that the depth dimension is lost in the projection onto the twodimensional retina, but also in the materials that the object is made of and how the object is illuminated. At ﬁrst sight, the reconstruction of the world in the mind is thus an impossible task. However, if the visual system were to use some assumptions about the world, and constraints in the way it reconstructs the world, the task becomes much more manageable. Some of the assumptions that one can make about the world are very powerful in constraining the number of possible solutions, while still maintaining a high predictive power in the physical world. For example, it is reasonable to assume that what is smoothly connected in the visual image is also smoothly connected in physical space. It is a highly coincidental situation indeed that two unconnected surface patches still form a smoothly connected patch in the visual image. It is more likely that an area of constant luminance, or a smoothly changing area, corresponds to a single, smoothly connected surface in physical space and that an area with a sharp transition indicates two separate surfaces. Obviously, the more assumptions one makes, the further the class of possible surface solutions is restricted. However, the chance that the reconstructed object is not accurate also increases with each additional assumption. Before we can begin to understand how objects are perceived, we need to know about the physical behavior of objects. How they reﬂect light, and what the descriptors of shape are that can be used. In other words, what kind of information about the depth dimension is preserved in the projection from a three-dimensional object to a two-dimensional retinal image? A pattern in the retinal image can only be an effective cue to the threedimensional shape if it has a causal relationship with the real shape of the physical object. In order to understand how the physical world is represented in the mind, one thus needs to study the physical properties of objects under projection. From our knowledge of the physical behavior of objects we might be able to derive potential cues for the visual system to derive a three-dimensional reconstruction of the world. The idea that the observer is inextricably linked with its environment was very fashionable at times in the twentieth century. Gibson (1966), for example, stated that an understanding of the physical properties of the world is essential for understanding perception. Once the physics of object properties is sufﬁciently clear for its present purpose, the question can be raised of what physical correlates perceived object properties have. It is clear that the perceived shape of things must correspond sufﬁciently reliably to the physical shape otherwise the percept would be quite useless. However, the physical shape in itself is not what the visual system processes. It rather processes light and dark distributions and their changes over time in the visual image. It is well known from the literature, and from everyday life as well, that shapes might look different if they are illuminated

On the visual appearance of objects

13

from a different direction (e.g. Nefs, Koenderink and Kappers, 2005) or when you paint them in a different color. In other words, we want to know what object properties make objects look big, three-dimensional, rough, shiny, etc. As will be clariﬁed later, there are physical relationships between shape, material and illumination once certain simple assumptions have been made. A further question emerges then. Namely, what is the internal coherence of the percept of an object as a whole? For example, one can come up with an algorithm to reconstruct the shape of an object from the shading pattern once an assumption is made on what the material and illumination conditions are. Changing these assumptions should change the reconstructed shape in systematic ways. However, is that actually the case? Does the visual system construct a coherent representation of the world, or are the percepts of shape, material and illumination essentially independent? The question that is raised is thus whether and how physical relationships between shape, material, and illumination are duplicated in perceptual relationships. In order to answer the questions that I raised above, I will ﬁrst give in Section 2 the necessary theoretical background on the physical properties and relationships between shape, material, and illumination of objects in projection. Secondly, in Section 3, the physical correlates for perceived object properties are discussed. Thirdly, in Section 4, I will discuss perceptual organization and whether perceived object properties conform to physical constraints; that is, whether they form a coherent set. In addition I will discuss how additional cues can be used to form a coherent percept.

2. THE PHYSICAL WORLD 2.1. Shape Before starting a discussion on perceived shape, it is useful to agree on a measure to determine the extent to which two different object shapes are different or the same. The ﬁrst step is to restrict shape to the ecological realm that can actually be seen. Every natural object is thoroughly fractured at the microscale and unstable, whereas at the macroscale it may be quite smooth and unchanging. The microscale is, however, not very interesting for visual perception. Likewise, the fact that the earth is round at the macroscale is not of any importance to perception since we are more concerned with smaller things. Given the restriction on the ecological realm, a transformational approach can be used to describe how two object shapes are different. For example, two objects might be the same in every way that can be discriminated by the spatial resolution of the visual system, except for their position or orientation in three-dimensional space. We can therefore say that these two objects are related by a translation and/or a rotation. All object shapes can be transformed into each other by some mathematical function. A prerequisite is that it is known which points on one object correspond to which points on the other object. Although this is not a trivial problem, let us for now assume that we do know. I think it is reasonable to say that two different sugar cubes that lie next to each other on a table and have the same global shapes are the same except for their position and orientation. That is, a sugar particle in the upper left corner of the ﬁrst object corresponds to a sugar particle on one of the corners of the second cube, within a reasonable spatial tolerance, but not to a random particle somewhere in the middle. At the end of the nineteenth century, the German mathematician Felix Klein proposed to order transformations according to the properties that they leave invariant. This ordering has become known as the Klein umbrella. The transformations that are grouped

14

Product Experience

under the Klein umbrella are not the only possible transformations one can make from one object to another, there are many more, less restrictive, geometries possible, such as a topology. However, the Klein transformations have some important properties that are highly relevant for perception. For example, they all preserve connectivity and smoothness. That is, all points on a surface that are smoothly connected to another point are still smoothly connected to the same point after transformation. Therefore, this kind of transformation does not tear a surface apart, nor does it make holes in it. Cederberg (2001) gives a relatively accessible text on modern geometries. The transformations in the Klein umbrella are illustrated in Figure 1.1. Although we do not do so here for illustration reasons, the images of Figure 1.1, and the deﬁnition in Equation 1.1, can be expanded into three spatial dimensions. In the remainder of the chapter we are interested in three-dimensional shape rather than two-dimensional shape. The most restrictive transformation is when two different objects are the same in every way except for their position. These two objects are related by a translation or a rotation. We say that these objects are isometries of each other. An isometry transformation preserves the absolute distance between all points in an object. Isometries are illustrated in Figure 1.1 in the left most column. A more general transformation than the isometry preserves only the ratios of distances between points of an object and some point in the ﬁeld. This is a called a similarity transformation. Dilation is a similarity transformation, but not an isometry. Note that all isometries are also similarities, because if absolute distances are preserved between all points, so are their ratios. Similarity transformations are illustrated in Figure 1.1 in the second column from the left. Storing a representation in memory in a similarity geometry instead of in a full Euclidean format has distinct advantages. For example, objects can be recognized irrespective of their size in either physical space or on the retina. That is, object representation and object recognition are size-invariant. The third group of transformations preserves parallelity but not necessarily position, distance or ratios of distances. All that is parallel in the original object is still parallel in Projectivity Affinity Similarity Isometry Translation

Rotation

Dilation

Strain

Collineation

Shear

FIGURE 1.1 The Klein umbrella. The black figure is in each case the original figure and the gray image is the transformed figure. From left to right isometries, similarities, affinities and projectivities are illustrated.

On the visual appearance of objects

15

the transformed object. These transformations are called afﬁnities. Straining and a shearing are both afﬁnities, but not similarities. Straining is increasing or decreasing the size of the object in one dimension only, whereas the dimension at right angles with the ﬁrst one remains unchanged. Shearing translates the points in the image in one dimension as a function of the second dimension. For example, the more you go to the right, the more the points are translated upwards. It is like holding on to left side of the object while pulling it up on the right side. Straining and shearing are illustrated in the third column from the left in Figure 1.1. All similarities are also afﬁnities, because they also preserve parallelity. The fourth group preserves collinearity. All points that are collinear, that is on a single straight line, in the original object are also collinear in the transformed object. Since areas of zero curvature have at least three collinear points, these areas thus remain invariant as well. These transformations do not add extra bumps or dimples on a surface. Transformations that preserve collinearity are called projectivities. All afﬁnities are also projectivities. An interesting property is that the cross-ratio of four distinct collinear points is invariant. The cross-ratio can be interpreted as the ratio of ratios of distances. More complex, but also invariant is a ﬁve point cross-ratio in which the points do not have to be collinear. Storing mental object representations in a projective geometry can have distinct advantages for object recognition. Objects that are stored in a projective format are invariant for changes in viewpoint. For example, the fact that the cross-ratio of any set of four collinear points, or the ﬁve-point cross-ratio, on an object remains invariant after a projective transformation can be exploited in machine vision for automated object recognition. The transformations that are shown in Figure 1.1 look complicated at ﬁrst glance, but they can be described by a simple matrix multiplication as in Equation 1.1. ⎡ xi ⎤ ⎡ a r u ⎤ ⎡ xi ⎤ ⎢ ⎥ ⎥ ⎢ ⎥ ⎢ ′ Equation 1.1: X ⫽ ⎢ yi ⎥ ⫽ A ⋅ X ⫽ ⎢ s b v ⎥ ⋅ ⎢ yi ⎥ ⎢ ⎥ ⎢ p q 1⎥ ⎢ ⎥ ⎢⎣ hi ⎥⎦ ⎥⎦ ⎢⎣ hi ⎥⎦ ⎢⎣ In this equation, X is the set of points of the ﬁrst object and X⬘ is the set of points on the transformed object. The points in the sets X and X⬘ have been put in the same order. That is, point Xi corresponds to point X⬘i. The two objects are two-dimensional, xi and yi are the coordinates of point i. We added a third dimension, h. The triplets {xi, yi, hi} are homogeneous coordinates. Cederberg (2001) describes the convenience and use of homogenous coordinates in more detail. The thing to remember about homogeneous coordinates is that all {xi, yi, hi} are equivalent to {xi/hi, yi/hi, 1}. The Matrix A is the transformation matrix. The parameters a and b strain the object, s and r shear the object in the image plane. Rotations can be obtained by speciﬁc combinations of a, b, s, and r. The parameters u and v translate the object in the image plane, and p and q shear the images in the extra dimension h; the latter induces a projective transformation in the image plane.

2.2. Material For the perception of material properties it is important to know how the reﬂection of objects can be described. The restriction is to the ecological realm that can actually be seen. Furthermore, objects that emit light themselves, such as lamps, or objects that store energy and emit it over a prolonged period of time, such as ‘glow-in-the-dark’ paint, are not considered here.

16

Product Experience

FIGURE 1.2 Lambert’s Law. Light beam B is spread out over a larger surface area than beam A. Therefore, the illumination per surface unit area is less in the case of beam B than in the case of beam A.

When light falls on a surface, it is reﬂected, absorbed, or passes through. The total amount of reﬂected light as a proportion of the light that falls on the surface is called the albedo or ‘whiteness’. Obviously an object cannot reﬂect more light than what falls on its surface. This has an interesting consequence. Namely, if light falls on a surface under an angle, the incident light is smeared over a larger area than when it hits the surface perpendicularly (see Figure 1.2). Therefore, the amount of reﬂected light per surface unit area depends on the angle of illumination relative to the surface. The apparent darkening of a surface depending on the illumination angle is called shading. Shading is a primary cue for the visual system for shape perception. The relationship between luminance and the angle of incidence is known as Lambert’s Law. Lambert’s Law can be expressed as in Equation 1.2: Equation 1.2: λ(n, i) ⫽ i ⋅ n ⫽ Iin * Cos(β ) In this equation, λ is the surface luminance. Luminance is thus the amount of light that is reﬂected from the object to the eye, whereas illuminance is the amount of light that shines on the object. The surface normal is denoted by n, and i is the vector describing the direction of illumination. The length of i is the illumination intensity Iin. The angle of incidence, that is the angle between the surface normal, n, and the direction of the incoming light, i, is denoted by β. These notions are illustrated in Figure 1.3. Each surface has its own reﬂection characteristics on top of Lambert’s Law. These characteristics are known as the Bidirectional Reflection Distribution Function, or BRDF. In general, the BRDF depends both on the angle of incidence and also on the viewing angle. The observed luminance can then be described as the product of the intensity of the illuminant, the dot product of the surface normal and the direction vector of the illuminant, and the BRDF as in Equation 1.3. Equation 1.3: λ(n, i, p) ⫽ i ⋅ n * BRDF(n, i, p) The vector p is the viewing direction. Nicodemus et al. (1977) give a good description of reﬂectance properties.

On the visual appearance of objects

17

FIGURE 1.3 The surface normal, n, is the vector that is perpendicular to the surface. The illumination direction is indicated with i. Vector p indicates the viewing direction. The angle α that is spanned between n and p is the viewing angle. The angle β that is spanned between n and i is the angle of incidence.

FIGURE 1.4 Computer renderings of a slightly bumpy sphere. The objects have, from left to right, a Lambertian, specular, backscatter and asperity BRDF. The two rows depict the same objects and BRDFs, but illuminated from different directions. The top row is illuminated from the viewpoint of the observer. The bottom row is illuminated from an area to the right of and above the observer.

In the special case that the BRDF equals one for all incidence and viewing angles, we say that the surface is a Lambertian surface. In that case, the viewing direction, p, does not appear on the right-hand side of Equation 1.3. Therefore, the luminance of a Lambertian surface is independent of the viewing direction. The incoming light is scattered from the surface in all directions irrespective of the angle of incidence. Many of the less-sophisticated computer graphics algorithms render objects using Lambertian surfaces since the luminance computation is easy, fast, and gives a reasonably realistic threedimensional impression. There are, however, no known materials that actually reﬂect light in a Lambertian fashion even though many materials come close. Figure 1.4 shows images of the same slightly bumpy sphere that are rendered with different BRDFs. From left to right in Figure 1.4 we used a Lambertian, specular, backscatter and asperity BRDF. A nearly Lambertian BRDF can, for example, be found on objects carved from chalk. In the case of specular reﬂection, incident light is mostly

18

Product Experience

reﬂected in the direction where the angle of incidence is mirrored in the surface normal (i.e. α ⫽ β in Figure 1.3). All mirrors, including Christmas balls, have a specular BRDF. Objects often show a combination of a Lambertian and specular BRDF. These objects have a Lambertian-like shading, but with added specular highlights, and are perceived as being ‘glossy’. Specular highlights are the bright spots on a glossy object where the illumination direction and the viewing direction are exactly mirrored in the surface normal. Examples of these BRDFs include glossy fruit, plastic, and porcelain. In the case of a backscatter BRDF, incident light is mostly reﬂected back into the direction of the light source. Backscatter BRDFs can be found on the retroreﬂectors on bicycles. Retroreﬂectors cast the light back towards the headlights of an approaching car. An asperity BRDF can frequently be found on materials with short hairs such as peaches and velvet. An asperity BRDF makes the areas near the contour of the object brighter. The two horizontal rows in Figure 1.4 depict the objects when illuminated from 0 and 45 degrees away from viewing direction. For illustration purposes, the illuminant is made to be stronger for the backscatter and asperity BRDFs than for the Lambertian and specular BRDFs (see also Pont and Te Pas, 2006). Some materials are partly translucent. That is, incoming light penetrates the surface and is scattered or reﬂected inside the object. This light is not absorbed, but eventually leaves the object again. A most-clear example is the reﬂectance of a diamond. The light that falls on a diamond penetrates the surface and is then reﬂected again and again in the facets inside the diamond until it ﬁnally ﬁnds a way out. The particular way a diamond is shaped causes the light to bounce so many times that the point where the light eventually leaves the diamond is unpredictable and highly unstable with small changes in the illumination direction. This gives a diamond its distinctive ﬁre. Another example of a translucent material is human skin. Incoming light is scattered below the surface through the skin layers. The translucency of the skin gives it its slightly reddish appearance and softens the edges of cast shadows on the body. I will not discuss translucency in this chapter any further. Nicodemus et al. (1977) describe an extended form of the BRDF that includes subsurface scattering as well.

2.3. Illumination A beam of light has intensity, direction and a wavelength spectrum. It always propagates in a straight line unless it is reﬂected. The illumination in a scene can all come from one direction, such as a scene that is directly illuminated by the sun. The illumination is in this case parallel. Common forms of illumination are illustrated in Figure 1.5. In the case of a spotlight or point light source that is relatively close to the scene, the illumination is convergent or divergent. Usually, however, there is light coming from multiple directions. Illumination that comes from all directions is called diffuse or ambient. In the case that illumination comes from only one half of all possible directions, it is called hemispherically diffuse. For example, in the case of an overcast sky, the majority of illumination is coming from all directions in the upper hemisphere (the sky) and very little from the ground. Of interest is to note that ambient light in combination with parallel light softens cast shadows. Parallel light gives hard cast shadows that hide many features of the object whereas it enhances surface imperfections (tiny bumps and dimples). Photographers often make use of ambient light from a relatively small range of directions with a ‘soft box’ or an umbrella to make the image esthetically more appealing. This kind of light is much more ‘forgiving’ of surface imperfections.

On the visual appearance of objects

19

FIGURE 1.5 Illumination conditions. Illumination can be parallel, divergent or convergent. Often light comes from all directions which is called ambient or diffuse. In the case light comes from only half of all possible directions, it is called hemispherically diffuse.

2.4. Color Of special interest in the appearance of objects is their color as it can set the objects apart from the environment and might signal their use or condition. For example, a slightly brownish tint on an apple signals that is rotten. Law-Smith et al. (2006) found that there are visual cues in female faces that are indicative of high estrogen levels and which are associated with ratings of femininity, attractiveness, and healthiness. Law-Smith et al.’s facial images suggest that a slightly reddish tint is one of those cues. Humans have four different types of photoreceptors that allow them to see light with wavelengths between roughly 350 nm (violet) and 750 nm (red). Three photoreceptors are associated with color vision whereas the fourth photoreceptor is associated with ‘night vision’. Wyszecki and Stiles (1982/2000) give an elaborate overview on color science, as well as a wealth of quantitative data on the subject. Objects reﬂect light for different wavelengths to different amounts and thereby create different colors. Elementary theory on color mixing can be found in any undergraduate textbook on sensory psychology. There are many systems available to order and describe colors. At the heart of all systems are the so-called CIE color matching functions. CIE stands for ‘Commission Internationale de l’Eclairage’ (International Committee on Illumination). The CIE color matching functions were ﬁrst reported in 1931 and have served since then as the standard deﬁnition of color, although other color matching functions have been measured as well. The color matching functions describe the intensities of three light sources with different spectra that, when they are combined, look the same as a light source of a single wavelength. Light that is composed of only a single wavelength is called monochromatic. Monochromatic light is, for example, used in, typically yellowish, sodium streetlights because they are particularly energy efﬁcient. Usually, natural light consists of a spectrum of different wavelengths.

20

Product Experience

The three light sources that were used to measure the CIE color matching functions were quite arbitrarily chosen. In fact one can perform the same procedure with three different light sources as long as they are sufﬁciently far apart in wavelength or color. It is relatively straightforward to calculate the correspondence between sets of color matching functions made with different light sources. I have plotted the CIE color matching functions in Figure 1.6A. The points of monochromatic lights form a continuous curve in the space spanned by the three light sources. A ‘color cone’ can be constructed by connecting the origin with all monochromatic points. All possible colors that can be perceived have a corresponding coordinate within the color cone. Usually, a cross-section is made through the color cone at which the intensity of the light is perceived as constant, as in Figure 1.6B. In Figure 1.6B all data points lay on a plane. This plane forms the CIE (x, y)-chromaticity diagram that might be familiar to the reader. Colors can be described with their coordinates in this plane, along with their intensity. Note that some colors, namely the purples, are not on the color-matching curve. There are no wavelengths that correspond to this color, because they can only result from a combination of a ‘reddish’ and a ‘bluish’ wavelength. Purples can be found on what is called the purple plane, the

B

A

Blueish

Blueish

Gr

ee

ish

dd Re Gre

nis

h

enis

dish

Red

h

C

h dis

Greenish

d

Re

FIGURE 1.6 Colors and object colors. (A) The color cone in the CIE (x, y, z) color space. (B) The positioning of CIE (x, y) chromaticity diagram in the color space. (C) All object colors in the CIE (x, y, z) color space that can be made with illuminant CIE-D65.

On the visual appearance of objects

21

ﬂat plane in Figure 1.6A, or purple line, the straight line in Figure 1.6B, in case of a chromaticity diagram. This plane closes the gap between the red side and blue side of the color cone. The chromaticity coordinates for any light source can easily be found by multiplying the color matching functions with the spectrum of the light source. The color matching functions can, for example, be obtained from the Color and Vision Laboratories of the Institute of Ophthalmology at the University College London (http://www.cvrl.org). It is possible that different spectra yield the same chromaticity coordinates. The colors of these light sources are thus the same. This phenomenon is known as metamerism. Usually, objects with metamer colors can easily be distinguished by looking at them through a colored ﬁlter or by shining light of a different spectrum on them. For example, a trick that is sometimes employed by the colorblind to distinguish red and green is to view them through a red ﬁlter. Red and green are metamer colors for the colorblind. Viewing a red object through a red lens does not change its intensity, but a green light that is viewed through a red ﬁlter turns to black. One particular shortcoming of the CIE (x,y)-chromaticity diagram is that it does not give a quantiﬁcation of how well two different colors can be discriminated. Furthermore, how far two different colors have to be apart in the chromaticity diagram in order to be seen as distinct depends on the color as well. Additional transformations can be made to rectify this problem, as is for example done in the CIE (x⬘,y⬘) chromaticity diagram and the CIE (L * u * v) and (L * a * b) color spaces. Not all colors can be made given a certain illumination spectrum. If there is no light in the red part of the spectrum, it is impossible to make an object look red, whatever its spectral reﬂection function. For example, the color of an object is very different in daylight than under the illumination of yellow streetlights. In fact, since sodium streetlights are monochromatic, there are no colors any more! All light that comes into the eye is of the same wavelength, irrespective of the material that has reﬂected it to the eye; only the intensity of the light differs. The casual observer might not even notice the absence of color under these conditions though, because there is still a lot of variation in light intensity. Given a certain illumination spectrum, it is possible to calculate all possible colors that can be made from shining this light on a surface. The resulting colors are called object colors. An object color is the color that the observer identiﬁes as the color of an object. Object colors can thus, depending on the spectral composition of the illumination, take on different values. The object colors for a given illumination spectrum ﬁll only a part of the color cone. The set of object colors thus ﬁlls a different volume, often called ‘color solid’, for each speciﬁc illumination spectrum. The volume is ﬁlled with all colors that can result from all possible reﬂectance functions and given a certain illumination spectrum. I have created such a color solid in Figure 1.6C. I have taken a standard illuminant, namely ‘CIE-D65’, and calculated all colors that can result under that illumination. The CIE has speciﬁed several standard illuminants for this purpose. There are many systems to order different object colors. I will brieﬂy mention one particularly popular model, namely the Munsell Color system. The Munsell system orders object colors on three dimensions namely: Luminance; hue; and saturation. Luminance is a value between black and white. Hue can be described as ‘red’, ‘blue’, ‘purple’, and so on. Saturation is the amount of hue as opposed to being gray. Pastel colors, for example, have a low saturation. The system is composed of a set of differently colored chips that can be arranged in a solid, much like in Figure 1.6C. The chips are chosen such that their discriminability increases in equal steps. The Munsell color system does not take into account how colors of materials change over viewing, illumination angle, and spectral composition of the light source. Calibrated chip sets for the Munsell are commercially

22

Product Experience

available. More detail on the CIE color matching functions, the Munsell color system, etc., can be found in Wyszecki and Stiles (1982/2000). Object colors as discussed so far do not form the entire story though. There are a few restrictions. First, the BRDF is dependent on the wavelength of illumination. A striking example is the Compact Disc. Light with the same angle of incidence but different wavelengths is reﬂected in different directions on a Compact Disc. A full description of the color of an object therefore involves the viewing and illumination directions as well. Secondly, the perceived color of objects also depends on the color of the environment. The chromaticity coordinates and the Munsell system is only valid when the object or light is viewed against a gray background. I will address this further in Section 3.2. Obviously, this has important consequences for the product designer, as the typical illumination and viewing conditions in which the product is going to be sold or used must be taken into account.

2.5. The SMI triangle It can be seen in Equation 1.3 that the luminance of a surface patch, as seen from the eye, depends on three components, namely (a) the surface normal; (b) the BRDF; and (c) the illumination. All that the visual system is presented with, however, is the object’s luminance in the direction from the surface to the eye. This gives rise to an interdependency of object properties which I will denote here as the Shape–Material–Illumination Triangle, or SMI triangle for short. The SMI triangle has two degrees of freedom. That is, you can choose any two vertices of the SMI triangle to be whatever you want, but the third one is then ﬁxed. When reconstructing the world, all three vertices of the SMI triangle are a priori unknown. This does not mean, however, that one cannot make an educated guess. However, if one wants to ﬁnd, for example, a solution for an object’s shape, one has to make assumptions on what the material and illumination conditions are. Ikeuchi and Horn (1981), for example, describe an iterative algorithm to reconstruct shape from shading. This algorithm assumes the BRDF and the illumination conditions to be known. One of the interesting questions that can be raised is what the resulting ambiguities are, given some reasonable assumptions about the real world. Let us ﬁrst assume that every pair of neighboring points that are smoothly connected in the image are also smoothly connected in three-dimensional space. Vice versa, every pair of neighboring points that are not smoothly connected in the image are also not smoothly connected in three-dimensional space, that is, connected but not smoothly, or not connected at all. Furthermore we assume that illumination and material are homogeneous over the object. These two assumptions already reject trivial solutions such as paintings on canvas or highly unlikely light sources on ﬂat surfaces such as computer beamer projections. Consequently, paintings and projections are perceived just as if they were three-dimensional scenes. The SMI triangle is best illustrated by looking at a speciﬁc example of an ambiguity in more detail. Although this example concerns the ambiguity of shape, similar examples can be made for illumination and material properties. One of the best-known ambiguities in constructing the surface of an object is the so-called bas-relief ambiguity, which translates from French as ‘low sculpture’. Bas-reliefs are relatively ﬂat sculptures that still give a very good impression of depth. Bas-reliefs can be found throughout the centuries. In spite of the fact that the bas-relief ambiguity has been known for a very long time, its mathematical structure and relationship with the illumination direction and intensity has been derived only recently (Belhumeur, Kriegman and Yuille, 1999). The bas-relief ambiguity is shown in Figure 1.7. The ambiguity lies in the fact that the same image can be obtained from objects that are scaled or sheared in the depth dimension when the illumination direction and intensity is changed at the same time.

On the visual appearance of objects

23

FIGURE 1.7 The bas-relief ambiguity. The top row shows an untransformed globular object from a frontal viewpoint, and the same object as seen from the top. The other objects show a transformed object with a transformed illumination direction. In these images, albedo is kept constant. The object in the second row from the top is sheared with a value of 0.3. The third row from the top is strained in depth in the direction towards the observer with a factor of 0.7. The bottom row shows the object when it is both strained (1.2) and sheared (⫺0.3). Despite the large differences in 3D shape, the images in the left column are nearly indistinguishable.

24

Product Experience

The angle of incidence can be expressed as the difference between the directions of the surface normal and the direction of illumination. An afﬁne transformation changes the direction of the surface normals in a linear manner. This gives us an intuitive understanding of the bas-relief ambiguity: If the surface normal is changed to the same extent as the illumination direction, the same luminance results and the images are therefore indistinguishable. Cast and body shadows are preserved exactly. However, in order to keep the shading pattern exactly the same in the image, a transformation in albedo is required as well. For example, when an object is ﬂattened the contrast in the image is lowered and the object is then typically perceived as being ﬂatter. In order to correct for this effect, the albedo of the surface has to be changed locally such that the surface becomes darker the more the surface normal turns away from the illumination direction. However, the necessity to transform the albedo becomes only apparent in very strong bas-reliefs. In Figure 1.7, I did not correct for albedo. Interestingly, introducing an ambient component in the illumination affects the contrast in the image as well; ambient light illuminates all parts equally irrespective of the surface normal, and therefore lowers the contrast within the image. Adding an ambient term can thus counterbalance the effects of straining and shearing. When an ambient component is added, the contrast in the image is reduced which leads to a ﬂattening effect and, when the ambient term is lowered, it leads to higher contrast and to an enhanced impression of depth in the object.

3. OBJECT APPEARANCE 3.1. Shape Several methods have been used in the past to measure the perceived shape of an object. These measures involve local depth comparisons, cross-sections, depth–width ratios, and surface attitude probes. For example, Koenderink, Van Doorn and Kappers (1992) introduced a popular method based on measuring the surface attitude. The method involves a surface attitude probe or a so-called ‘gauge ﬁgure’. The probe typically consists of a small circle with a little stick that is superimposed on the image. It resembles the shape of a pushpin. The aspect ratio and orientation of the gauge ﬁgure can be changed such that it looks as if the probe rotates in three-dimensional space. The probe is located on the image of an object and the observer sets the probe such that it looks like a small circle that is painted on the surface, with the little stick perpendicular to the surface. Across trials, the probe is placed at numerous points all over the image. The setting of the gauge ﬁgure is usually, but not necessarily, measured in slant and tilt. Slant is the angle away from the line of sight and tilt is the angle around the line of sight. Slant and tilt are illustrated in Figure 1.8. An example of the gauge ﬁgure task is shown in Figure 1.9. Figure 1.9A shows what the observer typically sees in an experiment during a single trial. The featured object in this case is an image of a globular virtual object that was created from a computer algorithm to simulate a three-dimensional shape with a glossy BRDF that was illuminated from a speciﬁc direction. Usually this kind of research is done on images of objects on a computer monitor in a darkened room rather than with real three-dimensional objects, for practical reasons. These images might be photographs taken of real objects or renderings of virtual objects that are created from a speciﬁc computer algorithm. In Figure 1.9B, I have accumulated all probe settings in one image. From the probe settings a surface can be recreated as in Figure 1.9C.

On the visual appearance of objects

25

FIGURE 1.8 Slant and tilt. A number of different circular patches are shown on a sphere. The slant of a patch is the angle at which the surface patch is turned away from the line of sight. Tilt is the angle around the line of sight. For example, patch A has a slant of 67 degrees and a tilt of 45 degrees. Patch B has a slant of 45 degrees and a tilt of 135 degrees.

FIGURE 1.9 The gauge figure task. (A) shows what the observer typically sees. The pushpin-like gauge figure can be manipulated by the observer and has to be oriented such that it looks like a circle painted on the object’s surface with the little stick as the outward surface normal. All observer settings are accumulated in (B). The settings are integrated into a ‘perceived’ surface in (C), for illustration purposes seen from a different viewing angle.

It should come as no surprise that the perceived surfaces of shaded objects show likeness with the ground truth up to a certain point. Human observers are also consistent in perceiving the three-dimensional shape of shaded objects, although perceived surfaces vary a lot in straining and shearing in depth when the procedure is repeated over different observers. Some observers set the gauge ﬁgures consistently to larger values than other observers. This might be interpreted as though these observers see more depth than others. However, it might also mean that observers map the aspect ratio of the gauge ﬁgure differently to a surface gradient in the mental world. There is somewhat less variance in straining and shearing when the procedure is repeated for the same observer over different sessions than when repeated over different observers. It has been found that most of the variance in perceived surface shape between observers or between tasks can be accounted for by afﬁne transformations (Koenderink et al., 2001).

26

Product Experience

We can now address the question of the effects of material and illumination conditions on perceived shape. A change in illumination direction can have signiﬁcant effects on perceived object shape. Curran and Johnston (1996), for example, found that the perceived curvature of spheres is affected by the position of the light source. Perceived curvature decreased as light source tilt (rotation in the image plane) increased and as light source slant (rotation away from the line of sight) decreased. Nefs, Koenderink and Kappers (2005; 2006) also found signiﬁcant effects of illumination direction on the perceived depth and on the overall attitude of the object. The effective cue seems to be the overall contrast in the image. Less contrast in the visual image seems to be interpreted as the object being ﬂatter. Increasing the level of ambient light makes the surfaces thus appear ﬂatter, since contrast is reduced. In addition to straining and shearing the perceived surface, a change in illumination direction also induces deformations in the perceived surface that cannot be described by afﬁne transformations. Koenderink, Van Doorn, Christou and Lappin (1996a,b) for example found systematic distortions of the perceived shape of human torso replicas with changes in illumination direction. For all observers in their study, those parts in the image that had a higher luminance were consistently interpreted as being nearer. Christou and Koenderink (1997) also concluded that even with images of simple ovoid shapes, subjects’ settings of surface attitude were systematically biased in the illumination direction, and were consistent with a regression to image luminance gradients. That is, dark parts are perceived as deeper and brighter parts of the objects are perceived as being closer. Nefs et al. (2005) also found that there are non-linear changes in perceived surface relief when the direction of illumination is changed. They described the change that was induced by changing the illumination direction as a shift in the position of the parabolic curves in the perceived surface. Parabolic curves are the sets of points on the object where the surface goes from being convex or concave to being saddle-shaped. In general, these points form closed curves on the surface. The shift in the convex, saddle-shaped and concave areas in Nefs et al.’s experiment is shown for one observer in Figure 1.10. The dark gray part represents a convex area, the black part is saddle-shaped and the light gray part is convex. The parabolic curves in the perceived surface reconstruction are located on the transitions from convex to saddle-shaped and from saddle-shaped to concave. The white jagged lines indicate where the parabolic curves of the physical shape are located. The knowledge of the effects of different BRDFs on perceived surface shape is somewhat less advanced. However, it is reasonable to conclude from Figure 1.4 that the BRDF clearly does have an effect. Most research has focused on the presence of specular highlights. Figure 1.9 shows an example of an object with three distinct specular highlights. There are good reasons for investigating specular highlights. First is a practical reason, specular highlights are easy to make in computer graphics and therefore easily available for researchers to play with. Secondly, specular highlights are not easily confused with Lambertian shading because of their high intensity. And thirdly, it has been argued that specular highlights are potentially informative of shape: Specular highlights are in general found on speciﬁc areas on an object (e.g. Longuet-Higgins, 1960; Oren and Nayar, 1996). Specular highlights have, for example, the tendency to cling to areas of high curvature and move rapidly over areas of low curvature when the illumination direction or the viewing direction is changed. This is also the reason why specular highlights follow predictable paths over the surface with changes in illumination or viewing direction. Specular highlights tend to move over the parabolic curves of an object. In the case of purely specular objects in complex environments we see a compression of the reﬂection of the world at areas where the object is strongly curved, and an expansion at areas of low curvature (e.g. Flemming, Dror and Adelson, 2003; Flemming, Torralba

On the visual appearance of objects

27

FIGURE 1.10 The top row shows the stimulus as used in Nefs et al. (2005). Observers made gauge figure judgments on these two images. In the bottom row, two diagrams are shown that indicate the convex, hyperbolic and concave areas in the perceived surface. The dark gray areas are convex, the black areas are hyperbolic and the light gray areas are concave. The jagged white lines show the positions of the parabolic curves of the real, physical shape are, which were the same for both objects. Reprinted by permission from Pion, London: Perception, 34, 275–287, copyright 2005.

and Adelson, 2004). For example, the reﬂection of the world in a Christmas ball makes everything look small, whereas the same world viewed in a ﬂat mirror appears to have its original size. On objects that have both ﬂat and curved areas, the world is thus compressed to a different extent depending on the surface curvature. The scientiﬁc evidence that the addition of specular highlights to otherwise Lambertian reﬂecting objects induces differences in perceived surface shape is contradictory though. On the one hand, there are studies that support an effect of specular highlights. Todd and Mingolla (1983) reported that the presence of specular highlights on cylindrical surfaces enhances the perception of surface curvature. Norman, Todd and Orban (2004) showed that shape discrimination was improved when specular highlights were added. Todd, Norman, Koenderink and Kappers (1997) reported that observers’ judgments were signiﬁcantly more reliable when specular highlights were added to Lambertian surfaces when objects were viewed stereoscopically. Blake and Bülthoff (1990; 1991) showed that specular highlights can be used by observers to disambiguate convex/concave surfaces in stereoscopic vision. On the other hand, there are studies that do not report signiﬁcant effects of the presence of specular highlights. Mingolla and Todd (1986) found that changing the illumination direction inﬂuenced the judged shape of ellipsoid surfaces, but the presence or absence of specular highlights did not have a signiﬁcant effect. Nefs et al. (2006)

28

Product Experience

also reported effects of illumination, but not of specular highlights, on shape perception for a range of globular objects. Norman, Todd and Phillips (1995) reported that observer settings were no less accurate for objects containing just specular highlights than for Lambertian objects or objects having both Lambertian shading and specular highlights. There are a couple of explanations for the different conclusions in the literature. An important reason is probably the speciﬁc shape of the objects that are used. For example, in the case of cylinders, specular highlights form a straight line over the cylinder. And in the case of an object with a large curvature contrast, the specular highlights may trace ridges and parabolic curves over the object. Khang, Koenderink and Kappers (2004) looked into the effects of Lambertian, specular, asperity, and backscatter BRDFs on perceived surface shape. They found that spheres and ellipsoids that were rendered with asperity or backscatter BRDFs were perceived as being ﬂatter than those rendered with a Lambertian or specular BRDF. Hue has no measurable effect on perceived shape (e.g. Claessen, 1996). Troscianko et al. (1991) also showed that iso-luminant gradients from red to green do not affect the perceived slant of a surface. An iso-luminant gradient from red to gray (saturation), however, did have an effect. There is a small interesting effect of hue on the perception of depth though when an object is viewed with two eyes. This effect is best seen when a red object is displayed in front of a blue background. The red part is sometimes seen to stand in front of the blue background. This effect is known as chromostereopsis. It is believed that this effect is caused by the different refraction indices for red and blue light in the cornea of the eye, which results in a binocular disparity between the red and blue parts of the image. Usually the effect is small though and has little consequence for visual perception. You can buy special ‘ChromaDepth™ 3-D’ glasses that enhance chromostereopsis by separating the refraction indices for red and blue further (Chromatek®).

3.2. Material A reduction in illumination makes an object on a gray background look as if it has a lower albedo when the luminance of the background is kept constant. If the background luminance is reduced, that is the contrast between the object and the background remains constant, the perceived albedo of the object is fairly constant over a large range. This is known as luminance constancy. Likewise there is color constancy. The color of an object does not look much different in the evening compared to midday, especially for the colors that have a low saturation. Nevertheless, the spectrum of light entering the eye at the beginning of the evening is quite different from the spectrum at midday. If the evening and midday colors of the object would be shown simultaneously against a gray background, the difference would be striking. What is important for perception is not so much the light intensity or its spectrum, but the luminance or color contrast with the environment. In many ways the human observer has no respect for absolute intensities, perception is foremost concerned with relative intensities. This principle comes back in many different forms in sensory psychology. For example, in luminance and color contrast illusions, but also in motion perception as in the Duncker illusion. The Duncker illusion is best illustrated by the apparent motion of the moon rushing against the moving clouds. Three contrast illusions, namely luminance, curvature and size, are illustrated in Figure 1.11. When predicting or evaluating the appearance of objects in terms of albedo, color, size, motion etc., it is thus advisable to take the environment into account. The effects of luminance constancy and color constancy have also been described elsewhere (e.g. Palmer, 1999). A large part of the literature on material perception is devoted to the perception of lightness/albedo. Todd, Norman and Mingolla (2004) investigated the lightness of objects

29

On the visual appearance of objects

(A)

Luminance contrast

(B)

Curvature contrast

(C)

Size contrast

FIGURE 1.11 Contrast illusions. (A) Luminance contrast. The gray patch in the white circle is perceived as being darker than the gray patch in the black circle, even though they have the same gray value. (B) Curvature contrast. The physically straight line appears to bend. (C) The black disk is perceived as larger when it is surrounded by small circles than when it is surrounded by large circles, although the physical size is the same.

as a function of the presence of specular highlights. Lightness is here deﬁned as the fraction of the amount of reﬂected light that is reﬂected in a Lambertian manner over the total amount of incident light. Todd et al. concluded that observers were quite able to discount the luminance intensity of specular highlights when judging lightness. An implication of this is that a mere glossy ﬁnish of a product does not change the perceived albedo of the underlying material. That is, objects do not look whiter (as opposed to gray) with a layer of varnish. Khang et al. (2003) investigated whether the perception of surface albedo is inﬂuenced by the mode of lighting in 3D geometrical shapes. They rotated a Platonic solid (in this a case a cube (six faces), a dodecahedron (12 faces) or an icosahedron (20 faces)) on a computer monitor. One of the faces of the solid was rendered with an albedo different from the other faces. Observers had to indicate whether there was a face with a different albedo from the other faces present in the object or not. It was found that observers performed best when the illumination was ambient, then hemispherically diffuse, and they performed worst when the illumination was parallel. It was also found that the number of faces that could be seen at any time was important. Observers performed best with the icosahedron, then with the dodecahedron, and they performed the least well with the cube. The symmetry of the object was found not to be an important variable in identifying the faces of polyhedrons with different albedo. The identiﬁcation accuracy was the same whether the polyhedron was regular or irregular. Of practical use is that parallel light can hide slight variations in albedo better than other illumination modes. In case the aim is to detect imperfections of albedo on the surface it is best to use an ambient illumination.

3.3. Illumination Pentland (1982) proposed that the direction of illumination in smoothly shaded convex objects could be estimated from the ﬁrst-order derivative of the image intensity and demonstrated that estimates of the illumination direction that are based on this idea correspond

30

Product Experience

well with estimates made by human observers. This study shows that observers have an idea where the light in a scene is coming from, which might in turn be important for shape and material perception. Khang et al. (2006) investigated the effects of material on perceived illumination direction. They showed that the type of BRDF inﬂuenced the accuracy of illumination settings. They concluded that observers might not take the effects of BRDF into account when judging the illumination direction, despite the large changes in the image intensity that it brings about. It has been suggested that observers are sensitive to the color of specular highlights (Yang and Maloney, 2001). Although not strictly true for all materials, the spectral composition of specular highlights is for many materials more similar to the light source than to the spectral composition of Lambertian shading of the object. This effect is often seen for varnished or wet materials. For example, when shining a white light on a glossy red plastic ball, there will be a white, rather than a red, specular highlight on the ball. On the other hand, the specular reﬂection of metals, such as copper and gold, is not color neutral to the light source; the specular reﬂection of gold is distinctly yellow for white illumination.

4. PERCEPTION 4.1. Perceptual organization The visual system is not a passive machine, but actively tries to make sense of the surrounding world. It reconstructs the world on the basis of the visual input into meaningful units. Gibson (1979) formulates this notion as: ‘Perceiving is an achievement of the individual, not an appearance in the theatre of his consciousness’. Several organizing principles have been identiﬁed in the past and are now commonly known as the Gestalt Laws. Central to Gestalt theory is the idea of ‘Prägnanz’. The Law of Prägnanz can be formulated as: ‘The psychological organization will be as “good” as the prevailing conditions allow’ (Koffka, 1935). The objective of the visual system is to achieve an organization that is as simple and regular as possible. In the context of shape, it means that a visual image is organized into the simplest and most regular shape possible given the visual image. This is a rather vague deﬁnition, because what exactly deﬁnes ‘good’ and what is a ‘good shape’? Several factors, or laws, have been described that contribute to perceptual organization, or to a good shape. The most important factors are: Good continuation; Proximity; Equality; Closure; Symmetry; and Common fate. The Gestalt factors are illustrated in Figure 1.12. Good continuation means that curves and surfaces are perceptually continued when they appear interrupted. The visual system identiﬁes curves and surfaces that are good continuations of each other and combines them into a single unit. The factor of proximity means that points, lines, etc. that are close to each other are grouped together. Equality refers to the tendency of the visual system to group parts of the visual system together that share some feature, such as color or shape. Closure means that parts of the image that form a closed area or volume tend to be seen as such. Symmetry in this context means a ﬁgure is the same as its mirror image. Finally, dot, lines, etc. that move in a common direction and speed are seen together. This is known as common fate. Although we did not go into that in this chapter since all our objects are stationary, structure-from-motion, that is common fate, is often an important cue for shape perception. For example, a well-camouﬂaged bird may go unnoticed as long as it keeps perfectly still. As soon as the bird starts to move,

On the visual appearance of objects

31

FIGURE 1.12 The Gestalt factors: Good continuation, Proximity, Equality, Closure, Symmetry and Common fate. The arrows in the bottom right picture indicate a motion in that direction.

camouﬂage is broken, because the observer sees a coherent motion in the bush. A fact that is generally underappreciated in textbooks, but which should be mentioned for good measure, is that many demonstrations of the Gestalt laws in the literature are subject to the time that is given to view them. Like any other process, perceptual organization takes time. The more time that is given to view the scene, the more complex and detailed the resulting organization may become. Irregularities that are at ﬁrst overlooked are incorporated at later stages. A fairly irregular shape may be perceived as regular when it is shown very brieﬂy, but at longer durations the irregularities become clear. The Gestalt principles are highly important in visual perception since the visual system tries to order the world accordingly. Sometimes however, they lead to incorrect or bistable interpretations. Many visual illusions are therefore associated with the Gestalt laws. A classic example of bistability is the well-known Rubin face/vase illusion where the perceptual organization into a foreground and a background switches between two states. In this illusion the observer ﬂips between a percept of two faces or a vase in the foreground. More interesting, artistic illustrations of bistability than the face/vase illusion can be found in the ‘Slave market with a disappearing bust of Voltaire’ by Salvador Dali, and in ‘Vanity’ by Charles Allen Gilbert. ‘Vanity’ was recently given new life in an advertisement for Dior that won an award for best European advertising photography in the Epica 2002 contest. Anderson and Winamer (2005) recently published one of the most fascinating new illusions. They showed that because of organizing an image into a foreground and background, the lightness of the depicted objects is strongly modiﬁed. This illusion is shown in Figure 1.13. The discs on the right of Figure 1.13A are perceived as black, while partly occluded by a white mist. The discs on the left of Figure 1.13A are perceived as white, while occluded by a dark fog. The images of all discs are exactly the same though. Likewise, the images of the two sets of chess pieces in Figure 1.13B are exactly the same, despite their strikingly different appearance in terms of albedo. Anderson and Winamer showed that this illusion is not a mere luminance contrast illusion. They showed this by

32

Product Experience

FIGURE 1.13 An illustration of the Anderson and Winamer (2005) illusion. In (A) the discs in the two images are exactly the same; in (B) the two sets of chess pieces are identical. Reprinted by permission from Macmillan: Nature, 434, 79–83, copyright 2005.

rotating the ‘mist’ over 90 degrees. This manipulation destroys the illusion in spite of the fact that the contrast in the image is unchanged. One of the problems with Gestalt theory is to quantify the contribution of each Gestalt law and to describe exactly what the contribution means. What exactly is a good shape and why is a square a better shape than a rectangle? The key is believed to be the simplicity of the resulting organization: The simpler the organization, the better. Simplicity can be described as the number of information bits that are necessary to describe the scene. Koffka (1935) gives an early example of this idea. Koffka suggested that a circle is more likely to be seen as a mere line than a triangle, because every element of the circle contains the principle of the whole. That is not the case for the triangle. That is, it requires more bits of information to describe a triangle than a circle. Yet another way to put it is that the circle has a higher level of symmetry than a triangle. Several coding systems have been proposed in order to evaluate the simplicity of the structure (e.g. Leeuwenberg, 1971; Van der Helm and Leeuwenberg, 1996).

On the visual appearance of objects

33

Important concepts in these approaches are regularity and symmetry. The effects of different kinds of symmetry in Same/Different tasks have, for example, been investigated by Leeuwenberg and Van Lier (2005). In addition to geometrical considerations for perceptual organization it may be argued that the visual system makes assumptions on what is statistically likely in the real world. For example, the majority of light usually comes from above, rather than from the ﬂoor. It makes sense to be biased towards light from above if the situation is ambiguous. In addition, Hill and Bruce (1993; 1994) found that people have a preference to see objects as convex rather than concave. It is often unclear whether a perceptual bias is based on geometric considerations, or on statistical likelihood. Regularity in the visual image, for example, is a very unlikely coincidence if it does not exist in three-dimensional space as well. Consider for example a sphere. From any viewpoint the sphere will appear as a circle in the visual image; that is, spheres yield stable projections. Ellipsoids (egg-shaped objects) project in general to ellipses in the visual image but in accidental viewpoints they project as circles; these projections are unstable. That is, if you see a circle it is more likely projected from a sphere than from an ovoid. A circle in the visual image is just too good to be incidental. Gibson (1979) takes the idea of perceptual organization one step further in his theory of information pick-up. According to Gibson not perceptual qualities, such as color and intensity, are perceived, but rather the information that is provided in their changes over space and time. Mere perception of color or curvature is unsatisfactory to Gibson. True perception is all about seeing the structure in the variation of the world and the structures that remain invariant over time. Gibson suggested that perception is thus not concerned with positions but with places, with objects rather than 2D-shapes and with substances (materials) rather than luminance. This is very much what we are after when discussing the appearance of objects: Object appearance is not about the perception of luminance, it is all about inferred shape, albedo, BRDF, illumination, etc. of the object. What observers are often particularly interested in is what you can do with an object. Observers speciﬁcally look for information in the visual image that allows them to infer what it is they can do with an object. A prominent feature in Gibson’s theory is therefore the inferred function, called affordance, of an object. The affordance of an object translates roughly into ‘this object is for … (raking, hammering, throwing, etc.)’. The affordance of an object is not only a property of the object, but also of the observer. For example, a big tree trunk does not afford throwing for a child, but it does for an adult interested in caber tossing.

4.2. The SMI triangle Koffka (1935) writes that prior to his time it was believed that the visual system could extract properties of the visual image independently of each other. For example, color could be extracted independently from motion. Koffka himself, however, believed that this was impossible. More recently, however, evidence has been published that suggests that his position is not quite tenable. The mere fact that you need some light in order to make a pattern move over the retina does not mean that you cannot perceive motion without being able to see light. There are, for example, patients who can see light but not movement (e.g. Zihl, Voncramon and Mai, 1983), and there are also cortically blind patients with relatively intact motion perception (e.g. Azzopardi and Cowey, 2001). Although evidence from patient studies has several problems, it suggests that the coupling of motion and light perception is not a simple fact. The problem of whether perceived object qualities have an inherent relationship goes one step further. These object

34

Product Experience

properties are not qualities of the visual image, but of the representation that is created from that. There are good reasons to expect a relationship between object properties just as there are good reasons not to expect one. A good reason for perceived properties to be coupled is of course the SMI triangle. Gregory (1973) showed a relationship between perceived convexity and illumination direction. He showed that both a mask and a mould of a face look like convex faces, but that the light seems to come from a different direction. That is, a convex mask that is illuminated from above looks like a convex face that is illuminated from above, whereas a concave mould of a face that is illuminated from above looks like a convex face illuminated from below. Johnston, Hill and Carmen (1992) showed that this also holds for faces turned upside down and for hollow potatoes as well. In these cases illumination and shape perception seem to be coupled. It is a common mistake to consider the perceived illumination direction as the reason why the hollow face is seen as convex. The more likely reason is that there is a prior for convexity (Langer and Bülthoff, 2001). Note also that the hollow face illusion is never bistable. Mingolla and Todd (1986) also showed a correlation between perceived shape and perceived illumination direction when observers viewed relatively ﬂat egg-like objects, but not when they saw egg-like objects of greater depth. Te Pas and Pont (2005) found that the perception of BRDF and illumination on spherical shapes was in many cases confounded. In many of the objects that they tested, a change in BRDF was instead perceived as a change in illumination conditions. They did not report whether the perception of BRDF and illumination were correlated on a trial-by-trial basis. Interestingly, when the task was to compare the BRDF of two objects, each with a different BRDF but illuminated the same, observers consistently reported the BRDF to be different. When the task was about the illumination direction, and the objects differed in illumination but the BRDF was the same, observers were less consistent in reporting the illumination to be different. This suggests that the processes underlying material and illumination perception are at least partly decoupled.

4.3. Additional cues Apart from shading there are usually more cues to shape, material and illumination available that are largely independent of the SMI triangle. The shape of the object in the visual image is a very good cue for three-dimensional shape. For example, Mamassian and Kersten (1996) concluded that observers used the occluding contour of the object rather than shading to estimate the local surface orientation of croissant-shaped objects. The occluding contour is where the object curves around to occlude itself from sight. That is, the occluding contour is the edge of the object’s silhouette. Erens, Kappers and Koenderink (1993) showed that the presence of an occluding contour is necessary for the perception of shape-from-shading. The absence of a well-deﬁned contour renders the sign of the surface shape (convex/concave) essentially ambiguous. Furthermore, even in images of objects that lack any shading, such as silhouettes, there are sufﬁcient cues to the three-dimensional structure, such as the occluding contour, in order to get to a reasonably accurate three-dimensional impression. Tse (2002), for example, argues convincingly that the three-dimensional shape of objects can be quite accurately predicted when unlikely combinations of object shape and viewpoints are rejected. Several authors have shown that the perception of material properties is inﬂuenced by perceived shape. The perception of shape is in these cases induced by contours and suggestive groupings of surfaces. Knill and Kersten (1991) showed that perceived surface curvature inﬂuences perceived albedo. Perceptions of surface attitude and curvature

On the visual appearance of objects

35

shape were induced by suggestive contours. Gilchrist (1971) showed that perceived lightness depends on the relationship of the luminance of surfaces that lie in the same depth plane, but not on surfaces that are merely adjacent in the retinal image. Soranzo and Agostini (2006) showed that increasing perceptual belongingness of surface patches into larger units improved the perception of lightness constancy. Object surface texture deforms in speciﬁc ways, depending on the surface attitude relative to the viewing direction. Also, because of linear perspective, the density of the texture increases with increasing distance. Note that the texture in the visual image strains in one direction because of a change in surface attitude of the object, but shows dilation, when the distance is changed. Here we make the assumption that texture is uniform over the object and without a speciﬁc direction, but there is a dependency of texture and shape, much like we have seen for shape, material, and illumination. Curran and Johnston (1996) have shown that the effect that the position of the light source has on perceived shape is reduced when texture cues are added. Pont and Koenderink (2003) showed that the direction of illumination is accurately predicted from the direction of the texture in the visual image. That is, bumps and pits in the surface create systematic distortions in the texture in the visual image, because they cast small shadows on the surface that tend to elongate in the direction of illumination. They also showed that illumination direction could be estimated in a similar way in complex scenes. Koenderink et al. (2003) showed that observers are good at estimating the tilt of the illumination direction, but are less proﬁcient at estimating the slant of the illumination direction. Te Pas and Pont (2005) reported also that observers are more reliable in their estimation of illumination direction when three-dimensional texture was present on the object. Three-dimensional texture did not prove helpful in discriminating objects with different BRDFs though. The effects of three-dimensional texture on shape perception have not been evaluated systematically yet. In cases where a higher accuracy of illumination perception is required it makes good sense to add three-dimensional texture to the objects in the scene. For example, when one wants to use the information on the illumination direction to estimate shape and material properties of other nearby objects that are not textured but are only shaded. Stereoscopic vision is another frequently cited cue to three-dimensional shape. Because the two eyes look at the world from slightly different viewpoints, the images in the two eyes are slightly shifted relative to each other. This shift is known as binocular disparity. In general, however, it is not the solution to unambiguous perception. First, binocular disparity works best only in a limited range of distances, and secondly, there is a considerable part (about 30%) of the general population that does not have the stereoscopic resolution that one would wish for (Richards, 1970). Thirdly, there is considerable uncertainty in the perceived distance of the object, which causes unpredictable effects on depth scaling (e.g. Drga and Harris, 2005). And fourthly, stereoscopic vision works only for relatively high spatial frequencies such as edges, lines, texture, etc. For example, Arndt, Mallot and Bülthoff (1995) showed that a reversal of the left and right half images of a stereoscopic image of a shaded sphere does not turn a convex sphere into a concave bowl. Only when texture is added does the sphere turn inside out, but still only reluctantly. Apparently, the prior for convexity is much stronger than the cue provided by stereoscopic viewing of shading. Several models have been proposed of how different cues can be integrated to resolve ambiguities in shape perception (e.g. Landy et al., 1995). Cue accumulation models do not in general take material and illumination ambiguities into account. They do not tell whether, for example, shape perception becomes more stable when cues to illumination are speciﬁed. There are no formal models that incorporate all these cues. The question of

36

Product Experience

whether reduction of ambiguity in one of the vertices of the SMI triangle leads to more perceptual stability in the other two vertices, and how that might best be achieved is still largely an open question, though with obvious consequences for disambiguating environments where shape and material information of objects is crucial for picking them up in the correct way, manipulating or recognizing them. For example, conditions that allow for making a good estimate of the BRDF might provide the observer with a clue to the type of material an object is made from, such as metal or cloth, which determines how hard you can squeeze it when picking it up. Other important applications are in virtual or augmented reality applications where the number of visual cues is often limited, and as a result shape or material might be become ambiguous.

5. CONCLUSION In this chapter I have described the roles of shape, material, and illumination for object appearance at three different levels, namely: At the physical level, the relationship between the physical world and the mental world, and ﬁnally at the level of the mind. First I discussed the physics and mathematics of shape, material, illumination and color. I made clear that there are inherent ambiguities in visual object perception. For example, the color of an object is not only determined by its reﬂectance function, but also by its illumination. This is important in conditions where objects have to stand out of the environment irrespective of illumination, such as trafﬁc signs or the clothes of road workers. Trafﬁc signs have to be clearly visible, also at night when the only illumination comes from the sodium streetlights that we discussed earlier. A retro-reﬂective property of the trafﬁc sign or the clothes of road workers that reﬂects the headlights of cars back to the source, as well as takes advantage of the more complex light spectrum, is clearly a desirable property at night. Another area in which metamerism is important is where objects have to blend in with the environment, such as in military camouﬂage. It would make sense to be well aware of the possibility that metamere colors are not metamere under different illumination conditions, when the observer uses a color ﬁlter, or even when the observer looks from a different direction. I have also illustrated that ambiguities of shape must fall within a certain class once some assumptions have been made. For example, we have seen how the bas-relief ambiguity is restricted to an afﬁne class of shapes. In the second part, I discussed how physical parameters affect perception. I discussed a method to measure perceived shape. Perceived shape can be signiﬁcantly altered by changes in material and illumination. An important factor in the perception of material, color and albedo, is contrast. The visual system operates mostly on contrast in luminance, color, etc., rather than its absolute value. For example, when a product needs to draw attention, it is better to increase the contrast in luminance or in the density of the surface texture or in the color, of the product with the environment rather than increasing the overall illumination levels. The luminance, color, etc. of the environment must therefore always be taken into account if one wants to predict what a product will look like, especially where it is important that objects are presented favorably, such as in shop windows and on supermarket shelves. Contrast is also an important factor for improving visibility of products for people with low vision. Rather than increasing the overall luminance, although that might also be beneﬁcial in many cases, increasing the color contrast or luminance contrast with the environment will make the product more visible. In the third part, I discussed how the visual system tries to organize the visual image into a sensible impression of the world. The Gestalt laws describe several factors that

On the visual appearance of objects

37

contribute to a perceptual organization. Proper structuring of the environment can disambiguate perception, overcome luminance contrast illusions, or just as well result in a visual illusion. In the design situation, ambiguity in the perceptual organization might, for example, be exploited in order to create an interesting artistic product, but the designer might also want to avoid ambiguity as much as possible depending on the application in mind. I discussed how the perception of shape, material and illumination might be coupled. This is important because it is a potential way to disambiguate the visual scene. It might be possible to make the material that an object is made of less ambiguous by providing cues to the shape, such as a clearly visible contour and surface texture. Or, observers might be able to recognize that an object is less ﬂat when it is clear to them that the illumination is highly ambient. For example, in interactive virtual reality or augmented reality applications where accurate perception of depth and distance is often required, the presence of visual cues is often limited which may result in a high level of ambiguity under these conditions. However, the ambiguity might be resolved by adding more visual cues. Adding additional independent visual cues to shape, material and illumination such as surface texture and 3D texture, binocular disparity, specular highlights, sharp contours, coherent motion, and regularity in the spatial layout, either in a real or a virtual environment, may all contribute to a more accurate visual perception of the product.

ACKNOWLEDGMENTS I wish to thank the members of the Vision Lab at the University of St. Andrews, and Samira Bouzit of the School of Psychology at the University of Southampton for providing invaluable feedback on the draft version of this chapter.

REFERENCES Anderson, B. L. and Winamer, J. (2005). Image segmentation and lightness perception. Nature, 434, 79–83. Arndt, P. A., Mallot, H. A. and Bülthoff, H. H. (1995). Human stereovision without localized image features. Biological Cybernetics, 72, 279–293. Azzopardi, P. and Cowey, A. (2001). Motion discrimination in cortically blind patients. Brain, 124, 30–46. Belhumeur, P. N., Kriegman, D. J. and Yuille, A. L. (1999). The bas-relief ambiguity. International Journal of Computer Vision, 35(1), 33–44. Blake, A. and Bülthoff, H. H. (1990). Does the brain know the physics of specular reﬂection? Nature, 343, 165–168. Blake, A. and Bülthoff, H. H. (1991). Shape from specularity: Computations and psychophysics. Philosophical Transactions of the Royal Society of London: Series B: Biological Sciences, 331(1260), 237–252. Cederberg, J. N. (2001). A course in modern geometries. NY: Springer. Christou, C. G. and Koenderink, J. J. (1997). Light source dependency in shape from shading. Vision Research, 37(11), 1441–1449. Claessen, J. P. (1996). Shaped by colour. Doctoral thesis, University of Delft, The Netherlands. Curran, W. and Johnston, A. (1996). The effect of illuminant position on perceived curvature. Vision Research, 36(10), 1399–1410. Drga, V. F. and Harris, J. M. (2005). Distance perception during binocular viewing via stereo-goggles at different display distances. Perception, 34 (Suppl.), 189–189. Erens, R. G. F., Kappers, A. M. L. and Koenderink, J. J. (1993). Perception of local shape from shading. Perception and Psychophysics, 54, 145–157. Flemming, R. W., Dror, R. O. and Adelson, E. H. (2003). Real-world illumination and the perception of surface reﬂectance properties. Journal of Vision, 3, 347–368. Flemming, R. W., Torralba, A. and Adelson, E. H. (2004). Specular reﬂections and the perception of shape. Journal of Vision, 4, 798–820. Gibson, J. J. (1966, 1983). The senses considered as perceptual systems. Westport, CT: Greenwood Press.

38

Product Experience

Gibson, J. J. (1979, 1986). The ecological approach to visual perception. Hillsdale (NJ), London: Lawrence Erlbaum Associates. Gilchrist, A. L. (1971). Perceived lightness depends on perceived spatial arrangements. Science, 195, 185–187. Gregory, R. L. (1973). The confounded eye. In: R. L. Gregory and E. H. Gombrich (Eds). Illusion in nature and art, pp. 49–96. London: Duckworth. Hill, H. and Bruce, V. (1993). Independent effects of lighting, orientation, and stereopsis on the hollow-face illusion. Perception, 22, 887–897. Hill, H. and Bruce, V. (1994). A comparison between the hollow-face and ‘hollow potato’ illusions. Perception, 23, 1335–1337. Howard, I. P. (2002). Seeing in depth. Volume 1, Basic mechanisms. Toronto: I. Porteous. Ikeuchi, K. and Horn, B. K. P. (1981). Numerical shape from shading and occluding boundaries. Artificial Intelligence, 17, 141–184. Johnston, A., Hill, H. and Carmen, N. (1992). Recognising faces: effects of lighting direction, inversion and brightness reversal. Perception, 21, 365–375. Khang, B. G., Koenderink, J. J. and Kappers, A. M. L. (2003). Perception of surface reﬂectance of 3D geometrical shapes: inﬂuence of the lighting mode. Perception, 32, 1311–1324. Khang, B. G., Koenderink, J. J. and Kappers, A. M. L. (2004). Shape constancy does not hold for images rendered with different types of material surfaces. Journal of Vision, 4(8), 79–79. Khang, B. G., Koenderink, J. J. and Kappers, A. M. L. (2006). Perception of illumination direction in images of 3-D convex objects: inﬂuence of surface materials and light ﬁelds. Perception, 35, 625–645. Knill, D. C. and Kersten, D. (1991). Apparent surface curvature affects lightness perception. Nature, 351, 228–230. Koenderink, J. J., Van Doorn, A. J., Christou, C. G. and Lappin, J. S. (1996a). Shape constancy in pictorial relief. Perception, 25, 155–164. Koenderink, J. J., Van Doorn, A. J., Christou, C. G. and Lappin, J. S. (1996b). A perturbation study of shading in pictures. Perception, 25, 1009–1026. Koenderink, J. J., Van Doorn, A. J. and Kappers, A. M. L. (1992). Surface perception in pictures. Perception and Psychophysics, 52(5), 487–496. Koenderink, J. J., Van Doorn, A. J., Kappers, A. M. L., Te Pas, S. F. and Pont, S. C. (2003). Illumination direction from texture shading. Journal of the Optical Society of America, 20(6), 987–995. Koenderink, J. J., Van Doorn, A. J., Kappers, A. M. L. and Todd, J. T. (2001). Ambiguity and the ‘mental eye’ in pictorial relief. Perception, 30, 431–448. Koffka, K. (1935). Principles of Gestalt psychology. London: Kegan Paul, Trench, Trubner. Landy, M. S., Maloney, L. T., Johnston, E. B. and Young, M. (1995). Measurement and modeling of depth cue combination: in defense of weak fusion. Vision Research, 35(3), 389–412. Langer, M. S. and Bülthoff, H. H. (2001). A prior for global convexity in local shape-from-shading. Perception, 30, 403–410. Law-Smith, M. J., Perrett, D. I., Jones, B. C., et al. (2006). Facial appearance is a cue to oestrogen levels in woman. Proceedings of the Royal Society of London B, 273, 135–140. Leeuwenberg, E. L. J. (1971). A perceptual coding language for visual and auditory patterns. American Journal of Psychology, 84(3), 307–349. Leeuwenberg, E. L. J. and Van Lier, R. (2005). Symmetry cues for matching mirrored objects. Spatial Vision, 18(1), 1–23. Longuet-Higgins, M. S. (1960). Reﬂection and refraction at a random moving surface. I. Pattern and paths of specular points. Journal of the Optical Society of America, 50(9), 838–844. Mamassian, P. and Kersten, D. (1996). Illumination, shading and the perception of local orientation. Vision Research, 36(15), 2351–2367. Mingolla, E. and Todd, J. T. (1986). Perception of solid shape from shading. Biological Cybernetics, 53(3), 137–151. Nefs, H. T., Koenderink, J. J. and Kappers, A. M. L. (2005). The inﬂuence of illumination direction on the pictorial reliefs of Lambertian surfaces. Perception, 34(3), 275–287. Nefs, H. T. Koenderink, J. J. and Kappers, A. M. L. (2006). Shape-from-shading for matte and glossy objects. Acta Psychologica, 121(3), 297–316. Nicodemus, F. E., Richmond, J. C., Hsia, J. J., Ginsberg, I. W. and Limperis, T. (1977). Geometrical considerations and nomenclature for reflectance #NBS MN_160. Washington: National Bureau of Standards, Department of Commerce. Norman, J. F., Todd, J. T. and Orban, G. A. (2004). Perception of 3D shape from specular highlights, deformations of shading, and other types of visual information. Psychological Science, 15(8), 655–570. Norman, J. F., Todd, J. T. and Phillips. F. (1995). The perception of surface orientation from multiple sources of optical information. Perception and Psychophysics, 57(5), 629–636.

On the visual appearance of objects

39

Oren, M. and Nayar, S. (1996). A theory of specular surface geometry. International Journal of Computer Vision, 24(2), 105–124. Palmer, S. E. (1999). Vision science: Photons to phenomenology. Cambridge, MA: MIT Press. Pentland, A. P. (1982). Finding the illumination direction. Journal of the Optical Society of America, 72, 448–455. Pont, S. C. and Koenderink, J. J. (2003). Illuminance ﬂow. Computer analysis of images and patterns. Lecture Notes in Computer Science, 2756, 90–97. Pont, S. C. and Te Pas, S. F. (2006). Material–illumination ambiguities and the perception of solid objects. Perception, 35, 1331–1350. Richards, W. (1970). Stereopsis and stereoblindness. Experimental Brain Research, 10, 380–388. Soranzo, A. and Agostini, T. (2006). Does perceptual belongingness affect lightness constancy? Perception, 35, 185–192. Te Pas, S. F. and Pont, S. C. (2005). A comparison of material and illumination discrimination performance for real rough, real smooth and computer generated smooth spheres. Proceedings APGV 2005, ACM SIGGRAPH, 75–81. Todd, J. T. and Mingolla, E. (1983). Perception of surface curvature and direction of illumination from patterns of shading. Journal of Experimental Psychology: Human Perception and Performance, 9(4), 583–595. Todd, J. T., Norman, J. F., Koenderink, J. J. and Kappers, A. M. L. (1997). Effects of texture, illumination and surface reﬂectance on stereoscopic shape perception. Perception, 26, 806–822. Todd, J. T., Norman, J. F. and Mingolla, E. (2004). Lightness constancy in the presence of highlights. Psychological Science, 15(1), 33–39. Troscianko, T., Montagnon, R., Le Clerc, J., Malbert, E. and Chanteau, P. L. (1991). The role of colour as a monocular depth cue. Vision Research, 31(11), 1923–1930. Tse, P. U. (2002). A contour propagation approach to surface ﬁlling-in and volume formation. Psychological Review, 109(1), 91–11. Van der Helm, P. A. and Leeuwenberg, E. L. J. (1996). Goodness of visual regularity: a nontransformational approach. Psychological Review, 103(3), 429–456. Wyszecki, G. and Stiles, W. S. (1982/2000). Color science: Concepts and methods, quantitative data and formulae. New York: John Wiley. Yang, J. M. and Maloney, L. T. (2001). Illuminant cues in surface color perception: tests of three candidate cues. Vision Research, 41, 2581–2600. Zihl, D., Von Cramon, D. and Mai, N. (1983). Selective disturbance of movement after bilateral brain damage. Brain, 106, 313–340.

2

THE TACTUAL EXPERIENCE OF OBJECTS MARIEKE H. SONNEVELD Delft University of Technology, Delft, The Netherlands

HENDRIK N.J. SCHIFFERSTEIN Delft University of Technology, Delft, The Netherlands

1. INTRODUCTION A child’s first tactual experiences with objects mostly involve being touched, for example by the latex gloves of the midwife, the towels she cleans the baby with, and the textiles of the clothes that separate the baby’s skin from its mother. Once children grow old enough to reach out and touch what surrounds them, their tactual experiences become active. They hold, squeeze, and swing whatever comes into reach. Their world becomes an exciting environment, in which they learn to develop their physical skills through manipulating balls, dolls, grandpa’s spectacles, bicycles, skates, and in which they learn how to avoid unpleasant encounters, such as with the sharp edges of the table. When tired of exploring and playing, a cuddly toy may wait for them to keep them company. And although children are aware that they are the active agent in kicking the ball and riding the bicycle, it is not always clear whether they are cuddling the toy or whether the toy is cuddling them; touching becomes interactive. This unavoidable reciprocity is characteristic for the senses of touch. Seeing does not imply being seen, neither does hearing imply being heard. But touching implies being touched simultaneously. Touching and being touched are integrated into one phenomenon; the tactual experience. In this chapter, we provide an overview of the different domains from which tactual experiences can be described and explored. These different domains are summarized in the Tactual Experience Guide that is discussed below (Sonneveld, 2007). A leading motive during the construction of this guide was to enable the description of the aesthetic aspects of tactual experience: What are the influential factors that make objects pleasant or unpleasant to touch? Product Experience Copyright © 2008 Elsevier Ltd.

41

42

Product Experience

Tactual experiences are common to everyday life. Nevertheless, it seems as if people hardly ever talk about them, and they appear to find it very difficult to talk about how objects feel. Suppose that you ask a person to describe how the pen he usually writes with feels. He would probably pick up the pen and start to manipulate it, turn it over, hold it in different positions, consider the different surface textures of the pen by stroking it, swing it between his fingers, possibly slightly hit the table with it, and put down some lines on a piece of paper. Obviously, this manipulation is different from the interaction when actually writing with it, or when carrying it around. Sooner or later the person will become aware of the complexity of the question and will reply: ‘What do you mean, how it feels? When I write with it, play with it, carry it, or what?’ We cannot simply ask people how an object feels, we need to take into account that the answer depends on the nature and context of the interaction with the object. In addition, people seem to lack the vocabulary to describe their tactual experiences of interacting with objects. To support designers as well as users in describing and aesthetically assessing the experience of interacting with objects, Sonneveld (2007) developed the Tactual Experience Guide. This guide helps people to describe their tactual experiences with objects, by offering a consistent framework of the different aspects of tactual experience (the content of the tool), and by offering a format that guides people through this experience (the design of the tool). The framework is based on a qualitative analysis of personal descriptions of pleasant and unpleasant tactual experiences with objects. These descriptions were collected through a printed questionnaire completed by 46 participants, 24 men and 22 women, varying in age from 18 to 67 years. The data analysis resulted in a set of themes that characterize the different aspects of tactual experience. These themes were grouped along the different domains of tactual experience that form the basis of the structure of the Tactual Experience Guide (Figure 2.1) and will be used as the outline of the current chapter: Moving with the object (Section 3); perceiving tactual

movements tactual properties

tactual experience affective behavior gut feelings

sensations

FIGURE 2.1 Overview of the five domains of tactual experience presented as a mind-map structure.

The tactual experience of objects

43

properties of the object (Section 4); sensing physical sensations (Section 5); experiencing the affective behavior of the object (Section 6); and experiencing the gut feelings involved (Section 7). Throughout this chapter, the descriptions of the different aspects of tactual experience will be illustrated with examples derived from this study. This chapter starts out with a description of the meaning of touch, based on a literature overview approaching touch from different perspectives. Next, the five domains of tactual experience are described, complete with background information for each specific domain. The chapter concludes with a section on the development of product designers’ aesthetic sensitivity for the tactual experience in human–product interaction, and recommendations for future research in the field of tactual aesthetics.

2. THE MEANING OF TOUCH 2.1. Touch: Physical encounters and awareness of oneself Physical interaction with the world is not limited to the hands, it involves the whole body. Physical engagement with the world, the awareness of touching and being touched, makes people aware of being a physical body themselves, sharing the physical world with other physical objects. It is within this embodied encounter that the ‘I’ experiences itself and its surrounding world simultaneously, making this encounter the basis for selfawareness (see Bermudez, Marcel and Eilan, 1995, for an overview). Although people can see their body, they need to sense their body to be aware of themselves. Touch allows sensing one’s own body, sensing the borders between the self and the outside world, and the interaction between the two. The neurologist Oliver Sacks emphasized this aspect of the physical experience of the body as the foundation for self-awareness in his descriptions of patients with disturbed self-perceptions. For example, a patient who did not experience his leg as part of himself any more tried to throw the alien leg out of the bed (Sacks, 1984; 1987). Such situations of disturbed tactual sensations may also occur in non-pathological cases such as when waking up with a numb arm or having an anaesthetized cheek at the dentist. When touching these body parts, they feel alien, as ‘not part of me’; they are experienced as ‘dead’ matter. These findings could be summarized in the statement ‘you are what you feel’ (Bergsma, 1987). A world in which touch is poorly addressed is likely to weaken the feeling of being in contact with the world, which may lead to a disappearing feeling of self-awareness. Paradoxically, touch may also blur the boundary between the experienced self and the world. The rubber hand illusion provides a striking example; when one is watching a rubber hand being stroked, while one’s own unseen hand is synchronously stroked, one may attribute the rubber hand to one’s own body and ‘feel like it’s my hand’ (Tsakiris and Haggard, 2005). Another phenomenon that blurs the experience of the border between the self and the outside world is people’s capacity to feel through objects, by ‘incorporating’ these objects as part of their own body (Polanyi, 1967). For example, a blind person feels the world at the end of his white stick, and a carpenter feels the wood through his saw.

2.2. Touch: A foundation for knowledge of the material world In his philosophy on education, Dewey states that the material world people live in and through forms the basis of learning and personal growth, because it is the basis of their

44

Product Experience

‘experiential world’. According to Dewey, one only learns and grows through and from experience, in physical interaction with a material world (Dewey, 1997 [1938]). Physical interaction puts the body, and thereby the tactual senses, back into experience as the foundation of knowledge. People can see the shapes and colors of the physical world, hear the events that take place, smell it, but it is only through touch that people learn about its materiality. The sight and sound of a frog, for example, give some clues for suppositions about its tactual properties, but it is only through touch that one learns about its physicality: Its weight, temperature, wetness, the texture and elasticity of its skin, its force and movements, and so on. The experience of touching a frog (and being touched by it) embodies what learning through touch is about. From this point of view, people actually need touch to get to know and understand the world. Likewise, people need touch to know and understand the man-made objects they are manipulating to grasp their meaning (Lakoff and Johnson, 1999). This knowing through bodily experiencing is different from the knowledge gained through thinking as deduction from theory.

2.3. Touch: A foundation for feelings and emotions Touching is being in physical contact and, as such, is the basis for the feeling of being in contact. Within this contact, touch is a strong basis for the development of feelings of affection and intimacy (Fields, 2003; Montagu, 1971) and is necessary for physical and mental development. Touch during the first phases of life has to be loving and protecting in order for a person to develop into a healthy, empathic human being. It is through touch that one experiences that one is safe and cared for. Experiments with rats have shown that touch-deprivation leads to growth retardation and withering (Montagu, 1971). This need for touch seems so primordial that monkey infants deprived from their mother prefer a terry-cloth surrogate mother without milk to hang on to, over a wire surrogate mother with milk. This is confirmed for people by observations of children who grew up in Romanian orphanages, where touch was infrequent due to understaffed situations. The need for loving touch remains throughout people’s lives, a phenomenon referred to as touch hunger (Fields, 2003). These insights on affective and emotional aspects of touch are based on studies of people touching people, and not on human–product interactions. However, some observations suggest that people’s affective and emotional development and well-being may also be affected by the way they are touched by objects. For example, the experiment with the monkey infants showed that being touched by a non-living terry-cloth surrogate mother contributed to the infant’s well-being. Furthermore, transitional objects such as a blanket or a teddy bear, described by Winnicott (1964) as objects that allow the child to feel safe in a world where the mother is temporarily absent, are illustrations of this affective meaning of touch embodied by objects. Another example is the cuddly walls developed for demented elderly to promote emotional well-being.

2.4. Touch: A communication channel for affection Touch implies contact and thus bodily involvement, whereas seeing and hearing are distant senses and thus are more apt to create distance and ‘objectification’ in social contact. Touch is therefore often considered as our most social sense. Interpersonal touch tells us whether we are safe, cared for and have value (Fields, 2003; Finnegan, 2002). Fagan (1998) suggested that touch is the first language that children learn to communicate

The tactual experience of objects

45

interpersonal affection. Fagan distinguished between various kinds of touch: Ritual touch (e.g. a handshake), athletic touch (e.g. in contact sports), punishing touch, nurturing touch, intimacy-evoking touch and sexual touch. These categories are not mutually exclusive; a formal handshake in a greeting ritual, for example, may feel intimate if it feels softer or lasts longer than expected. To be able to function properly in social contacts, one has to be able to understand (and express) the language of touch properly. Thayer (1982) established a taxonomy of touch based on and illustrated by the relationships people have: A functional–professional type (the doctor touching his patient for examination), social–polite type (the handshake), the friendship–warmth type, the love– intimacy type, and the sexual type. This sequence of categories can be characterized by increasing intimacy: More body parts get involved, longer and more frequent instances of touching, and an increasing variety of the types of touch involved. These social aspects of interpersonal touch that contribute to the meaning of touch may also apply to our relationships with products. The way in which people interact with products may express different kinds of affective relationships, ranging from the functional–professional to the intimate.

3. TACTUAL INTERACTION 3.1. Active and passive touch In human–object interactions, we can distinguish between touching an object and being touched by an object. These two distinct phenomena are referred to as active and passive touch, respectively (Gibson, 1962). Active touch produces a perception of the object being touched: One is exploring the object’s properties (objective pole). On the other hand, passive touch with the same object (being touched by the object) gives an internal sensation: One experiences the sensations in the body, what is being done to the body (subjective pole) (Gibson, 1962). To illustrate this distinction, imagine picking up a glass of wine, handling it in your hands, gently turning it to move the wine: You perceive its shape, its temperature, its fragility, and the movement of the liquid. On the other hand, imagine lying on the bench of a masseur who is putting hot stones on your back: You sense the pressure on your back, the warming of your skin, but you do not sense the shape and the size of the stone. In the active mode, if you focus on your hand you are able to feel the pressure of the glass against your skin, and in the passive mode you may perceive the weight of the stones on your back. Thus touching and being touched occur simultaneously in a physical encounter. Apparently, in actively reaching out to manipulate and touch the world your attention is directed towards the object, whereas in being touched your attention is directed towards the sensations caused by that object. But, in interaction, one can be made aware of both. The difference between touching or being touched is not only a matter of attention and being active or passive. It is also related to the body parts involved in the interaction, because the skin of the different body parts differs in the suitability for active or passive touch. The skin of the palm of the hands and the soles of the feet seems especially suited for touching, whereas the hairy skin, covering the rest of the body, is better equipped for signaling the locus of events touching the body (Bolanowski, 2004). Touching and being touched is not limited to the contact between the body and an object. People have the capacity to touch the environment through other objects (Burton, 1993). Some of these objects are our own non-neural extensions, such as nails, teeth, and

46

Product Experience

hair, in anatomy and physiology referred to as accessory organs (Saladin, 2001). But our interest here focuses on the man-made objects people touch through: People touch the bread through the knife they cut with, they touch the road through the bicycle they are riding on, the tennis ball through the racket they hit with, and the paper through the pen they are writing with. Although touching through intermediate objects may be different from direct touch (Lederman and Klatzky, 1999), it forms an interesting part of people’s everyday tactual experiences. In the following paragraphs, we first discuss the properties one perceives when actively touching objects, followed by the sensations perceived when being touched by objects.

3.2. Exploration strategies When you start to think about it, it is an astounding achievement of tactual perception that we perceive and recognize an object as one object. For example, when holding a glass of wine the skin is touched at different places with different pressure intensities. Joints, muscles and tendons have specific positions and apply specific forces to different parts of the glass. Therefore, it is remarkable that all these impressions are integrated into the perception of one glass of wine. This capacity of identifying an object as one object can be fooled: Aristotle discovered that rolling a pen over two crossed fingers with one’s eyes closed gives the impression of rolling two pens over the fingers. Nevertheless, generally when people use only their sense of touch, they identify the integrity of objects correctly. When people are presented with an object, they typically first try to identify it: They want to know what it is (e.g. see Gibson, 1962; Schifferstein and Cleiren, 2005). Empirical studies have shown that the tactual (haptic) system is quite rapid and accurate in recognizing three-dimensional, familiar objects. Klatzky, Lederman, and Metzger (1985) demonstrated that blindfolded subjects recognized 96% of common objects within 5 seconds, and 94% in 1–2 seconds. Tactual perception of an object and its properties is dependent on movement: It is through movement, in interaction, that people perceive tactually. Klatzky et al. (1985) studied the movements made by blindfolded people when physically exploring tactual properties of objects, and concluded that people use specific exploratory movements for the perception of specific tactual properties. In addition, these authors found that for a consistent tactual perception of the object, tactual scanning should be done in a systematic way. These tactual scanning strategies have to be learned: Experience in touching is needed to be able to explore efficiently and accurately (Davidson, 1985). Figure 2.2 shows an overview of the exploratory procedures distinguished by Klatzky et al. (1985) and indicates the corresponding tactual properties these procedures are specifically suited for. Turvey (1996) observed and researched ‘dynamic touch’ as an exploratory procedure. This procedure implies that people swing objects to ‘get a feel for them’. Dynamic touch is functional for the perception of geometrical properties, size, and weight, but especially appropriate for determining the moment of inertia of an object: Its reaction to rotation. Dynamic touch forms the basis for understanding how to use an object as a tool: How to hit a nail with a hammer and where to hit the ball with your tennis racket.

3.3. Motivations to move To understand how people interact with objects, it is important to understand why they interact in the way they do (Hodgson, 2001). In other words, we need to understand

47

The tactual experience of objects

Lateral Motion (Texture)

Pressure (Hardness)

Unsupported Holding (Weight)

Enclosure (Global Shape; Volume)

Static Contact (Temperature)

Contour Following (Global Shape; Exact Shape)

Moving Parts

FIGURE 2.2 Overview of the exploratory procedures that people use to determine specific tactual properties of objects (adapted from Klatzky et al., 1985).

their motivation for interaction. This motivation is evidently elicited by and reflected in the function of a product, but it is not necessarily limited to that function. In this Section, we give an overview of the different motivations we encountered in studies on tactual experience (see Figure 2.3): • Interaction for practical, functional use, as a tool: In many cases, objects are used with a functional goal, as a tool. The intention of the interaction is directed towards the outside world, for practical reasons. Evident examples are the use of scissors to cut paper, the use of a knife to slice bread, the use of a camera to take a picture, or the use of a car to get somewhere. However, the functional goals do not necessarily align with the goals the object was intended for. For example, scissors may be used to open a jar of paint. • Interaction to play: Another form of interaction occurs when the object is used for non-functional reasons, for playing in the broadest sense of the word, including playing sports or just messing around. Some objects are actually intended to play with; in this case the motivation is inherent to the object’s function. Examples are tennis rackets and yo-yos. But many people play with objects that were not initially meant to play with. This playing has a specific character: It involves physically moving and interacting with the object, just for the sake of the resulting sensation. It is sometimes referred to as ‘thoughtless’ playing with the object. However, in some cases the playing may involve a

48

Product Experience

to play? to explore?

to use?

how do I move … to take care? by accident?

to carry?

FIGURE 2.3 The map on Movements presents the different motivations people have to interact with an object: Why do you touch the object and how do you move?

challenge, like tearing up a coaster into as many parts as possible. In addition, playing may involve an element of cuddling, like when you stroke a pen against your skin, just because it feels so good. • Interaction to care for and to be taken care of: A specific theme in the interaction with objects is ‘to take care’. First, objects are used by people for personal care, that is, to brush one’s teeth, or to comb one’s hair. Furthermore, supporting someone can be regarded as a way of taking care; in that sense, chairs and beds take care of people. Again, this taking care can be the object’s primary function, like a coat that keeps someone warm, or a chair that supports, or it can be sought for independently from its function. For example, you can hold a coffee mug against your cheek to warm yourself. A second form of taking care is when people take care of the object: To wash it, repair it, store it, and so on. In other words, ‘taking care’ is a mutual aspect of the human–product interaction. • Interaction to explore: Regardless of its function, an object can be touched for the sake of exploring it, because it is unknown and a person wants to discover how it feels. This motivation is not necessarily restricted to unknown objects. A familiar object can also be touched just for the sake of touching it, to make contact with it. • Interaction to carry: Some interactions derive from the fact that objects are movable or portable, which involves a specific kind of interaction: Carrying the object. This carrying can be done in different ways: In a pocket, on your back, in your hands, and so on. • Interaction by accident, by coincidence: Furthermore, some interactions are not intentional or prompted by a specific motivation, but just happen by accident: People accidentally sit on something, or bump into something.

49

The tactual experience of objects

4. TACTUAL PROPERTIES OF OBJECTS Although people perceive an object as a whole, rather than as the sum of its different properties, this paragraph focuses on the perception of the different properties (Figure 2.4). For each property, the characteristic exploratory movements (see Section 3.2, Klatzky et al., 1985) are described, together with the possible dimensions on which people perceive these properties. These perceptual dimensions have been derived from experiments using sorting tasks, where participants sorted materials on the basis of their tactual similarities (e.g. see Giboreau, Navarro, Faye and Dumortier, 2001). Overall, tactual properties can be related to: The substance, i.e. the materials the object is made of: Its hardness, elasticity, plasticity, temperature and weight. The surface of the object: Its texture and patterns. The structure, i.e. the geometrical aspect of the object: Its global shape, exact shape, volume and weight distribution (balance). The moving parts of the object: The way the parts move in relation to one another.

elasticity?

movements?

force?

hardness?

material properties?

moving parts?

texture? how do I perceive its ...

geometry?

temperature?

shape? weight? size/ volume?

balance?

FIGURE 2.4 The map on Tactual Properties of an object presents the different properties perceived in tactual experience: How do you perceive the object’s tactual properties?

50

Product Experience

4.1. Hardness, elasticity and plasticity The hardness, stiffness and elasticity of an object’s materials are explored when people exert pressure on the object, for example when they squeeze the object. Other possible movements are pulling, pushing on it, and bending or wrenching it. These movements have in common that they try to transform the object. These properties evaluate the material’s resistance against, or compliance to, transformation: Hardness and softness are explored when exerting pressure (Klatzky, Lederman and Reed, 1987), and stiffness and flexibility, are explored when bending and wrenching (Ashby and Johnson, 2002). Once the material is transformed, the material’s elasticity and springiness depend on the way the material behaves when the pressure is released: Does it come back to its initial shape, or does it stay transformed? If it returns to its initial shape, the material is perceived as elastic. If it remains transformed (e.g. clay), this material property is referred to with the term ‘plasticity’.

4.2. Temperature People react differently to temperatures that are extremely high or low, compared with, temperatures that are close to their body temperature. Very high and very low temperatures are perceived instantaneously at contact, causing a strong withdrawal reflex. Temperatures that are not threatening need time to be perceived: People will leave their hands on a location for a while to perceive the difference between their body temperature and the temperature of the object. People perceive the warmness or coldness of objects, because objects with a temperature above or below body temperature cause a temperature flow. An object feels cold if it extracts warmth from the skin. To produce this sensation, the object does not only need to have a temperature below body temperature, it also has to extract warmth at a fast rate: The material needs to have low temperature resistance. Examples of such materials are glass and metal (Ashby and Johnson, 2002). Materials with high temperature resistance, such as wood or plastics, generally feel ‘warm’ even if their temperature is below body temperature. Due to the process of temperature flow, the temperature of objects changes over time when they are held, which eventually leads to neutral thermal perception (not warm, nor cold). The larger the difference between the object temperature and the skin temperature, the more accurate the temperature perception is (Tritsch, 1988).

4.3. Texture and patterns Texture is related to the properties of the material the object is made of and to the structure of the surface resulting from production techniques and surface treatment. Surface texture includes patterns, such as structured or randomly distributed details. Texture is explored through stroking the surface of an object. Stroking is necessary for the detection of fine surface textures (Hollins and Risner, 2000). Textures with coarser patterns may also be perceived through static touch (Lederman, 1981). In addition, surface texture is perceived when holding an object, thereby assessing the grip on that object (friction). Texture perception is probably one of the most studied tactual phenomena (Craig and Rollman, 1999), but researchers do not agree on an unequivocal set of perceptual dimensions for surface texture. Hollins et al. (1993) found that participants judged texture on three dimensions, the first two of which were the most important: Rough/smooth and soft/hard. The third dimension was related to the elasticity (‘springiness’) of the surface. In a follow-up study, the first two dimensions were again found, but the third

The tactual experience of objects

51

dimension was now defined as sticky/slippery (Hollins et al., 2000). Hollins et al. found bumpy/flat as an additional fourth dimension, but this dimension was not independent from the first three. In both experiments, a set of widely varying materials was used, such as sandpaper, velvet, and wood. Picard et al. (2003) investigated the perceptual dimensions of everyday tactual textures of textiles, and came up with the following set of four dimensions: Soft/harsh, thin/thick, relief/no relief, and hard/mellow. Some of the differences in outcomes between these studies may have depended on the presented materials. The perception of the roughness of a surface is not equal for all body parts: The lips and the fingers are most sensitive, while the heel, the back, and the thigh are least sensitive (Stevens, 1990). Furthermore, roughness perception is dependent on the way one moves. For example, roughness perception becomes more intense when the applied finger force increases (Lederman, 1974). For practical applications, the grip of objects is very important. The grip is mainly dependent on the degree of friction between hands and objects, which is influenced by the condition of the skin, such as dry or sweaty, and the amount of dirt on the skin. Slightly wet hands offer a greater friction force, which is why people spit on their hands before executing a task requiring firm grip. But too much water (or perspiration) forms a layer between hand and object, which causes slipping. The use of soap reduces the coefficient of friction of the hands (O’Meara and Smith, 2001). For soapy hands, textured surfaces offer the best grip, whereas for dry hands, smooth materials perform best (see Bobjer, Johansson and Piguet, 1993; Buchholz, Frederick and Armstrong, 1988). Friction is also dependent on the size of the contact area: The larger the contact area, the more friction (Highley, 1977).

4.4. Shape and size of the object Geometrical properties of objects are explored when grasping the object, holding it, manipulating it, and following the contours with the fingers. In addition, the size and shape of bigger objects are explored through dynamic touch, by swinging and wielding them. Shape discriminations can be based on the following characteristics (Lederman and Klatzky, 1987): (1) abrupt surface discontinuities, such as edges (no edge versus edge) and holes (hole versus no hole, shallow hole versus deep hole); (2) continuous 3D surface contours, such as curved versus flat; (3) orientation of surfaces (horizontal, vertical, slant). The importance of various shape properties seems to differ when they are perceived through touch compared to when they are perceived through vision. For example proportion, a typical aspect of visual perception, is not directly nor spontaneously perceived through tactual exploration (Appelle, Gravetter and Davidson, 1980). Changes in curvature are considered very important when perceiving visually, but changes in orientation of the object (its position in space) are not. This is quite the opposite when perceiving through the tactual system (Goodnow, 1969). Curvature perception depends on the direction of the scanning hand’s movement: Symmetrically curved edges often feel skewed in the direction of the moving hand (Goodnow, Baum and Davidson, 1971). In addition, shape perception is influenced by what has been perceived previously, a so called after-effect. For example, after prolonged perception of a concave surface, a flat surface is perceived as convex, and vice versa (Vogels, Kappers and Koenderink, 2001). Size in tactual perception is referred to as volume, length and width (Lederman and Klatzky, 1987). After-effects have also been described for the perception of size. After prolonged perception of an object with a certain length, longer objects are perceived as shorter and shorter objects as longer than their actual length. In general, compared to

52

Product Experience

the visual space, the tactual space seems smaller. Thus objects that were first touched and then seen look smaller than expected on the basis of touch (Teghtsoonian and Teghtsoonian, 1970).

4.5. Weight and balance An object’s weight is explored by holding the object, and by moving it up and down. Weight distribution is explored through dynamic touch, by swinging and wielding the object, or by trying to hold it still in a specific position (Turvey, 1996). Kreifeldt (2001) studied the object’s moment of inertia, perceived as an object’s resistance to rotational movement. This moment of inertia depends on where you hold the object. He illustrates this with the example of a baseball bat. The bat has a specific weight and a specific centre of gravity, but the way it is experienced depends on where you hold it: At the hitting hand or at the holding end. Weight is perceived as heavy or light, and weight distribution as balanced or unbalanced. Charpentier was the first to demonstrate, in 1891, that the perceived weight of an object depends not only on its physical mass but also on its size (Murray et al., 1999). When holding two objects of equal mass but of different size, subjects will consistently report the bigger one as lighter (Murray et al., 1999). This illusion also occurs in purely tactual situations, where the subjects are not able to see the objects (Amazeen, 1997). However, this effect emerges only when the grip on the objects is loose; it does not occur for firm grip (Ellis and Lederman, 1999). The perception of weight is influenced by what has been perceived previously. After prolonged holding of two objects of different weight in each hand, the weight of two objects of the same weight is estimated as different (de Mendoza, 1979). Weight perception is also influenced by the temperature of the object. For all body parts, cold objects rested on the skin feel heavier than thermally neutral ones. Warmth intensifies weight perception as well, but this effect is not present in all body parts. For example, it is present on the forearm, but it does not occur on the forehead (Stevens, 1980). Weight perception also seems to be influenced by other tactual properties of the object. For example, when lifting an object with the distal pads of the thumb and index finger at its sides (precision grip) the perceived weight depends on the object’s surface texture: The smoother the texture, the heavier the object is perceived. This is explained by the fact that to prevent the object from slipping, a greater normal force has to be applied when the object’s surface is smoother (Flanagan et al., 1995). Objects may consist of various constructions and mechanisms, and of moving parts. In the field of Human Factors, movement in human–product interaction has been studied extensively. This has resulted in an overview of the different types of grips used when holding an object (for example precision grip, force grip, antenna finger), different types of movements (for example translation, rotation), force exertion, and related types of control mechanisms (for example buttons, handles, wheels). For an overview of the resulting descriptions the reader is referred to MacKenzie and Iberall (1994).

5. TACTUAL SENSATIONS: BEING TOUCHED BY OBJECTS Like tactual perceptions, tactual sensations depend on the movements one makes when touching an object (Gibson, 1963). However, sensations and perceptions differ in the way they emerge from touching an object: The perception of an object may be considered as invariant throughout moving, whereas tactual sensations vary while moving. To illustrate

53

The tactual experience of objects

this, consider touching a wooden cube. You will perceive its shape through enclosure and contour following, moving the cube around in your hands. Throughout these different movements, the perception of the shape is invariant: It is a wooden cube. But this is not the case for the tactual sensations involved: The pressing of the edges of the corners, or of the flat surfaces of the cube on your skin varies with every movement of the hands and with every position of the cube. Thus the perception of the tactual invariant properties of an object coincides with varying sensations (Gibson, 1963). A physical property of an object may evoke many different sensations, depending on the way one interacts with it. This Section presents the skin and body senses and the related tactual sensations (Figure 2.5).

5.1. The skin and the skin senses Our skin is our largest organ: In adults it has a surface of 1.5 to 2 m2, is 0.5 to 4 mm thick (depending on the body part), and amounts to about 15% of total body weight (Saladin, 2001). Two types of skin cover the body: The glabrous (hairless) skin of the palms of the hands and the soles of the feet, and the hairy skin covering the rest of the body. The two types of skin seem to be equipped for different functions: The glabrous skin is suited for active touch in exploring and manipulating the world, and the hairy skin is suited for passive touch in signaling the locus of events, because of the following differences (MacKenzie and Iberall, 1994): • The glabrous skin is thicker (especially the epidermis), tougher, and more resistant to pressure. • The epidermis of the glabrous skin contains fat pads on the fingers and the bulges on the palm of the hand. These fat pads make the skin comply with the grasped object, thus facilitating a stable grip.

vibration?

itch?

body postures ? how do I sense ...

body temperature?

muscle force? pressure? pain?

FIGURE 2.5 The map on Bodily Sensations involved in tactual experience: What do you sense in your body when touching the object?

54

Product Experience

• The glabrous skin has a papillary structure: The epidermal ridges form the palmar print and the fingerprints. This structure has a sensory function: It allows the sensors to register lateral pressure. Hence, they contribute to the accuracy of the sense of touch. Furthermore, the ridges are important in grasping. They offer more grip on the grasped object, much like the profile of tires offers more grip on the road. • The distribution of the sweat glands is denser in the glabrous skin of hands and feet than in other parts of the body. The glands also differ in the way they respond to stimuli: The glands in the hand respond more to applied force (thus again facilitating grip), whereas the glands in the hairy skin respond more to temperature, thus facilitating temperature regulation. • The hairy skin lacks the Meissner’s corpuscles, responsible for the sensations of light touch and vibration. Therefore, people are unable to perceive subtle tactual details such as texture differences with body parts covered with hairy skin. Both skin types consist of three different layers, the epidermis, the dermis and the hypodermis containing a variety of sensors: Mechanoreceptors, sensitive to mechanical transformation of the skin; thermoreceptors, detecting cooling or warming of the skin; nociceptors, involved in the sensation of pain when the skin is (almost) damaged. Once the skin sensors are stimulated, neural fibers conduct this sensory information to the central nervous system. For each type of sensor, the corresponding neural fiber can either be slowly adapting (firing continuously) or rapidly adapting (only firing when the stimulus changes). These differences in adaptation rate strongly influence the duration of tactual sensations. Some sensations vanish rapidly because the sensor is adapted to the new situation (for example small temperature differences), whereas other sensations will last over time (for example deep pain) because the sensors do not adapt at all.

5.2. The skin sensations The skin sensations can be divided into three types, according to the type of sensors involved (Saladin, 2001; Vander, Sherman and Luciano, 2001): 1. Touch sensations and sensations derived from touch, such as superficial and deep pressure, and vibration (mainly mechanoreceptors involved). 2. Warm and cold sensations (mainly thermoreceptors involved). 3. Pain sensations (mainly nociceptors involved). Overall, sensations can be considered on the aspects of location (where), quality (what), intensity (how strong) and duration (how long does it last) (Gibson, 1963). We distinguish between the following types of skin sensations here (Figure 2.5): • Light touch: Light touch is what one senses when being touched without the skin being deformed. Light touch is mostly detected by rapidly adapting sensors (Saladin, 2001; Vander et al., 2001). This rapid adaptation allows people to forget about the clothes that touch them during the day. • Pressure: Pressure is maintained touch. It is experienced when an object is pressing on one’s skin and, thereby, deforming the skin. Pressure sensors are slowly adapting. That is why the sensations of deep and heavy pressure are usually difficult to ignore. • Vibration: Vibration in the skin is experienced when rapidly adapting touch sensors are stimulated rhythmically, for example when the hand strokes a texture, or when one sits on a chair and a truck is driving by, causing vibration of the floor. Receptor organs in the upper layer of the skin are sensitive to low frequency stimulation, the deeper receptors to high frequency stimulation (Sekuler and Blake, 1994).

The tactual experience of objects

55

• Cold and warmth: Although people intuitively consider warm and cold two opposites of one dimension, the sensations of warming and cooling of the skin are elicited by two different sensory systems. The skin easily adapts to temperatures between 20 and 40ºC, thus resulting in a thermal neutral perception of the object. Below 20 and above 40ºC, one perceives a cold or a warm object, but the sensation does not adapt over time. Above 45ºC, the tissue starts to be damaged, and the sensation becomes one of pain (Ganong, 2001; Sekuler and Blake, 1994). • Pain: When pain is induced by stimulation of the skin it is called superficial pain. Pain from muscles, bones and joints is called deep pain. Pain as a bodily sensation is referred to as somatic pain. The subjective experience of pain is a complex phenomenon; it differs from the other senses because it is intimately related to the affective meaning of the circumstances in which it is experienced, and the motivations of the subject. An extensive discussion on pain perception can be found in Kruger (1996). • Itch and tickle: Itch and tickle are skin sensations elicited by stimulation of nonmyelinated fibers in the skin that are highly similar to pain sensors. Itch and tickle are produced by mild stimulation when moving something across the skin (Saladin, 2001; Vander et al., 2001). Itch can also be produced by chemical stimulation of the skin. The difference between tickle and itch is not clearly defined. • Physical pleasure: Olausson et al. (2002) discovered a system of non-myelinated, slow conducting sensors in the hairy skin, that responds when a person is being touched lightly, producing a faint sensation of pleasant touch, without producing the sensation of being touched. Therefore, it seems plausible to consider the experience of physical pleasure as a physical sensation per se. The researchers suggested that people might possess a special system for limbic touch, underlying emotional responses to caress-like skin-toskin contact between individuals. Because the study used a soft brush as a stimulus, this system is not confined to interpersonal touch, but includes touching by objects.

5.3. The body senses and sensations In addition to the skin sensations, active touch involves the two body senses: Proprioception that perceives body posture, and kinaesthetics that perceives body movement (Figure 2.5). Body posture and body movement are sensed through sensors in muscles, tendons and joint tissues (Saladin, 2001; Vander et al., 2001).

5.4. Tactual sensitivity Tactual sensitivity relates to the capacity to sense if one is touched, where one is touched, for how long, and with what intensity (Lederman and Klatzky, 1998). Because the sensors in the skin are not equally distributed, tactual sensitivity depends on the locus on the body. The fingertips and lips contain most sensors per mm2 (Stevens, 1990). In addition, sensitivity depends on the spatial aspects of the afferent neurons. Receptive fields of fibers in the upper layer of the skin are relatively small (2–4 mm) and overlapping. Thus they form a sensitive system to locate a point on the skin. The deeper receptors have larger receptive fields, making the location of a point on the skin less accurate (Sekuler and Blake, 1994). Finally, sensitivity is related to the size of the reception areas in the somatosensory cortex. These areas are not proportional to the different body parts: The lips and hands cover the largest area, whereas the back and the calf cover relatively small parts. These differences in sensitivity are reflected in the homunculus defined by Penfield (Sekuler and Blake, 1994) that depicts the different body parts proportional to their reception areas in the cortex (Figure 2.6).

56

Product Experience

FIGURE 2.6 Penfield’s Homunculus.

The sensitivity due to the distribution of sensors and to the spatial characteristics of the corresponding neurons throughout the body cannot be altered by training. However, the size of the reception areas in the brain can be enlarged by training and experience. For example, blind people do not possess better discriminative abilities than sighted people (Hanninen, 1972), but they have become better in recognizing objects (Berla and Butterfield, 1977) and patterns (Craig, 1988) through touch. Tactual sensitivity is not a static aspect of an organism, but varies in time. Sensitivity decreases through aging. For example, the spatial acuity of the skin of the fingertip deteriorates with age (Stevens and Choo, 1996). Various diseases or pathological conditions can disturb the tactual senses, such as diabetes, leprosy, multiple sclerosis and Parkinson’s disease (Pratorius, Kimmeskamp and Milani, 2003). Damage of the central nervous system or peripheral nerve tissue through accident or tumors may cause a loss of sensitivity as well (Franzen and Lindblom, 1976).

6. THE BODY LANGUAGE OF OBJECTS The affective aspects of a tactual experience with an object can be characterized as the experienced body language of the object. In physical interaction objects are experienced as expressing affective behavior through their physical reactions to our actions. This affective behavior can be described along a number of themes that are discussed below (Figure 2.7).

6.1. Personality A person’s personality is sometimes characterized by words typically used to describe material properties that can be perceived tactually: People are experienced as weak, strong, hard, soft, flexible, rigid, warm, or cold, and so on. Conversely, objects may be regarded as entities expressing personality. In this case, people talk about objects as if they possess human characteristics. For example, they may say about an object that ‘it is obeying, but with dignity’, or that ‘it feels strong and playful’. According to Govers (2004), product personality can be defined as the set of human characteristics that people use to describe an object. What makes the tactual characteristics particularly interesting in this respect is that people seem to transfer the perceived tactual qualities

57

The tactual experience of objects

transparency?

physical skills? power match?

feedback?

perfect match?

how do I experience its ...

emotions?

personality?

familiarity? intentions?

FIGURE 2.7 The map on the Affective Behavior of an object involved in tactual experience: How do you experience the object’s affective behavior?

directly to the product personality: A cold object expresses a cold personality, and a flexible product may be experienced as a flexible personality. Because objects seem to have a personality, they seem to become social entities and, consequently, may evoke feelings and emotions that usually only apply to the interpersonal domain, such as feelings of sympathy. For example, people may report that they feel sorry for an object that seems sad, because it has a broken part.

6.2. Intentions People have specific motivations to interact with objects (Section 3.3), and people experience objects as having intentions as well. Sometimes an object seems to have a will of its own: It wants (or refuses) to be explored, to take care, to cooperate, to play, and so on. This perceived intentionality emerges because objects seem to display a specific physical behavior in interaction. People move in order to experience an object. In return, when the object moves this is experienced as meaningful, intentional behavior (Heider and Simmel, 1944; Michotte, 1963). Thus when an object does not react, it is experienced as refusing, rejecting. And when a chair collapses under somebody’s weight, that chair is experienced as letting that person down. Sonneveld (2007) found the following intentions in participants’ descriptions: • Wanting to be touched and explored (e.g. ‘the balloon had some kind of “I don’t want to be in your hands” reaction’). • Wanting to cooperate (e.g. ‘my walking shoes are much too heavy, they feel like they do not want to walk at all, they are too tired to do so’).

58

Product Experience

• Wanting to play (e.g. rubber bands, springs, and some on/off switches elicit repetitive movements, because they seem to ask you to go on playing with them). • Wanting to take care of and, related, wanting to love or hurt somebody (e.g. objects that literally support and take care of people due to their function, such as furniture and clothes).

6.3. Integrity: Tactual feedback In interaction, an object provides people with information. On the one hand, it provides information about itself, for example about its properties, about what it is and what it is doing. On the other hand, it provides information about the physical world around it, about what is going on. With the information the object supplies, it can guide people in what they are trying to achieve, but it may also mislead them. Therefore, the way in which objects give tactual feedback is experienced as the integrity of the object. To start with, integrity is related to whether the object gives any feedback at all: Products can be rich or poor in tactual information. For example, touch screens do not let the user feel what they are actually doing, whereas other interfaces, such as steering wheels of cars, let you know exactly what is going on. Next, if objects give feedback about what is happening, they may or may not seem honest. For example, coffee mugs of porcelain provide information about the temperature of the coffee inside, whereas polystyrene foam cups do not. In contrast to the porcelain cups that provide the correct information, people feel fooled when the coffee in the polystyrene cup is much hotter than expected. How tactual feedback is experienced is largely dependent on the expertise of the user: An experienced car mechanic will get a lot more information through manipulation of the different parts of an engine than a layperson. The appreciation of the integrity of objects is not unequivocal; being teased or fooled may be part of a pleasant experience, depending on the context of the interaction. For example, in the case of a toy crocodile the lack of integrity makes it exciting: The crocodile bites when you press one of his teeth, but you never know which tooth it is going to be.

6.4. The perfect match When touching an object, people evaluate the way it fits them. People seem to enjoy the feeling that something feels perfectly right: The perfect match. The degree of match is primarily determined by the object’s geometrical properties (Section 4.4), such as the fit of shoes and clothes, the shape and balance of tools, and the temperature of a shower. In addition, the degree of match can be related to dynamic properties, such as when the object moves with you as if it dances with you. A perfect match may be experienced immediately during the first encounter, or it may emerge from an intensive interaction over time. For example, a fountain pen adapts to the writer’s hand, and scissors adapt to the hairdresser’s hand. Over time, these objects become so well adapted to the owner’s hand that they seem impossible to work with for others.

6.5. Familiarity: Feeling ‘mine’ or alien When an object is touched frequently, it becomes familiar, it feels like one’s own, and one recognizes it as such. In contrast, an unfamiliar object may be experienced as ‘alien’ or ‘someone else’s’. Being new and unfamiliar can be a burden: People reported that as a child they had difficulties in accepting new clothes, because they did not feel as ‘own’.

The tactual experience of objects

59

This theme differs from the perfect match in the sense that familiarizing does not necessarily lead to a perfect fit, it just increases the familiarity of the object. Vice versa, if the size is right an unfamiliar object may already fit perfectly at the first encounter. Familiarity grows over time; the memories of past experiences contribute to the current experience. In addition, touch allows people to recognize objects in an affective, intimate way. One person said ‘I was wondering if it was my grandmother’s cupboard that I used to play with. When I let my hands glide across the surface of the woodcuttings, I suddenly knew: It’s hers. This tactual memory was very strong’. On the other hand, to feel somebody else ‘through’ an object can be a bewildering experience, because it evokes unwanted feelings of intimacy. For example, when you sit down on a toilet seat that is still warm from the person who sat there just before you, it feels like the seat was not yet ready to be used again; it almost feels like a strange, intimate contact with somebody else. Feelings of familiarity may be enhanced by elements that make the product unique (see Chapter 17). First, an object may be unique in shape, texture, or other details, which allows the user to recognize it as his ‘own’ object. Next, small changes that occur over time due to the interaction with the object, such as small dents, scratches or other imperfections, allow the user to recognize the object as the familiar object.

6.6. Power match and being in control A physical encounter may elicit an assessment of power: Who is the strongest? Some products literally challenge people to arm wrestling, like a marmalade jar with a tight lid. Who does not want to win, and stick the lid with triumph in the air once the jar has been defeated? In addition, the power match addresses the question about who is in control in the interaction: Who leads whom? Power seems to be mainly explored in the first encounter. It is not only about winning and being the strongest, but also about exploring limits: How far can I go? How much can I bend this stick? How far can I stretch this band? Furthermore, this theme assesses the requested physical effort one has to invest in the interaction. It can be overwhelming to sense the power one has, to feel that one is completely in control, for example in driving a car. But the opposite can be exciting as well: Luna park attractions are exciting because people experience that they are completely out of control.

6.7. Challenge of developing physical skills Physical interaction involves physical skills. Objects differ in the way they challenge people to develop their skills. Some objects require great skills, such as musical instruments, and some do not, such as objects with push buttons. People may have to invest a lot of effort to develop the required skills, for example in the case of learning to play the guitar. Once acquired, the practice of a physical skill can be a source of pleasure, of flow (Csikszentmihalyi, 1990). People are programmed biologically to like to develop their physical skills and to exercise them (Veenhoven, 2006). To develop skills, one has to develop ‘tactual knowledge’ through interacting with the objects. For example, a masseur needs to ‘know’ from experience the human body in its different qualities, to be able to work with it. Likewise, a sculptor needs to ‘know’ the material he is working on from experience. This aspect is closely related to the theme of tactual feedback. Objects differ in the way they allow people to develop a personal style when developing these skills, often referred to as doing things with a ‘personal touch’. Some objects

60

Product Experience

prescribe skills in a rather straightforward way, such as pressing the buttons on a microwave oven. Other objects allow more freedom to develop one’s own style, such as making cocktails with a cocktail shaker. In contrast, some objects seem to have their own, personal user manual that only the user knows. This is a particular form of physical skill and of ‘knowing’, because it has nothing to do with an established skill. This skill is specific for objects with stubborn personalities that can only be managed in a particular way. The theme of physical skills is closely related to the theme of power. When a person needs physical skills to interact with certain objects, the objects will have power over that person as long as (s)he has not developed these skills. For example, a car seems uncontrollable when you do not know how to drive it, but you gain control over it over the course of your driving lessons.

6.8. Attention: Tactual transparency and tactual noise When touching an object, people are in contact with that object, but their attention is not necessarily directed towards the object. The theme ‘transparency’ refers to the capacity of the object to allow people to feel through the object, to incorporate it, and to direct their attention to something else in their environment. For example, when riding a bicycle, you feel the road surface through the handle-bars and through the saddle. However, an annoying tactual sensation can create ‘tactual noise’ that reduces the degree of transparency, for example, when the handles get sticky they will attract the majority of attention and, thereby, divert the attention that was first directed towards the road. Condoms and surgical gloves with true touch are typical examples of products for which manufacturers have tried to decrease tactual noise, but nevertheless these objects may stay in one’s focus of attention. Condoms are a good example of the dual design possibilities of tactual transparency; either they are designed as tactually transparent as possible, or the designer accepts the fact that people perceive them anyway and tries to design a particular tactual experience.

6.9. Conclusion on the themes The previous paragraphs discussed the different themes involved in the body language of objects. Although some themes are more relevant than others to understand and describe specific experiences, they are all useful to describe the different aspects of a tactual experience as a whole. The themes should, therefore, be seen as different aspects of the same phenomenon. Also, they are not mutually exclusive, but they are related to each other. Thus, whatever the perspective taken, the other themes will emerge as context.

7. THE FEELINGS INVOLVED IN TACTUAL EXPERIENCE In human–product interaction, the experienced affective behavior of the object is reflected in the feelings elicited by this behavior. These different types of feelings are described below (Figure 2.8): • Physical pleasure: Lust and pain or disgust. Obviously, pleasantness in tactual experience is related to lust and physical pleasure. The pleasure is generally characterized by descriptions like ‘it feels good’, with superlatives such as ‘delicious’ and ‘delightful’. When people are asked to focus on tactual experiences, they are often amazed by the

61

The tactual experience of objects

bodily reaction? self experience?

physical pleasure? what is my ...

action tendency?

vulnerability?

energy?

affective reaction?

FIGURE 2.8 Map on Gut Feelings involved in tactual experience with an object: What feelings do you have when touching the object?

pleasantness of the experience: ‘It was even better than I thought it would be’. The physical pleasure does not seem to have one single counterpart, some descriptions of unpleasantness involve pain (e.g. ‘painful’, ‘it hurts’, ‘it stings’), some refer to light or strong discomfort (e.g. ‘uncomfortable’, ‘annoying’, ‘terrible’), while others contain elements of disgust (e.g. ‘repulsion’, ‘a dirty feeling’). People did not describe their feelings of physical pleasure or pain and disgust elaborately; these descriptions often missed nuances. In many cases, these feelings were exemplified by expressive sounds, such as ‘wow’, ‘mmmm’, ‘ugh’, or ‘yuk’ and the experience seemed to be grounded in a physical reaction accompanied by bodily responses, such as goose bumps, shivers, and feelings of nausea. Physical (un)pleasantness should not be confused with the overall assessment of the experience, because in some cases physical unpleasantness is nevertheless appreciated. As one person responded: ‘Its sliminess was disgusting, but at the same time that was also what made it attractive to touch’. And another one said: ‘It hurts so good’. • Affection: Love and hate. The feelings involved in interactions with products seem to be related to the experience of mutual affection, of feeling love for the object, as well as feeling loved by the object. This theme reflects the perspective described in Section 2.4, considering touch as a communication channel for affection. People refer to this theme in various ways. First of all, they indicate that they feel love or feel loved (e.g. ‘it has cuddle value for me’, ‘I feel tenderness’), or that they feel hatred or feel being hated (e.g. ‘I see it as a necessary evil’, ‘It drives me mad’). In some cases, the degree of affection seems to depend on the degree of intimacy between the person and the product (e.g. ‘I have my own way of cuddling it’).

62

Product Experience

A related set of feelings involve respect and contempt. As one person states: ‘When you push it harder it still feels soft, but at the same time it gives some resistance, which evokes respect’. Another set of feelings addresses acceptance and rejection. For example: ‘It accepts me the way I am’. Furthermore, affection may involve feelings of attachment. • Vulnerability: Trust and fear of getting hurt. Touch involves the body in physical contact, and confronts people with their physical vulnerability. This theme involves feelings of trust and distrust. It has to do with people’s fear of getting hurt, and also with feelings of being reassured, of trusting objects, and feeling safe with them. In addition, mixed experiences may be reported as well: ‘Under water, the muddy soil was an unknown world, but once you overcame your fear, it was a source of pleasure’. This theme is also related to feelings of freedom and oppression. For example, whereas some clothes give ‘the feeling of being free, like Peter Pan’, other clothes can make you ‘feel oppressed, suffocating’. • Energy: Tension and relaxation. The physical interaction with an object influences a person’s energy level. The energy may increase or decrease, and the energy may be experienced as positive or negative energy. This leads to reports of being physically excited, stressed, relaxed or washed-out. • Action tendency: Approach and avoidance. Action tendencies are components of emotions. An action tendency implies that the actual touching did not occur yet. Hence, the tactual experience may already start before you actually have physical contact with the object; you could call it ‘touching with the eyes’. Similarly, once a person knows how bad something feels, he may try to avoid it. Beside a tendency to approach or avoid, people may experience a tendency to hold on to or let go, and to take care of or neglect, or even to destroy. • Tactual characteristics reflected in self-experience: The tactual properties of objects can be transferred to the self-experience of the person who touches the object. For example, touching something cold can make one feel cold; or touching something dirty can make one feel dirty. The same phenomenon occurs for the experienced personality aspects of the object. For example, when a woman experiences an object as impressive, using the object may make her feel more impressive herself. An attempt to characterize these themes all together led to the umbrella concept of gut feelings as the basic concept for the emotions and feelings experienced in tactual interaction (Figure 2.8). Gut feelings are characterized as feelings that emerge from a non-reflective, direct interaction with the world, and may be related to the visceral level of interaction defined by Norman (2002). Gut feelings are related to our intuitive orientation on the world; they are grounded in our physical actions and physiological reactions, rather than in our cognitive orientation to the world, which is grounded in our thoughts. This explains why it is difficult for people to talk about touch and feelings: It is a non-verbal (or as some would say pre-verbal), intuitive mode of interaction.

8. EDUCATING THE TACTUAL SENSES In design education, students need to develop their awareness, insight and sensitivity for the different aspects of aesthetic experience in human–product interaction. This education cannot be achieved by a mere transfer of knowledge through lectures and readings. It is generally acknowledged that personal hands-on experiences should be part of the educational setting. To educate the designer’s senses, it seems therefore appropriate to develop tools and methods that offer a conceptual framework about the sensory experience,

The tactual experience of objects

63

that is embedded in practical, sensory experiences. Such an approach is based on the interplay between cognitive learning and perceptual learning; once people have a set of concepts to describe what they feel, they will be able to perceive more nuances, and thus describe their experiences more extensively. And vice versa, the more they have experienced physically, the more they will be able to give content to the specified concepts (Chollet, Valentin and Abdi, 2005). Product design education does not seem to offer such educational tools and methods yet. But other domains, as for example art education or wine tasting, offer valuable insights in possible starting points for such educational tools and methods. In art education, several methods were developed to learn to look. These methods offer a set of concepts people may use when looking at and experiencing art. For example, Visser (1986) developed a framework that guides people through the process of experiencing art, focusing on the formal aspects of art perception, such as materials, composition, size, space, point of view, abstraction, and so on. On the other hand, Armstrong (2000) proposed a set of different stages that people may go through when experiencing art, such as gathering information, dreaming, contemplation, investment (engagement), and so on. Both approaches are characterized by a conceptual framework illustrated with examples that the perceiver must apply in his own situation to assimilate their meaning. In the domain of the sensory evaluation of food, tools and methods were developed to enable people to train their sensory sensitivity. For example, the sensory user’s manual for wine tasting (LaMar, 1997) provides an overview of the different sensory aspects of wine and instructions on how to drink it in order to perceive its different aspects. People who attend wine-tasting workshops are thus presented with words that describe sensory aspects, but these words remain empty shells when they are not supported by wine tasting itself. The concepts become meaningful when they are experienced physically. It is only through experience that the concepts become embodied knowledge, and thus recognizable in experience. Therefore, to become wine-tasting experts, people need to build up a personal, embodied ‘database’ of sensory experiences, linked to the framework that is offered, to be able to work with that framework when actually tasting wine. Likewise, tools and methods in the education of the tactual senses in product design should provide designers with a frame of thought introducing the sensory, perceptual and experiential aspects of the specific sensory domain, a structure to guide the designers through these aspects, and insight in appropriate ways to explore these aspects. Moreover, these tools should support designers to build up a personal, embodied database linked to the conceptual framework that is offered on tactual experiences, and to customize the framework to fit the personal world of experience. It is from this starting point that the Tactual Experience Guide was developed. It offers a basis for the development of the aesthetic sensitivity of product design students, provided it is embedded in a personal hands-on exploration of the presented concepts. To ensure these hands-on experiences, product design education should include design courses addressing the aesthetic aspects of the senses.

9. FUTURE DEVELOPMENTS By presenting the content of the Tactual Experience Guide (Sonneveld, 2007) and some of the qualitative data on which it is based, the present chapter has provided a conceptual framework for the description of tactual experiences through words. During design

64

Product Experience

courses at Delft University of Technology, using and evaluating the guide has demonstrated the usefulness of words to get insight into the experience, but simultaneously it has made clear that words are limited in expressing nuances in tactual aesthetics. Therefore, we should explore the possibility to include the use of images, sounds, tangible samples, and so on, when using the Guide. In addition, the evaluation suggested that the Guide could be accompanied by a toolbox containing objects and materials that illustrate the different aspects of tactual experience. As it is only through hands-on experiences that one gains insight in tactual aesthetics, such hands-on experiences should be provided when being introduced to the Guide. Another limitation of the Tactual Experience Guide is that it does not provide insight into the underlying relations between the different aspects it describes. In visual aesthetics, relations between visual properties are explored to understand the underlying principles of aesthetic experience (Kreitler and Kreitler, 1972). These principles include concepts such as contrast, composition, proportion, (dis)continuities, ambiguities, and so on. In addition, the domain of visual aesthetics explores the relationship between visual properties of objects and exposure to these objects, leading to insights such as the MAYA-principle: Most Advanced Yet Acceptable (see Chapter 10). The field of tactual aesthetics lacks insight in such underlying principles. To develop the research field of tactual aesthetics, future research should focus on the underlying principles of aesthetic experience, thus possibly revealing some universal tactual Gestalt principles, and revealing insight in relations between aesthetic experience and interaction factors such as exposure frequencies (Hekkert, 2006). Finally, the insight that tactual experience in human–product interaction can be described as the body language of two social entities opens up new ways to explore the aesthetics of tactual experience, inspired by inter-human behavior and relationships. Possible sources of inspiration are insights in body language (Mehrabian, 1972); attachment (Mugge, Schifferstein and Schoormans, 2006); personality studies (Govers, 2004; Totton and Jacobs, 2001); haptonomics (Veldman, 1996); and social well-being (Ryan and Deci, 2001). For example, Ryan and Deci (2001) concluded that well-being in a relationship between two people is dependent on three factors: One should feel understood, feel competent, and have fun. It is intriguing to translate these findings into requirements for well-being in human–product relationships and to research their implications. Likewise, according to Veldman (1996) experiencing the integrity of the other, and trusting the other, form a necessary and solid base for an exchange of affection. Again, this may offer interesting hypotheses for human–product relationships that could lead to new insights in product design. To conclude this brief overview of possibilities, Totton and Jacobs (2001) observed that people all need to feel understood and accepted by other human beings. Likewise, we could hypothesize that people all need to feel understood and accepted by at least some of the objects that surround them. These directions for future developments in Tactual Aesthetics are not exhaustive; other directions may be explored as well. Nevertheless, the directions mentioned above illustrate that the presented conceptual framework forms a consistent basis to build future theory on; it can be the starting point for developing new hypotheses and research questions, exploring the full diversity of the field of Tactual Aesthetics. In addition, we think that the content and the structure of the conceptual framework will help to bridge the gap between research and design. By supporting designers in developing their tactual sensitivity and exploring the tactual aspects of the objects in their design projects, new insights in Tactual Aesthetics may be obtained with relevance to researchers in the field.

The tactual experience of objects

65

REFERENCES Amazeen, E. L. (1997). The effect of volume on perceived heaviness by dynamic touch: with and without vision. Ecological Psychology, 9, 245–263. Appelle, S., Gravetter, F. J. and Davidson, P. W. (1980). Proportion judgments in haptic and visual form perception. Canadian Journal of Psychology, 34, 161–174. Armstrong, J. (2000). The intimate philosophy of art. London: Penguin Books. Ashby, M. and Johnson, K. (2002). Materials and design: the art and science of material selection in product design. Oxford: Butterworth-Heinemann. Bergsma, J. (1987). Identiteit, lichamelijke verstoring en autonomie. Lochem: De Tijdstroom. Berla, E. P. and Butterfield, L. H. (1977). Tactual distinctive features analysis: training blind students in shape recognition and in locating shapes on a map. Journal of Special Education, 11, 335–346. Bermudez, J. L., Marcel, A. and Eilan, N. (1995). The body and the self. London: MIT Press. Bobjer, O., Johansson, S.-E. and Piguet, S. (1993). Friction between hand and handle. Effects of oil and lard on textured and non-textured surfaces; perception of discomfort. Applied Ergonomics, 24, 190–202. Bolanowski, S. J. (2004). Passive, active and intra-active (self) touch. Behavioral Brain Research, 148, 41–45. Buchholz, B., Frederick, L. J. and Armstrong, T. J. (1988). An investigation of human palmar skin friction and the effects of materials, pinch force and moisture. Ergonomics, 31, 317–325. Burton, G. (1993). Non-neural extensions of haptic sensitivity. Ecological Psychology, 5, 105–124. Chollet, S., Valentin, D. and Abdi, H. (2005). Do trained assessors generalize their knowledge to new stimuli? Food Quality and Preference, 16, 13–23. Craig, J. C. (1988). The role of experience in tactual pattern perception: a preliminary report. International Journal of Rehabilitation Research, 11, 167–183. Craig, J. C. and Rollman, G. B. (1999). Somesthetis. Annual Review of Psychology, 50, 305–331. Csikszentmihalyi, M. (1990). Flow: the psychology of optimal experience. New York: Harper Collins. Davidson, P. W. (1985). Functions of haptic perceptual activity in persons with visual and developmental disabilities. Applied Research in Mental Retardation, 6, 349–360. de Mendoza, J.-L. J. (1979). Demonstration of an aftereffect occurring in the tactile-kinesthetic domain. Psychological Research, 40, 415–422. Dewey, J. (1997 [1938]). Experience and education. New York: Touchstone. Ellis, R. R. and Lederman, S. J. (1999). The material-weight illusion revisited. Perception and Psychophysics, 61, 1564–1576. Fagan, J. (1998). Thoughts on using touch in psychotherapy. In: E. W. L. Smith, P. R. Clance and S. Imes (Eds.) Touch in psychotherapy. New York: Guilford. Fields, T. (2003). Touch. London: MIT Press. Finnegan, R. (2002). Communicating. The multiple modes of human interconnection. London: Routledge. Flanagan, J. R., Wing, A. M., Allison, S. and Spenceley, A. (1995). Effects of surface texture on weight perception when lifting objects with a precision grip. Perception and Psychophysics, 57, 282–290. Franzen, O. and Lindblom, U. (1976). Tactile intensity functions in patients with sutured peripheral nerve. In: Y. Zotterman (Ed.) Sensory functions of the skin with special reference to man. Oxford: Pergamon Press. Ganong, W. F. (2001). Review of medical physiology. New York: McGraw-Hill. Giboreau, A., Navarro, S., Faye, P. and Dumortier, J. (2001). Sensory evaluation of automotive fabrics: the contribution of categorization tasks and non-verbal information to set-up a descriptive method of tactile properties. Food Quality and Preference, 12, 311–322. Gibson, J. J. (1962). Observations on active touch. Psychological Review, 69, 477–491. Gibson, J. J. (1963). The useful dimensions of sensitivity. American Psychologist, 18, 1–15. Goodnow, J. J. (1969). Eye and hand: differential sampling of form and orientation properties. Neuropsychologia, 7, 365–373. Goodnow, J. J., Baum, B. and Davidson, P. (1971). A haptic error: skew in symmetrical curve. Perception and Psychophysics, 10, 253–256. Govers, P. (2004). Product personality. Unpublished PhD dissertation, Delft University of Technology, Delft, the Netherlands. Hanninen, K. A. (1972). Review of the educational potential of texture and tactually discriminable patterns. Journal of Special Education, 5, 133–141. Heider, F. and Simmel, M. A. (1944). An experimental study of apparent behavior. American Journal of Psychology, 57, 243–259. Hekkert, P. (2006). Design aesthetics: principles of pleasure in design. Psychology Science, 48, 157–172. Highley, D. R. (1977). Frictional properties of the skin. Journal of Investigative Dermatology, 69, 303–305. Hodgson, J. (2001). Mastering Movement. New York: Routledge.

66

Product Experience

Hollins, M., Bensmaia, S., Karlof, K. and Young, F. (2000). Individual differences in perceptual space for tactile textures: evidence from multidimensional scaling. Perception and Psychophysics, 62, 1534–1544. Hollins, M., Faldowski, R., Rao, S. and Young, F. (1993). Perceptual dimensions of tactile surface texture: a multidimensional scaling analysis. Perception and Psychophysics, 54, 697–705. Hollins, M. and Risner, S. R. (2000). Evidence for the duplex theory of tactile texture perception. Perception and Psychophysics, 62, 695–705. Klatzky, R. L., Lederman, S. J. and Metzger, V. A. (1985). Identifying objects by touch: an ‘expert system’. Perception and Psychophysics, 37, 299–302. Klatzky, R. L., Lederman, S. J. and Reed, C. (1987). There’s more to touch than meets the eye: the salience of object attributes for haptics with and without vision. Journal of Experimental Psychology: General, 116, 356–369. Kreifeldt, J. G. (2001, June 27–29). Designing ‘feel’ into a product. Paper presented at the International Conference on Affective Human Factors Design, Singapore. Kreitler, H. and Kreitler, S. (1972). Psychology of the arts. Durham, NC: Duke University Press. Kruger, L. (1996). Pain and touch. San Diego: Academic Press. Lakoff, G. and Johnson, M. (1999). Philosophy in the flesh: the embodied mind and its challenge to western thought. New York: Basic Books. LaMar, J. (1997). Tasting wine: a sensory user’s manual. Retrieved 23 March 2007, from http://avalonwine. com/Tasting-Wine.php Lederman, S. J. (1974). Tactile roughness of grooved surfaces: the touching process and effects of macro- and microsurface structure. Perception and Psychophysics, 16, 385–395. Lederman, S. J. (1981). The perception of surface roughness by active and passive touch. Bulletin of the Psychonomic Society, 18, 253–255. Lederman, S. J. and Klatzky, R. L. (1987). Hand movements: a window into haptic object recognition. Cognitive Psychology, 19, 342–368. Lederman, S. J. and Klatzky, R. L. (1998). The hand as a perceptual system. In: K. J. Connolly (Ed.) The psychobiology of the hand. London: MacKeith Press. Lederman, S. J. and Klatzky, R. L. (1999). Sensing and displaying spatially distributed fingertip forces in haptic interfaces for teleoperator and virtual environment systems. Presence, 8, 86–103. MacKenzie, C. L. and Iberall, T. (1994). The grasping hand. Amsterdam: North Holland. Mehrabian. (1972). Non-verbal communication. Chicago, Il: Aldine-Atherton. Michotte, A. (1963). The perception of causality. London: Methuen. Montagu, A. (1971). Touching. New York: Columbia University Press. Mugge, R., Schifferstein, H. N. J. and Schoormans, J. P. L. (2006). Product attachment and product lifetime: the role of personality congruity and fashion. In: K. M. Ekstrom and H. Brembeck (Eds.) European Advances in Consumer Research, 7, 460–466. Duluth, MN: Association for Consumer Research. Murray, D. J., Ellis, R. R., Bandomir, C. A. and Ross, H. E. (1999). Charpentier (1891) on the size–weight illusion. Perception and Psychophysics, 61, 1681–1685. Norman, D. A. (2002). Emotion and design: attractive things work better. Interactions, 9(4), 36–42. Olausson, H., Lamarre, Y., Backlund, H., et al. (2002). Unmyelinated tactile afferents signal touch and project to insular cortex. Nature Neuroscience, 5, 900–904. O’Meara, D. M. and Smith, R. M. (2001). Static friction between human palmar skin and five grabrail materials. Ergonomics, 44, 973–988. Picard, D., Dacremont, C., Valentin, D. and Giboreau, A. (2003). Perceptual dimensions of tactile textures. Acta Psychologica, 114, 165–184. Polanyi, M. (1967). The tacit dimension. Garden City, NY: Doubleday. Pratorius, B., Kimmeskamp, S. and Milani, T. L. (2003). The sensitivity of the sole of the foot in patients with Morbus Parkinson. Neuroscience letters, 346, 173–176. Ryan, R. M. and Deci, E. L. (2001). On happiness and human potentials: a review of research on hedonic and eudemonic well being. Annual Review of Psychology, 52, 141–166. Sacks, O. (1984). A leg to stand on. London: Picador. Sacks, O. (1987). The man who mistook his wife for a hat and other clinical tales. New York: Harper and Row. Saladin, K. S. (2001). Anatomy and physiology: the unity of form and function. New York: McGraw-Hill. Schifferstein, H. N. J. and Cleiren, M. P. H. D. (2005). Capturing product experiences: a split-modality approach. Acta Psychologica, 118, 293–318. Sekuler, R. and Blake, R. (1994). Perception (3rd Ed.) New York: McGraw-Hill. Sonneveld, M. H. (2007). Aesthetics of tactual experience. Unpublished PhD dissertation, Delft University of Technology, Delft, the Netherlands. Stevens, J. C. (1980). Thermal intensification of touch sensation: further extensions of the Weber phenomenon. Sensory Processes, 3, 240–248.

The tactual experience of objects

67

Stevens, J. C. (1990). Perceived roughness as a function of body locus. Perception and Psychophysics, 47, 298–304. Stevens, J. C. and Choo, K. K. (1996). Spatial acquity of the body surface over the life span. Somatosensory and Motor Research, 13, 153–166. Teghtsoonian, R. and Teghtsoonian, M. (1970). Two varieties of perceived length. Perception and Psychophysics, 8, 389–392. Thayer, S. (1982). Social touching. In: W. Schiff and E. Foulke (Eds.) Tactual perception: a sourcebook. Cambridge: Cambridge University Press. Totton, N. and Jacobs, M. (2001). Character and personality types. Buckingham: Open University Press. Tritsch, M. F. (1988). The veridical perception of object temperature with varying skin temperature. Perception and Psychophysics, 43, 531–540. Tsakiris, M. and Haggard, P. (2005). The rubber hand illusion revisited: visuotactile integration and selfattribution. Journal of Experimental Psychology, 31, 80–91. Turvey, M. T. (1996). Dynamic touch. American Psychologist, 51, 1134–1152. Vander, A. J., Sherman, J. and Luciano, D. (2001). Human Physiology. The mechanisms of body functions. New York: McGraw-Hill. Veenhoven, R. (2006). Healthy happiness. Rotterdam: Erasmus University. Veldman, F. (1996). Haptonomie: Wetenschap van de affectiviteit (Haptonomics: Science of affectivity). Utrecht: Erven J. Bijleveld. Visser, A. (1986). Hardop kijken (Looking aloud). Nijmegen: SUN. Vogels, I. M. L. C., Kappers, A. M. L. and Koenderink, J. J. (2001). Haptic after-effect of successively touched curved surfaces. Acta Psychologica, 106, 247–263. Winnicott, D. W. (1964). The child, the family and the outside world. Middlesex, UK: Penguin.

3

THE EXPERIENCE OF PRODUCT SOUNDS RENÉ VAN EGMOND Delft University of Technology, Delft, The Netherlands

1. WHETHER TO BE SILENT In our daily life we are confronted with the sounds of industrial products. However, people are often not aware that they ‘use’ these sounds in a functional or experiential way. Many people use auditory cues ‘unconsciously’. For example, the sound of a water boiler informs one in which phase the boiling process is. Furthermore, people derive not only functional information from the sound, but also information concerning an object’s physical properties. For example, you hear if a big truck is coming towards you even if the truck is far away. This sound is quite different from the sound of an approaching sports car. Moreover, even if the sports car was nearby instead of far away – making the sound louder – the truck still ‘sounds’ bigger. This indicates that people estimate the size of objects on cues different from loudness. Studies have shown that people use sound to estimate the size and the shape of objects (Carello, Anderson and Kunkler-Peck, 1998; Grassi, 2005; Kunkler-Peck and Turvey, 2000), to estimate the size and speed of rolling balls (Houben, Kohlrausch and Hermes, 2004; 2005), and to identify material properties (Giordano and McAdams, 2006; Hermes, 1998). In addition, sound has been used in estimating the quality of engines of vacuum cleaners (Benko et al., 2004; 2005). These examples show that sound is an important carrier of information that people use in interaction with the environment and objects. Research in the field of product sound design has often dealt with the issue of how to make products silent (e.g. Park, Lee and Lee, 2004; Rittmueller, Mann and Holger, 1997). In my view to silence products is not our main goal (although sometimes very important), but the goal is to enhance the experience of a product with proper sound design. Silence should be one of the means available to a sound designer, and will Product Experience Copyright © 2008 Elsevier Ltd.

69

70

Product Experience

be dependent on the type of task a product will be used for. For example, it is wellknown that background noise that exceeds 40 dB(a) limits the concentration of people (Lindeman and Stekelenburg, 1981, p. 114). Therefore, one should consider this aspect when designing a product for office use. The fans of personal computers that emit noise at such a loudness level that they have been compared with jet engines are notorious. Or the noise certain ink jet printers make, caused by the transport of the paper and the movement of the ink cartridge. The former example is, of course, worse because personal computers make the noise during the entire day, whereas in the latter example one chooses the time for printing. Silence can also – and probably should – be used in the design of alarm sounds for hospital situations. Personnel often switch off these sounds because they are too loud and too irritating (Chambrin, 2001; Edworthy and Hards, 1999; Friesdorf, Buss and Gobel, 1999; Solsona et al., 2001). Alarms go off when the value of a measurement device breaches a certain threshold (e.g. if the heart rate suddenly drops). However, often only the value of the threshold has been implemented, and not a duration that would indicate how long a certain threshold can be breached without becoming too dangerous. In other words, there is no time-dependency. Consequently, alarms continue to go off after the threshold has been breached for only a short time, even if the situation is no longer critical. The experience of alarm sounds would improve considerably if these sounds did not depend on these short temporal breaches of thresholds. The term sound quality has often been used in relation to reducing noise or in relation to psychoacoustic measures as, for example, sharpness and roughness (e.g. Brandl, Biermayer and Pfluger, 1999; Ih et al., 2003; Keiper, 1997; Lyon, 2000). However, recent discussions suggest that, dependent on the definition of quality, other aspects – e.g. the association of meaning – should be introduced (e.g. Blauert and Jekosch, 1997; Jekosch, 2004). In this chapter, the term product sound experience will be used, because it is a more general concept (related to the emotional, semantic, and sensorial experience), and because the term is more neutral. It also is a term that is related to the human aspect and not to how the sound can be described in terms of spectral composition or signalto-noise ratios. The latter sound features may evoke or be related to a certain experience and are therefore important to understand, but a one-to-one mapping of certain features to an experience will be too simplistic. In the next paragraph the domain of product sounds will be defined.

2. THE DOMAIN OF PRODUCT SOUNDS The domain of product sounds can be considered as a subdomain of environmental sounds (e.g. Ballas, 1993, 2002; Ballas and Howard, 1987; Björk, 1985; Gaver, 1993a, 1993b; Kidd and Watson, 2003; Marcell et al., 2000). A number of studies investigated memory, recognition, identification and the descriptors of environmental sounds. Environmental sounds comprise the sounds that we hear during our daily life, or – in other words – the sounds that surround us. This means that environmental sounds comprise a diversity of sounds, e.g. car sounds, birds whistling, water sloshing, thunder, horns, bells, dogs barking, etc. Of course, these sounds are often culture or country dependent. In the desert, one will perceive different sounds than in the streets of New York. For example, Ballas (1993) has asked people to keep a diary of the sounds they heard. The sound of highway traffic occurred quite often. There will be many places in the world where this sound is not heard, indicating the importance of contextual information.

The experience of product sounds

71

Although sounds of products are with us during our daily life, a perceptual domain of product sounds has not been established. We define the domain of product sounds as being the sounds that are produced by industrial products, cars, and user interfaces. Among product sounds, we distinguish between consequential and intentional sounds. Consequential sounds (mechanical sounds, Västfjäll et al., 2002) are a consequence of the moving parts of products and are affected by the interaction when a person uses a product. For example, a toothbrush sound will be generated by the engine (its rpm), the gear, the transmission, and the pressure and movement applied by the user. Intentional sounds are deliberately added to a product. These sounds are synthesized or recorded. They are radiated through a loudspeaker or by a vibrating surface. User interface and alarm sounds belong to this category. The sounds are often more musical or speech-like than the consequential sounds. Consequently, the design and the experience of intentional sounds will differ from consequential sounds. Before describing the specific properties of the consequential and intentional sounds, a short introduction of the spectral and temporal structure of sound will follow in the next paragraph.

3. SPECTRAL AND TEMPORAL STRUCTURE OF SOUNDS Three perceptual domains of sound can be distinguished: Speech sounds, musical sounds, and environmental sounds (sometimes called everyday sounds). Certain sounds may be classified in more than one domain. In these cases, the context in which the sound is heard will be the determinant factor to relate it to a specific domain. For example, singing produces speech sounds (especially if one considers the Sprechstimme (‘speech-voice’)), but this does not warrant to state that every speech sound is a musical sound. Each domain has its own characteristic features. Sounds are described by an amplitude fluctuation as a function of time in the temporal domain and as a sum of sinusoids (overtones) in the spectral domain. Each event in the temporal domain can be mathematically transformed (i.e. Fourier transformation) into the spectral domain and vice versa. A Fourier-transform of a time-signal results in a series of sinusoids (with their relative amplitude and phases). This sum of sinusoids determines the spectral composition of the sounds. Natural sounds – and most synthetic sounds – are composed from several sinusoids called the overtone series. The number of overtones, their frequencies and amplitude fluctuations determine the sound color or timbre. Vowels and consonants can be considered as the basic constituents of speech. Vowels are recognized by their timbre characteristics. Formants are specific areas in the speech spectrum. The relationship between formants is an important aspect in the recognition of vowels. Consonants are noise or click-like sounds. Although speech has been applied in user interfaces, it has mostly been used to inform a user. Some interfaces have been developed to recognize a user’s speech commands, but they often fail. One of the reasons is that the systems have to be trained to recognize specific voices. Human speech recognition not only analyzes the sound, but in addition applies top-down knowledge on grammar, lexicon, and uses the meaning of the produced sentences. One fills in gaps that have not been perceived in the speech sound. In this chapter we will not consider the perception or design of speech in user interfaces. Two categories of musical instruments can be distinguished: (a) pitch-producing (e.g. violin, guitar, piano, etc.); and (b) ‘noise’ producing (e.g. drums, wood-blocks). These categories are not that strict. For example, timpani are often tuned in fourths, thus pitch can be attributed to these ‘noise-like’ sounds. Most sounds of pitch-producing instruments consist of overtones that are proportionally related by the harmonic series (1:2:3:4:5:6:7:8, etc.), or have small deviations from the harmonic series (1:2:3.01:4.02,

72

Product Experience

etc.). The deviations from the harmonic series introduce inharmonicity. This inharmonicity has a physical cause, e.g. the stiffness in metal strings. A sound with a fundamental of 100 Hz would consist of a series of harmonics of: 100 Hz, 200 Hz, 300 Hz, 400 Hz, 500 Hz, 600 Hz, etc. This is the ‘ideal’ sound that hardly exists. It is interesting to note that the harmonic series contains the intervals of the Western tonal system. The proportion 1:2 is the octave, 2:3, the fifth, 3:4, the fourth, 4:5 the major third, and 5:6 the minor third. In terms of pitches, if the fundamental tone is C then the series of overtones (until the 6th harmonic) will be the following: C, C, G, C, E, G. This series is often considered as the underlying principle of Western tonal harmony (see, e.g. Parncutt, 1989). The number of overtones between the harmonics determines how noiselike a sound is (the amount of tonalness decreases). This can be captured by the harmonicsto-noise ratio (see, e.g. Boersema and Weenink, 2006). It is also worthwhile to note that the onset (start) of a musical tone often introduces noise-like elements that are important in people’s judgment on the naturalness and quality of a sound (e.g. Fletcher, 1962). Musical sounds – and even melodies – are considered to be product sounds if they have been implemented in industrial products. For example, sounds in user interfaces, alarm sounds, and start-up sounds in computers are considered product sounds.

4. PRODUCT SOUNDS As mentioned before, product sounds are a perceptual subdomain of environmental sounds and also comprise the use of musical sounds (musical instruments or synthesized sounds). In most instances, synthetic sounds are implemented in the way they have been designed; they will not change due to product–user interaction. As a result the sound will be the same every time it is triggered. Although this simplifies the design of these sounds, the experience of the sound will be affected. The reason for this is that a perfectly designed sound will evoke a desired experience. However, in this case the experience of the sound will also depend on the repetition by which it is heard. A repeated sound that remains perceptually equal will come to be experienced more negatively. This appears to be in conflict with the mere exposure effect reported by Zajonc (1968). However, if one considers the nature of user interface sounds then the following explanation may be given. User interface sounds are often short, of a simple structure, and are repeated many times. In a meta-analysis of the relevant literature, Bornstein (1989) found that the mere exposure effect is stronger for complex stimuli, for multiple stimuli, and for brief stimuli that are not repeated that often. Most user interface sounds have just the opposite features and may therefore be disliked with repetition. Consequently, a sound designer should consider introducing small deviations in the spectral-temporal structure of the sounds. These fluctuations will result in minor differences between repetitions, but nonetheless all sounds will be perceived as similar to the original sound. The similarity will evoke the familiar experiential properties, whereas the deviations will avoid boredom or irritation. In the next paragraph, the spectral and temporal structure of product sounds will first be discussed without the influence user actions have on the sound. Then the effect of the user on the sound production will be discussed, and its consequences on the sound properties.

4.1. Spectral and temporal structure of product sounds The amount of structure in both the temporal and spectral domain of a product sound determines how well a sound can be recognized and remembered (Özcan and

The experience of product sounds

73

Van Egmond, 2007). Sounds with no-structure in the spectral domain will sound very noise-like. In this case, the sound will be hard to manipulate to achieve a proper experience. The lack of structure in the temporal domain (e.g. absence of a strong perceived periodicity in sounds with an engine or a hierarchical rhythmic structure in alarm sounds) will make it harder to encode a sound. If a sound is highly structured in both domains recognition and memory are high, whereas for sounds that are unstructured in both domains recognition memory will be low. Özcan and van Egmond (2007) distinguished three levels of structure in the spectral domain and three levels of structure in the temporal domain. In the spectral domain, a highly structured sound consists of an (harmonic) overtone series, and therefore will sound tonal. These sounds almost do not contain noise (or noise that is relatively low in intensity compared to the overtones). Medium structured sounds in the spectral domain stem from products that contain an engine with a certain rpm (revolutions per minute). The rpm will result in a perceivable periodicity in the sound that generates an (harmonic) overtone series. However, these sounds consist of a considerably large amount of noise caused by the moving of, for example, gears. In other words, the intensity of the noise is relatively high compared to the intensity of the overtones generated by the periodicity. Sounds that only consist of noise or (rapidly) changing spectral structure over time are the least structured. Not only will the memory and recognition decrease with the amount of structure, but the possibility of designing a sound experience for a product will be much more difficult if structure is lacking. We have also distinguished three levels of temporal structure for product sounds. Product sounds that are highly structured in the temporal domain will have a distinctive (hierarchical) rhythm-like structure, and will therefore be more easily remembered (see Povel and Essens, 1985). An example is the sound of an alarm clock with a distinctive rhythm. Products with engines that run at a certain rpm will have a cyclical temporal structure. These sounds are considered medium structured. The rpm causes a periodicity – one experience the cyclic temporal structure – but this periodicity produces continuous events, not discrete events in the temporal domain. In contrast to discrete events, continuous (cyclic) events in the temporal domain will not evoke a strong rhythmic pattern. Products that operate with airflow or water-flow will produce sounds with the least recognizable temporal structure. The aforementioned descriptions of the spectral and temporal structure of product sounds are without the influence of user actions. In other words, these are products that run stationary. The user actions will influence the spectral–temporal structure of a product sound and will be determined by the type of control system. Three control systems can be distinguished: Autonomous; partly autonomous; and user-dependent control systems. The comparability of the sounds produced by similar events decrease in the following way: Autonomous; partly autonomous; and user-dependent. The comparability of the sounds of similar events will affect the possibility of making proper predictions concerning experiential aspects. Products that have autonomous control systems (e.g. central heating systems) will produce sounds that are very similar (equal) if events are repeated. In fact, minor fluctuations in the sound will be more dependent on the external factors of the environmental context than that a user action will influence the sound. Consequently, the experience of the sound will be highly predictable. For example, a central heating system will be dependent on the temperature of the surrounding context that influences the time of heating and the temperature of the water that needs to be heated. The spectral–temporal composition of the sound of a product with a partly autonomous control system will be dependent on user actions. However, the deviations in the

74

Product Experience

spectral–temporal composition will be small. Consequently, the experience of the sound will still be predictable within small perceptual boundaries. For example, the sound event of a drip filter coffeemaker will be dependent on the amount of coffee it has to brew, how the filter is placed, etc. However, when the coffeemaker is switched on it will run autonomously. The spectral–temporal composition of products with user-dependent control systems can vary considerably. Two manifestations of sound can be discerned: (i) The stationary sound (i.e. the sound a product generates when it is switched on but not used); (ii) The sound when the product is in use. The experience of the sound of these products in use and when they are stationary differ. For a company this may mean that if they want to design the shelf-presence of a product sound then they should consider the experience of the sound when the product is switched on without any user action. For example, a toothbrush can often only be switched on and cannot actually be used in a store. The design of the stationary sound is easier than the design of a sound that pertubates perceptually under the influence of usage. There are two main reasons for this. First, the spectral composition of the sound will change when the product is used (e.g. by increasing the amount of pressure on a toothbrush). Differences in spectral compositions may evoke different experiences. Consequently, it will be more difficult to design the preferred sound experience. Secondly, user actions are not systematic and will generate random fluctuations that are harder to predict, and therefore more difficult to incorporate in the sound design. Three examples of product sounds will be given. In Figure 3.1 the spectrogram of an alarm clock sound is presented. In the spectrogram frequency as a function of time is represented with grey shading to indicate the intensity of the frequencies. It can be seen that a tone with duration of approximately 350 ms is repeated for four times (i.e. discrete events). The pitch of this sound is approximately 890 Hz, it can be seen that there is a strong emphasis for the sixth overtone (around 5300 Hz, darkest shading). In addition,

Figure 3.1 Spectrogram of an alarm clock sound. Time has been represented on the x-axis. Frequency has been represented on the y-axis. Intensity of overtones has been indicated with shades of gray.

The experience of product sounds

75

the ninth through the fourteenth overtones also have a relatively strong intensity. This indicates that the sound will be perceived as somewhat sharp, because the sensation of sharpness depends on the amount of energy in the higher frequencies. Consequently, the sound may be perceived as relatively unpleasant. This is, of course, a desirable experiential property for an alarm sound if one needs to wake up. In Figure 3.2 the spectrogram of the sound of an electric toothbrush is presented. It can be seen that there are no discrete events; the sound events are continuous. It can also be seen that the sound contains noise, because hardly any frequency bands can be distinguished. Nevertheless, in Figure 3.2 some darker frequency bands can still be distinguished, caused by the periodicity of the engine. The user involvement is reflected in this figure, especially if one considers the dark band in the spectrogram around 2500 Hz. This band fluctuates over time due to the pressure the user applies to the brush. With an increase of pressure the frequency decreases, whereas if the pressure decreases the frequency increases. One can hear this in the perceived pitch of the sound. This demonstrates that there is a certain bandwidth in which the spectral content of the sound fluctuates. Consequently, the experience of the sound may be dependent on the amount and the strength of these fluctuations. In a pilot study, we investigated the effect of pressure on the preference and experience of a toothbrush sound. In this study, the pressure was systematically varied by putting weights on the head of the toothbrush. Thirty participants were asked to indicate which sound they preferred the most using a paired comparison paradigm. Preference was negatively associated with the amount of pressure (r .92). In other words, if the pressure increases then the preference decreases. Zampini, Guest and Spence (2003) have investigated the effect of a toothbrush sound on the experience of the vibrotactile stimulation on teeth. They have found that if the loudness level or the intensity of the higher frequencies decreases the ‘tactile’ feeling of roughness and unpleasantness of the toothbrush also decreases.

Figure 3.2 Spectrogram of the sound of an electric toothbrush. Time has been represented on the x-axis. Frequency has been represented on the y-axis. Intensity of overtones is indicated with shades of gray.

76

Product Experience

Figure 3.3 Spectrogram of the sound of a vacuum cleaner. Time has been represented on the x-axis. Frequency has been represented on the y-axis. Intensity of overtones has been indicated with shades of gray.

In Figure 3.3 the spectrogram of the sound of a vacuum cleaner is shown. It can be seen that there is even less structure in the frequency domain. In addition, the energy of the sound appears to be more concentrated in the lower frequency range, whereas for the toothbrush the lower frequencies do not contain that much energy and the higher frequencies are relatively high in energy (note that the sounds have all been normalized for intensity to the same sound pressure level). In Figure 3.3 vertical lines have been drawn to identify the areas in the spectrogram in which changes occur. In these areas, the dark bands in the spectrogram are interrupted and the amount of energy for the higher frequencies increases (this area becomes darker). This change results from the blocking of the nozzle of the vacuum cleaner when the hose is moved back and forth. In conclusion, consequential product sounds are often noisy. This noise (random structure) complicates the design and predictability of these sounds. The spectral– temporal composition of the sound is not only dependent on how the product has been constructed, but in addition is dependent on the usage actions (e.g. pressure). The structural aspects of intentional sounds (e.g. feedback or alarm sounds) can be designed, and thus the experience of these sounds is largely under control (not withstanding that contextual aspects may also influence these sounds). In addition, the improved technology nowadays allows the generation of high quality (CD standard) sounds. However, in many cases the design of the sounds has not been explicitly based on the type of interaction the designer wanted. In comparison with consequential sounds there is so much design freedom (e.g. timbre, pitch, inter-onset intervals) that it becomes difficult to make choices. From ecological acoustics (e.g. Gaver, 1993a,b) it is known that sounds often convey a meaning. That is, one can hear the type of source, the location of the sound, the context in which the sound is present, and whether the source of the sound is dangerous. Therefore, it is important that sounds in user interfaces are designed to convey a specific meaning. Alarm sounds in intensive care units are a ‘good’ example of improper – or not well-thought-through – sound designs.

The experience of product sounds

77

Figure 3.4 Two spectrograms of alarm sounds of a device that controls a patient’s breathing from the same manufacturer.

The use of alarm sounds in intensive care units generates many problems (Edworthy, Loxley and Dennis, 1991; Edworthy and Meredith, 1994; Welch, 1999; Yan Xiao and Seagull, 2003). Manufacturers often implement alarm sounds because they want to reduce the probability of liability law-suits (Edworthy and Adams, 1996). Because alarms go off unnecessarily and too often, staff in intensive care units often turn the alarms off. Because they are occupied with coping with their annoyance, they forget to attend to the actual problem (e.g. Block, Nuutinen and Ballast, 1999; Edworthy and Adams, 1996; Momtahan, Hetu and Tanssley, 1993). The large number of sounds, the lack of conveying a well-defined meaning (e.g. type of device, the level of urgency), and the masking effects by other alarms and noise-producing sources make alarm sounds very difficult to remember and to recognize (Edworthy and Adams, 1996; Momtahan, Hetu and Tanssley, 1993; Welch, 1999). Often alarms that need to convey a certain meaning (e.g. cardiac arrest) may differ between devices. This finding is not only true for devices of different manufacturers, but also for devices from the same manufacturer. For example, we recorded the sound of a device that controls the breathing of a patient. If the pump stops then this alarm sounds. In Figure 3.4 spectrograms of sounds are shown that stem from two versions of this device from the same manufacturer. It can be readily seen that the sounds differ in their spectral composition. In addition, the spectrogram depicted on the left consists of the same tones that are repeated. The tones have a duration of 250 ms followed by a pause of 250 ms. The spectrogram depicted on the right consists of five tones. The arrows indicate at what point the new tone starts. It can be seen that there is a small overlap of the tones (the next tone starts while the previous tone is still sounding). Furthermore, the temporal structure of the second sound is more rhythmic than of the first sound. This rhythm can be characterized as: short-short-longshort-long. After the last tone has sounded, a pause starts in which the decay of the last sound can still be heard. After 3 seconds this rhythm is repeated again. It can be concluded that the left and the right sound do not contain any similarity. Consequently, staff of the intensive care unit will have to remember two different sounds that correspond to the same cause. This example demonstrates that a lot can still be improved in the consistent design of alarm sounds.

5. PROCESS OF AUDITORY PERCEPTION Several theories and models exist that describe the process of auditory perception or specific aspects in this process (e.g. Bregman, 1994; Handel, 1991; McAdams, 1993; Yost, 2007;

78

Product Experience

Fastl and Zwicker, 2007). I have been developing a conceptual framework for the process of auditory perception that guided my thinking and the set up of research projects. In this conceptual framework, I make a distinction between experiential aspects of sounds that stem from a bottom-up process and experiential aspects of sounds that stem from top-down processing (or ‘knowledge-rule based aspects’). This distinction is coarse, and I am aware of the many subtleties that have been reported in the literature concerning the interaction of bottom-up and top-down processing of sound. However, the distinction is of importance in the experience of product sounds because it allows pinpointing certain effects as a result of bottom-up processing which are salient and unaffected by prior knowledge. Figure 3.5 represents a flowchart of the conceptual framework. On top a sound has been symbolically represented in the temporal domain. The sound will be modified by

FIGURE 3.5 Conceptual framework of auditory perception.

The experience of product sounds

79

the outer and middle ear, but the main processing occurs in the inner ear. The middle ear functions as a half-open organ pipe with a resonance frequency between 2 kHz and 5 kHz. This enhances our sensitivity for this frequency range. The sound is processed in the inner ear by the cochlear mechanics and neural activity (see, e.g. von Békésy and Rosenblith, 1951; Yost, 2006). The inner ear (with the basilar membrane as an important constituent) can be considered as a low pass filter. The higher frequencies are mapped at the start of the cochlea, whereas the lower frequencies are mapped at the end of the cochlea. Thus, each region in the inner ear is sensitive for a certain frequency. The inner ear decomposes a sound into its constituent frequencies. In a way, this resembles the mathematical technique of the Fourier transform in which a signal in the temporal domain is transformed into the frequency domain (as a result the constituent frequencies can be determined). The spatial mapping of frequency is projected on the brain through the auditory nerve. This is called a tonotopic representation of the sound. Because of the cochlear composition (fluids and organ of Corti cause integration of single resonances) there are areas of resonance. Critical bands are areas in which the summation of loudness and masking effects take place (see Fletcher, 1940; Fastl and Zwicker, 2007). For example, frequencies within a critical band contribute less to the perceived loudness than frequencies outside a critical band. Furthermore, the understanding of the masking of tones (a tone with a certain intensity can be made inaudible by another tone, noise-band, or noise) made it possible to develop coding techniques like mp3. In addition to loudness, sharpness, harmonicity, and roughness can be explained by our understanding of the functioning of the inner ear. These measures are sensations that contribute to the sensory pleasantness of a sound. Many of the psychoacoustical measures (see for an overview, Allen, 1999; Fastl and Zwicker, 2007; Hirsh and Watson, 1996; Moore, 2003; Yost, 2007) have been implemented in algorithms for sharpness, harmonicity, loudness, roughness, and tonalness. Aures (1985) and Fastl and Zwicker (2007, p. 245) proposed models of sensory pleasantness on the basis of these values. The models differ in the weighing of the factors. An increase in the value of loudness, sharpness, and roughness all result in a decrease of sensory pleasantness. An increase of tonalness (named tonality by Fastl and Zwicker, 2007, but this is to my opinion too much related to the musical concept of tonality and causes confusion) results in an increase of sensory pleasantness. Tonalness can best be described as the amount of harmonic structure in a sound in relation to the amount of noise. Because psychoacoustic experiments demand a large amount of experimental control, the sounds employed to determine these effects are often synthesized (e.g. frequency modulation, amplitude modulation, noise bands). As mentioned before, psychoacoustical measures are often used because these measures can be implemented in algorithms (see Keiper, 1999). These implemented psychoacoustics measures will be referred to as psychoacoustic metrics. Lim (2001) has used psychoacoustic metrics to predict the quality of the sound of power windows. Västfjäll et al. (2002) have found that these metrics do not predict whether a sound is related to the pleasantness and activation dimensions of emotional experience. In a graduation project (Bouter, 2006) we have investigated if the psychoacoustic metrics of roughness, loudness, sharpness, and harmonicity (an implementation for tonalness) are associated with participant responses on self-report scales of roughness, loudness, sharpness, and harmonicity for product sounds. The metrics of loudness, sharpness and harmonicity were highly correlated with the constructs (r .82, r .97, r .89, respectively), whereas the correlation between the roughness metric and the corresponding construct was not that high (r .47).

80

Product Experience

These auditory sensations are important in our understanding of the experience of product sounds because they are very difficult to suppress. In other words, if these factors are perceptually salient their effect is hard to change by an attribution of meaning. Top-down influences (the next step in the model) may influence but will not completely override these primary sensations. Thus, a proper sound design should always control for these primary sensations. Consequently, if these primary sensations are under control then top-down influences may contribute more to the experience of the sound. This will eventually lead to a richer sound experience. In the last stage of the conceptual framework, information from memory influences the processing of a sound. This information may consist of the attribution of meaning, but it can also consist of more abstract structural aspects that describe a sound (e.g. the increase and decrease pattern in pitch). For example, if one of the alarm sounds of Figure 3.4 is coded in memory, then several aspects are represented. First, the timbre of the sound (audition-specific image) should be coded. Secondly, the temporal structure (the rhythmic pattern) will be represented. Thirdly, the name of the medical device that produces the sound is represented. Fourthly, the type of medical failure and the level of urgency associated with the sound will be represented (attribution of meaning). Fifthly, the context in which the sound is heard (intensive care unit) is important. When the sound is heard and encoded on a subsequent occasion, these aspects (if there is recognition) will be activated and will be combined with the sensory sense of urgency (caused by the tempo, the repetition) that will evoke the necessity for a staff member of the intensive care unit to act. Note that the combination of top-down and bottom-up sensations that the sound makes evokes a need to act. If somebody who is not a staff member hears the sound but is not present in the intensive care unit, a sense of urgency may be induced by the rhythmic nature of the sound, but no knowledge is available concerning the actual failure. Therefore, the need to act will not be induced. Note that there is a difference between evaluating a sound and experiencing an emotion while listening to the sound. Nevertheless, Västfjäll et al. (2002) have demonstrated that affective reactions and affective evaluations are associated. Thus, although this distinction is important from a theoretical point of view, it may be difficult to measure experimentally. Possibly the context and the attribution of meaning are important to induce a difference in experience and in evaluation. However, these aspects have not been controlled in the study of Västfjäll et al. (2002). Thus far, the sensation-driven and more cognitively induced experiences have been discussed. However, is it possible to enhance our understanding of how these aspects interact and what type of emotions they evoke? In addition, which aspects in product sounds contribute to the sensations and which contribute to the cognitive aspects? In other words, can we gain more insight into where in the process of auditory perception certain perceptions or emotions occur? Several studies have shed light on these issues. For example, for the recognition of the rhythmic structure a mental frame of reference needs to be induced (see e.g. Povel and Essens, 1982), suggesting that it involves topdown knowledge. On the other hand, the perception of the tempo will be more bottomup driven. Many studies have investigated affective reactions to acoustic stimuli using rating scales (Bradley and Lang, 2000; Edworthy, Hellier and Hards, 1995; Meunier, Habault and Canevet, 2001; Solomon, 1959; Von Bismarck, 1974a,b). Rating scales have been successfully employed in many studies to obtain emotional responses and convey the intensity of the evoked emotion. However, they do not convey where the evaluation took place in the perceptual process. Thus, these scales cannot be used to differentiate between basic and more cognitive emotions. Basic emotions are the outcome of a simple appraisal.

The experience of product sounds

81

They are needed for survival purposes and are often directly related to an action (e.g. Izard, 1977; Plutchik, 1980). For example, the emotion ‘fear’ that is the outcome of an appraised ‘physical threat’ will result in the action to flee. More complex, or evolved, appraisals evoke ‘cognitive emotions’ (see Ortony, Clore and Collins, 1988). An example of a cognitive emotion is inspiration, which requires an appraised ‘mental illumination’ (see Desmet, 2002). This knowledge is essential if one wants to design a product sound, because it will determine which features in the sound to manipulate. For example, to evoke basic emotions manipulating aspects like, for example, roughness or sharpness will be important, whereas for the more cognitive emotions attribution of meaning will be more important. It is hypothesized that the distinction between basic and cognitive emotions results from a difference in processing. The more cognitive emotions will need more processing time because more information (e.g. structural aspects, association of meaning) from memory has to be accessed. It is hypothesized that the basic emotions will rely more on the sensations discussed before. If the two emotion types are measured using verbal selfreports, the meaning of these emotion words needs to be accessed in memory. In addition, the bottom-up processing of the sounds (i.e. the sensations) will be independent of the type of emotion. Thus, if a sound is evaluated on a basic emotion, then long-term memory is only accessed once. However, in order to evaluate the more cognitive emotions, longterm memory in the conceptual framework needs to be accessed twice: Once to access the meaning of the words, and once to access the structural aspects, associations of meaning of the sounds. Consequently, the processing time will take longer. Van Egmond, Desmet and Van der Helm (2004) have tested this hypothesis by using different sounds and letting listeners evaluate these on ‘basic’ and more ‘cognitive’ emotions. Rating values and response times have been collected in this study. Responses to the ‘basic’ emotions were significantly faster than responses to the ‘cognitive’ emotions. This study shows that response times can be successfully used to differentiate between emotion types.

6. DESIGNING THE EXPERIENCE OF CONSEQUENTIAL PRODUCT SOUNDS Just as one designs visual concepts for products and tests the experience of these concepts, there is a need to design auditory concepts that a ‘future’ product will produce. Several studies have investigated perceptual attributes for sound design and have demonstrated the usefulness of sound design for product sounds (e.g. Bowen and Lyon, 2003; Collins, 2001; Jordan and Engelen, 1999; Louwers, 1993; Lyon, 2000, 2003). Consequential sounds are a result of how an appliance has been constructed. Thus, the sound is dependent on, for example, the type of engine, the type of material, and the gears. Therefore, if a designer wants to create a consequential sound in the conceptual phase it will be a difficult task, because the sound needs to be realistic without the product being there. For example, if one wants to design the sound of an electric toothbrush one needs to model or to mimic the new product. This means that knowledge of the construction, the engine, the gears and the material has to be implemented in order to create a new sound. Gaver (1986) has suggested that sounds in user interfaces (i.e. intentional sounds) should afford certain actions. Therefore, these sounds should be caricatures of natural sounds (named auditory icons) because this will facilitate conveying the meaning of a sound. Gaver (1993a,b) elaborated on these ideas more extensively. He proposes not only involving the perceptual and spectral aspects in our understanding of environmental sounds, but also understanding the physical processes and mechanisms that cause the

82

Product Experience

sounds to sound as they sound. Furthermore, he has identified that the type of interaction of an object with the environment is important in our understanding of the produced sound. Therefore, several interaction types have been discerned (scratching, scraping, bouncing, etc.). Lyon (2000) has modeled several vibrating systems and has related them to perceptual judgment in order to obtain predictions on the sound quality of products. The question still remains: How realistic are these synthesized sounds? In a number of studies and projects for industry we have proposed a design method for producing sounds on the basis of recordings and manipulations of recorded parts of product sounds, and we proposed to make this part of the design process (Özcan and Van Egmond, 2006; Van Balken, 2001; Van Egmond, 2006). This methodology consists of four stages. In the first stage the sounds are recorded. Secondly, the sounds are analyzed (spectral-temporal, psychoacoustical, and perceptually). Thirdly, a realistic sound design concept is proposed based on preferred experience, and the sounds are designed. In the fourth stage, the sounds will be tested with respect to how successful the design is in relation to the required sound experience. These stages can be recursively applied if the sound experience is not as desired. In the next paragraphs the different stages will be discussed.

6.1. Recording One of the most important aspects in the recording of sound is the choice and the placement of a microphone, or the use of even more than one microphone. In music recordings, knowledge is available on how to record different instruments. This has resulted in standards of which microphones to use and where to place them (see Dickreiter, 1984). However, this knowledge is not easily applicable to the recording of product sounds. Although one can learn from the way musical instruments are recorded (e.g. where to place a microphone for a wind instrument or where to put the microphone for a relatively large instrument), the placement and the choice of microphones for the recording of product sounds is often a combination of common sense and trial-and-error. In addition, the sound level needs to be measured in order to know the loudness of sound. A standard calibrated sound level meter is appropriate to measure the sound before recording it (e.g. Bruel and Kjaer Type 2260). As mentioned before, usage actions may influence the sound of an appliance. Therefore, the sound should be recorded with, and without, user involvement. In addition, the separate sound-producing parts of the appliance will be recorded separately. This allows an analysis of the separate sound sources that will enable one to determine which aspect (or part) contributes to a certain wanted or unwanted experience. Other candidates for sources (replacement) are also recorded. These sources should offer the same functionality but achieve this functionality in another way. For example, one could use a boiler to warm water, but also a heating element. These will all be used in the design phase to generate sound concepts.

6.2. Analysis In the second stage, several methods of analysis will be conducted on the sounds using implemented algorithms, in addition to an analysis by the investigator’s or designer’s ear. The spectral–temporal composition of the sound will be determined and will be related to the sound-producing parts. This process will be exemplified with the analysis and design of a coffeemaker sound (Van Balken, 2001). In Figure 3.6 the amplitude of the signal in the time domain (upper graph) and the spectrogram (lower graph) of the recording of a coffeemaker have been depicted. The amplitude of the time signal has been normalized. It can be seen that the entire process after the coffeemaker has been switched on takes

The experience of product sounds

83

Figure 3.6 The sound of a coffeemaker represented in a time–amplitude and in a time–frequency/intensity domain (Van Balken, 2001).

approximately 100 seconds. In the graph specific events and sound producing parts have been indicated. Because the valve ticking is a very characteristic sound, this part has been magnified in the amplitude-time graph (the small click can thus be distinguished). After approximately 75 seconds the water has reached the correct temperature (approximately 95ºC) and a decay of the amplitude can be seen. Five seconds later, the pump starts and forces the water through the valve and the coffee pad. The pump sound is relatively high in intensity and will therefore evoke a sense of unpleasantness. From 85 seconds through 100 seconds the coffee flows into the cup, after which the pump stops. An event that is noticeable is the squeak that occurs approximately 2 seconds after the pump starts. It can be seen in the spectrogram (bottom graph) that this squeak contains many high frequencies of a relatively high intensity. This means that the sound will be experienced as very sharp and unpleasant. In Table 3.1 the sound events are represented as a function of the sound producing parts and use stages. It can be seen that there are nine distinctive sound events, and that most events are independent of user actions (only the opening of the lid is user dependent). In addition, most of the events occur during the brewing process. This brewing process is also the period in which multiple sources contribute to the sound (see also Figure 3.6). The next step in the analysis phase will be to determine the psychoacoustical metrics (e.g. roughness, sharpness, loudness) previously described for each of the identified events. Because sound varies over time and many different events occur, it is important

84

Product Experience

TABLE 3.1 Characteristic cofeemaker sounds with related product parts and use stages Sound

Sound producing product parts

Use stage

Squeaking Sucking Rustling Ticking Whirring Squeaking Rustling Coffee flow Rustling

Metal opening system Air escaping valve for the reservoir Boiler Valves between boiler and spout Pump Air driven through the hole in pad holder Brew chamber Spout Boiler

Opening lid Placing the reservoir Heating Heating Brewing Brewing Brewing Brewing Post-brewing

Figure 3.7 Top figure: Aggregated pleasantness ratings as a function of time (in seconds). Bottom figure: The sound scenario of Figure 3.6 (Van Balken, 2001).

to know which of the events results in a – for example – unpleasant experience. In order to relate sound events to a judgment of pleasantness, an online measurement task is used. In this task a listener is asked to indicate the experience of the sound (e.g. how pleasant) while listening to it. A slider device consisting of a linear potentiometer is employed to determine the rating value and the time on which the rating occurred. A computer samples the responses on the slider with a certain frequency that depends on the needed temporal resolution. In this way the rating data can be mapped onto the sound. With this device fluctuations in people’s judgment can be tracked, and the boundaries can be determined at which a judgment changes. In Figure 3.7 an example of this type of response has been represented. In the bottom part of this figure an abstract representation of the sound events as they can be seen in Figure 3.6 is given (note that we shortened the duration of the first part of the heating process in the recording). In the top figure the aggregated number of times (over listeners) people scored in one of the five categories (pleasant, fairly pleasant, neutral, fairly unpleasant, unpleasant) has been presented as a function of time. It can be readily observed by comparing the top and bottom graph that the heating process has been judged overall fairly pleasant, whereas as soon as the pump starts the judgment becomes unpleasant. Note that because the squeak is very short in relation to people’s behavioral reaction of moving

The experience of product sounds

85

Figure 3.8 The sound events of a redesigned sound of a coffeemaker (Van Balken, 2001).

the slider, this event cannot be found in the pleasantness ratings. However, additional interviews have indicated that this has been perceived as a very unpleasant event. In the next stage the results of the first two stages will be used to come up with a different sound scenario and a concept for a sound design.

6.3. Concept and sound design phase After the problems with the original sound have been determined, the sound needs to be improved. In this phase several scenarios and sound design concepts are created that should evoke the desired experience. The experience will be determined by the usecues one wants to convey (e.g. when does something start or stop) and the experiential aspects (sometimes based on a company’s brand values). In Figure 3.8 a new sound scenario for the coffeemaker is presented based on the experience of quality, warmness, coziness, and hominess. The sound events have been presented as a function of time. It can be seen that the duration is shorter than the duration of the scenario presented at the bottom of Figure 3.7 of the original sound. It is envisioned that this should evoke a sense of quality because devices are considered to be of higher quality if they are faster than similar devices (comparable to the acceleration rate of cars). In addition, the heating starts and then stops and starts again. This introduces a kind of wave-like feeling in the sound that can be considered cozier because it resembles the breaking waves on a sea-shore. The heating also continues when the pump starts, removing the small silent period in the original sound just before the pump starts. This masks the start of the pump and creates a more natural flow. In the original sound the silence before the brewing and the pump start possibly evokes a suspenseful experience that enhances the negative effect induced by the pump. Because the pump and the squeak were the main contributors to the most unpleasant reactions to the sound (see Figure 3.7), the squeak has been removed and the pump runs more regularly and starts up more gently. The new sound is synthesized using recordings of sound-producing parts, other parts than those present in the current appliance, sound filtering techniques to simulate different damping or resonant aspects, and some physical modeling algorithms (for example, to simulate air flow). It is essential at this stage to synthesize a realistic sound concept of which the constituent parts can be used in the construction of a new appliance.

6.4. Evaluation In the evaluation phase, listeners will judge the sounds in two ways. First, the sounds are evaluated using the aforementioned online measuring technique. Because this technique takes a long time, only a small selection of attributes will be used. Secondly, after

86

Product Experience

a listener has completely listened to a sound, it is evaluated by rating a larger list of attributes. In this way, the holistic experience of the sounds is measured. The attributes will reflect the desired experience. Together these evaluations will determine if the new sound design concepts fulfill the required experience. If not, then new concepts will be developed and the cycle starts all over again.

7. CONCLUSION This chapter presents an overview of the existing literature that is relevant for the understanding of the experience of product sounds and their design. It also contains a positioning to clarify what ‘good’ sound design can mean in the experience of industrial products. However, several topics have not been tackled. For example, the design of intentional sounds has not been discussed at full length, although the design of these sounds offers designers a lot of freedom. In fact, it should be possible to design these sounds in such a way that they would evoke exactly the preferred experience. In this way, the visual and auditory design could complement or strengthen the experience. Furthermore, in Section 6 the redesign of a sound has been discussed. In other words, the sound of an existing product has been improved to evoke a preferred experience. However, in the conceptual phase of the design process for entirely new products it is much harder to include sound design, because there is a lack of tools and methods. Just like sketching tools for visual design, a tool for sketching a sound would help designers in this conceptual phase. This type of tool would allow the design of a sound on the basis of the desired product experience. In this way, sound design would be an integral part in the development process for new products. Although most people are confronted with the sound of products throughout their daily life, the research into the auditory processing of these sounds, the experience they evoke and how they can be changed is limited. The challenge lies in that one has to try to integrate knowledge from several disciplines (psychology, acoustics, psychophysics, music) and to disclose this in such a way that designers can use it. The current chapter started with the question ‘Whether to be silent’ is the best to develop sounds for products. I hope it has been demonstrated in that there is more to the experience of product sounds than silence.

ACKNOWLEDGMENTS Spectrograms have been made with Praat (Boersma and Weenink, 2006). I would like to thank my colleagues at the Faculty of Industrial Design Engineering at the Delft University of Technology and those in the field of auditory perception for their discussions. Since 2003 I have worked with Elif Özcan and Rolf den Otter, this chapter would not have been possible without their contributions and discussions on this topic.

REFERENCES Allen, J. B. (1999). Psychoacoustics. In: J. G. Webster (Ed.) Wiley encyclopedia of electrical and electronics engineering, pp. 422–437. New York: John Wiley. Aures, W. (1985). Berechnungsverfahren für den sensorischen Wohlklang. Acta Acustica, 59, 130–141. Ballas, J. A. (1993). Common factors in the identification of an assortment of brief everyday sounds. Journal of Experimental Psychology – Human Perception and Performance, 19, 250–267. Ballas, J. A. (2002). What is that sound? Some implications for sound design. Proceedings of Journées Design Sonore. www.sfa.asso.fr/DS2002/procs/ballas.pdf.

The experience of product sounds

87

Ballas, J. A. and Howard, J. H. (1987). Interpreting the language of environmental sounds. Environment And Behavior, 19, 91–114. Benko, U., Petrovcic, J., Juricic, D., Tavcar, J. and Rejec, J. (2005). An approach to fault diagnosis of vacuum cleaner motors based on sound analysis. Mechanical Systems And Signal Processing, 19, 427–445. Benko, U., Petrovcic, J., Juricic, D., Tavcar, J., Rejec, J. and Stefanovska, A. (2004). Fault diagnosis of a vacuum cleaner motor by means of sound analysis. Journal Of Sound And Vibration, 276, 781–806. Björk, E. A. (1985). The perceived quality of natural sounds. Acustica, 57, 185–188. Blauert, J. and Jekosch, U. (1997). Sound-quality evaluation – a multi-layered problem. Acustica, 83, 747–753. Block, F. E., Nuutinen, L. and Ballast, B. (1999). Optimization of alarms: a study on alarm limits, alarm sounds, and false alarms, intended to reduce annoyance. Journal of Clinical Monitoring and Computing, 15, 75–83. Boersma, P. and Weenink, D. (2006). Praat: doing phonetics by computer (Version 4.5.08) [Computer program]. Retrieved December 20, 2006, from http://www.praat.org/ Bornstein, R. F. (1989). Exposure and effect: Overview and meta-analysis of research, 1968–1987. Psychological Bulletin, 106, 265–289. Bouter, P. (2006). The association between the psychoacoustic measurement and subjective measurement of loudness, roughness, sharpness, and noisiness of product sounds. Internal Report, Delft University of Technology. Bowen, D. L. and Lyon, R. H. (2003). Mapping perceptual attributes of sound to product design choices. Noise Control Engineering Journal, 51, 271–279. Bradley, M. M. and Lang, P. J. (2000). Affective reactions to acoustic stimuli. Psychophysiology, 37, 204–215. Brandl, F. K., Biermayer, W. and Pfluger, M. (1999). Objective assessment of vehicle interior noise quality. Sound and Vibration, 33, 20–28. Bregman, A. S. (1994). Auditory scene analysis: the perceptual organization of sound. Boston: The MIT Press. Carello, C., Anderson, K. L. and Kunkler-Peck, A. J. (1998). Perception of object length by sound. Psychological Science, 9, 211–214. Chambrin, M. C. (2001). Alarms in the intensive care unit: how can the number of false alarms be reduced? Critical Care, 5, 184–188. Collins, M. (2001). The sound and the future. The engine design of the Harley-Davidson. I.D. magazine, 52–57. Desmet, P. M. A. (2002). Designing emotions. Doctoral Dissertation, Delft University of Technology, Delft, The Netherlands. De Wit, M. G. M., Verbunt, J. P. M. and Vael, J. E. M. (1993). From noise and vibration to functional sound. Philips Research Bulletin on Sands, 1–4. Dickreiter, M. (1984). Mikrofon-Aufnahmetechnik. Stuttgart: Hirzel Verlag. Edworthy, J. and Meredith, C. S. (1994). Cognitive-psychology and the design of alarm sounds. Medical Engineering and Physics, 16(6), 445–449. Edworthy, J. and Adams, A. (1996). Warning design: a research prospective. London: Taylor and Francis. Edworthy, J. and Hards, R. (1999). Learning auditory warnings: the effects of sound type verbal labeling and imagery on the identification of alarm sounds. International Journal of Industrial Ergonomics, 24, 603–618. Edworthy, J., Hellier, E. and Hards, R. (1995). The semantic associations of acoustic parameters commonly used in the design of auditory information and warning signals. Ergonomics, 38, 2341–2361. Edworthy, J., Loxley, S. and Dennis, I. (1991). Improving auditory warning design – relationship between warning sound parameters and perceived urgency. Human Factors, 33, 205–231. Edworthy, J. and Stanton, N. (1995). A user-centered approach to the design and evaluation of auditory warning signals.1. Methodology. Ergonomics, 38, 2262–2280. Fastl, H. and Zwicker, E. (2007). Psychoacoustics. Facts and models (3rd Ed.) Berlin: Springer Verlag. Fletcher, H. (1940). Auditory patterns. Review of Modern Physics, 12, 47–66. Fletcher, H. (1962). Quality of piano tones. Journal of the Acoustical Society of America, 34, 749–761. Friesdorf, W., Buss, B. and Gobel, M. (1999). Monitoring alarms – the key to patients’ safety in the ICU? Intensive Care Medicine, 25, 1350–1352. Gaver, W. W. (1986). Auditory Icons: Using sound in computer interfaces. Human–Computer Interaction, 2, 167–177. Gaver, W. W. (1993a). How do we hear in the world – Explorations in ecological acoustics. Ecological Psychology, 5, 285–313. Gaver, W. W. (1993b). What in the world do we hear. 1. An ecological approach to auditory event perception. Ecological Psychology, 5, 1–29. Giordano, B. L. and McAdams, S. (2006). Material identification of real impact sounds: Effects of size variation in steel, glass, wood, and Plexiglas plates. Journal of the Acoustical Society of America, 119, 1171–1181. Grassi, M. (2005). Do we hear size or sound? Balls dropped on plates. Perception and Psychophysics, 67, 274–284.

88

Product Experience

Handel, S. (1991). Listening (2nd Ed.) Boston: The MIT Press. Hermes D. J. (1998) Auditory material perception. IPO Annual Progress Report 33, 95–102. Hirsh, I. J. and Watson, C. S. (1996). Auditory psychophysics and perception. Annual Review of Psychology, 47, 461–484. Houben, M. M. J., Kohlrausch, A. and Hermes, D. J. (2004). Perception of the size and speed of rolling balls by sound. Speech Communication, 43, 331–345. Houben, M. M. J., Kohlrausch, A. and Hermes, D. J. (2005). The contribution of spectral and temporal information to the auditory perception of the size and speed of rolling balls. Acta Acustica United with Acustica, 91, 1007–1015. Ih, J. G., Lim, D. H., Shin, S. H. and Park, Y. (2003). Experimental design and assessment of product sound quality: application to a vacuum cleaner. Noise Control Engineering Journal, 51, 244–252. Izard, C. E. (1977). Human emotions. New York: Plenum. Jekosch, U. (2004). Basic concepts and terms of ‘quality’, reconsidered in the context of product-sound quality. Acta Acustica United with Acustica, 90, 999–1006. Jordan, P. W. and Engelen, H. (1999). Sound design for consumer products. Paper presented at the Proceedings of Stockholm, Hey Listen! Stockholm. Keiper, W. (1997). Sound quality evaluation in the product cycle. Acustica, 83, 784–788. Keiper, W. (1999). Psychoacoustics in industry: Needs and benefit. Acustica, 85, 665–666. Kidd, G. R. and Watson, C. S. (2003). The perceptual dimensionality of environmental sounds. Noise Control Engineering Journal, 51, 216–231. Kunkler-Peck, A. J. and Turvey, M. T. (2000). Hearing shape. Journal of Experimental Psychology – Human Perception and Performance, 26, 279–294. Lim, T. C. (2001). Correlations between deficiencies in power window systems influencing sound quality and some psychoacoustic metrics. Applied Acoustics, 62, 1025–1047. Lindeman, H. E. and Stekelenburg, M. (1981). Lawaai (Noise). Stafleu’s Wetenschapelijke Uitgeversmaatschappij. Louwers, M. T. (1993). Introductory research on the effects of operational sounds on the quality perception of consumer products. IPO Report 932, Eindhoven. Lyon, R. H. (2000). Designing for product sound quality. New York: Dekker. Lyon, R. H. (2003). Product sound quality – from perception to design. Sound and Vibration, 37, 18–23. Marcell, M. E., Borella, D., Greene, M., Kerr, E. and Rogers, S. (2000). Confrontation naming of environmental sounds. Journal of Clinical and Experimental Neuropsychology, 22, 830–864. McAdams, S. (1993). Recognition of sound sources and events. In: S. McAdams and E. Bigand (Eds.) Thinking in sound: the cognitive psychology of human audition. Oxford: Oxford University Press. Meunier, S., Habault, D. and Canevet, G. (2001). Auditory evaluation of sound signals radiated by a vibrating surface. Journal of Sound and Vibration, 247, 897–915. Momtahan, K., Hetu, R. and Tanssley, B. (1993). Audibility and identification of auditory alarms in the operating-room and intensive-care unit. Ergonomics, 36, 1159–1176. Moore, B. C. J. (2003). An introduction into the psychology of hearing (5th Ed.) Amsterdam: Academic Press. Ortony, A., Clore, G. L. and Collins A. (1988). The cognitive structure of emotions. Cambridge: Cambridge University Press. Özcan, E. and Van Egmond, R. (2006). Product sound design and application: overview. In: M. Karlsson (Ed.) Design and Emotion 2006 Conference Proceedings, pp. 1–19. Göteborg: Chalmers University. Özcan, E. and Van Egmond, R. (2007). Memory for product sounds: The effect of label type. Acta Psychologica, 126, 196–215. Park, C. W., Lee, S. I. and Lee, S. J. (2004). Effect of suction nozzle modification on the performance and aeroacoustic noise of a vacuum cleaner. Ksme International Journal, 18, 1648–1660. Parncutt, R. H. (1989). Harmony: A Psychoacoustical Approach. Berlin: Springer Verlag, Plutchik, R. (1980). Emotion: A psychoevolutionary synthesis. New York: Harper and Row. Povel, D. J. and Essens, P. (1985). Perception of temporal patterns. Music Perception, 2, 411–440. Rittmueller, S. P., Mann, J. A. and Holger, D. K. (1997). Iterative design method to reduce the exhaust and venting noise of a vacuum cleaner motor with minimum use of absorptive foam. Noise Control Engineering Journal, 45, 63–68. Solomon, L. N. (1959). Search for physical correlates to psychological dimensions of sounds. Journal of the Acoustical Society of America, 31, 492–497. Solsona, J. F., Altaba, C., Maull, E., Rodriguez, L., Bosque, C. and Mulero, A. (2001). Are auditory warnings in the intensive care unit properly adjusted? Journal of Advanced Nursing, 35, 402–406. Van Balken, J. (2001). Designing coffeemaker sounds. Exploring the experience of Senseo’s sound. Unpublished MSc Thesis, Delft University of Technology.

The experience of product sounds

89

Van Egmond, R. (2004). Emotional experience of frequency modulated sounds: Implications for the design of alarm sounds. In: D. de Waard, K. A. Brookhuis and C. M. Weikert (Eds.) Human factors in design, pp. 345–356. Maastricht, The Netherlands: Shaker Publishing. Van Egmond, R. (2006). Designing an emotional experience for product sounds. In: M. Karlsson (Ed.) Design and Emotion 2006 Conference Proceedings, pp. 1–19. Göteborg: Chalmers University. Van Egmond, R., Desmet, P., Van Der Helm, A. (2004). Basic and cognitive emotions in sound perception. In: A. Kurtgözü (Ed.) Proceedings of the Fourth International Conference on Design and Emotion, pp. 1–11. Ankara: METU. Västfjäll, D., Gulbol, M. A., Kleiner, M. and Garling, T. (2002). Affective evaluations of and reactions to exterior and interior vehicle auditory quality. Journal of Sound and Vibration, 255, 501–518. Von Békésy, G. and Rosenblith, W. A. (1951). The mechanical properties of the inner ear. In: S. S. Steven (Ed.) The handbook of experimental psychology. New York: John Wiley. Von Bismarck, G. (1974a). Sharpness as an attribute of the timbre of steady sounds. Acustica, 30, 159–172. Von Bismarck, G. (1974b). Timbre of steady sounds: A factorial investigation of its verbal attributes. Acustica, 30, 146–159. Welch, J. (1999). Auditory alarms in intensive care. In: N. A. Stanton and J. Edworthy (Eds.) Human factors in auditory warnings. Aldershot: Ashgate, 319–342. Yan Xiao and Seagull, F. J. (2003). An analysis of problems with auditory alarms: Defining the roles of alarms in process monitoring tasks. In: E. Haas and J. Edworthy (Eds.) The ergonomics of sound: Selections from Human Factors and Ergonomics Society Annual Meetings, 1985–2000. Santa Monica: Human Factors and Ergonomics Society, 43–47. Yost, W. A. (2007). Fundamentals of Hearing: An Introduction (5th Ed.). New York: Academic Press. Zajonc, R. B. (1968). Attitudinal effects of mere exposure. Journal of Personality and Social Psychology Monographs, 9, 1–27. Zampini, M., Guest, S. and Spence, C. (2003). The role of auditory cues in modulating the perception of electric toothbrushes. Journal of Dental Research, 82(11), 929–932.

4

TASTE, SMELL AND CHEMESTHESIS IN PRODUCT EXPERIENCE ARMAND V. CARDELLO Natick Soldier Center, MA, USA

PAUL M. WISE Monell Chemical Senses Center, PA, USA

1. INTRODUCTION 1.1. Importance of taste, smell and chemesthesis to product experience The oldest human senses are those that respond to chemicals in the environment. A primitive amoeba responds to a drop of vinegar placed in a Petri dish by moving away from it. Anthropomorphically, we could say that the amoeba ‘experiences’ this unpleasant ‘product’ and behaves in accordance with its negative consequences. Among mammals, the chemical senses play no less of an important role. In humans, the senses of taste, smell, and chemesthesis have become secondary to those of vision and audition, but they still play a pre-eminent role in how we experience a wide range of everyday objects, including all foods and beverages, as well as many important products in the perfume, healthcare and personal product markets. In spite of the importance of the chemical senses to human experience, they have not received the scientific attention that the ‘higher’ senses have received. However, exciting discoveries during the past decade have produced greater interest in the chemical senses. Among these discoveries have been the identification of receptor proteins that encode odorant qualities, and the discovery of basic tastes beyond those we were taught as children. These and other scientific findings have opened the doors to further discoveries that will enable a better understanding of how we experience products using the chemical senses and how we might design products that better meet the utilitarian and aesthetic needs of today’s user. Product Experience Copyright © 2008 Elsevier Ltd.

91

92

Product Experience

1.2. Chemical senses as integrated perceptual systems Scientists working in the areas of physiology, biochemistry, and sensory psychology have traditionally treated sensory systems as discrete, passive, linear systems. Although this view is still predominant in the area, from the perspective of ‘product experience,’ J. J. Gibson’s analysis of human sensory systems is worth considering. Gibson (1966) viewed the human senses as ‘perceptual systems’, i.e. active interrelated systems designed to extract information from the environment for a purpose. In the case of smell, he described this purpose as ‘detecting things at a distance by means of their odors’. In the case of taste, the purpose was ‘the control of ingestion … standing guard over eating, selecting certain things and rejecting others.’ Taken together, the primary purpose ascribed by Gibson to the chemical senses was appetitive or ‘food getting’ (Gibson, 1966, p. 138). Gibson’s notion that the human senses are information gathering systems has relevance to the study of product experience. When we encounter products, whether they are food products on a supermarket shelf or in a restaurant, perfumes in a department store, automobiles in a showroom, or apparel in a clothing store, we are engaged in the process of information gathering for a purpose. The purpose is decision-making related to purchase, consumption or use of the product. However, obtaining sensory information about a product through the senses is only part of the story. Sensory information almost always occurs within a context of other information. Food items in a restaurant occur within a context that includes the price of the item, the appearance and cleanliness of the restaurant, and the perceived nutritive value of the food. Perfumes and automobiles are perceived within a context of marketing images and sales offers, while clothing and certain personal care items are perceived within gender-based, social, and perhaps moral or religious contexts. Thus, the experience of a product must take into account not only its sensory attributes, but the situational, cognitive, social, and cultural lenses through which that information must pass before a judgment can be made regarding its goodness, pleasantness, desirability for purchase, consumption or use. In this chapter we examine the role of three sensory systems that signal the presence of chemicals in the environment, and that influence our product experiences. Taste (gustation) is the sensory system that detects chemicals dissolved in the liquids we drink and the foods we eat. Smell (olfaction) detects air-born chemicals emitted by natural, living, and synthetic products. Chemesthesis detects chemical irritants via the same skin-sense nerves that give us sensitivity to touch, temperature, and pain. These three sensory systems have different specialties, but work together to contribute to our overall experience of the products we use. We will begin by outlining the anatomical, physiological, and biochemical mechanisms that underlie chemosensory experience. This will be followed by a discussion of fundamental sensory and perceptual phenomena that operate when products are experienced through these senses. From there, we will examine how chemosensory product experiences are quantified for practical applications in industry. Lastly, we will examine the roles of non-sensory information, beliefs, expectations and product context on experience, concluding with a discussion of how age, gender, social, and cultural variables can influence chemosensory product experience.

2. TASTE 2.1. The experience of taste Taste is the sensory experience that results from stimulation of chemoreceptors located on the tongue, palate, pharynx, larynx, and other areas of the oral cavity. The human

Taste, smell and chemesthesis

93

experience of taste includes at least five distinct qualities: Bitter, sour, sweet, salty and umami taste, all of which play a significant role in product experience and in helping us to decide what potential foods and/or beverages we should ingest or reject. In addition to these qualities of taste experience, there are two other dimensions of sensory experience that apply to all sensory systems. These are the intensity or magnitude of the taste, smell, etc. and its affective (like/dislike) dimension. These three elements of sensory experience will recur throughout our discussions in this chapter. As noted in the Introduction, the chemical senses evolved as important survival mechanisms, signaling stimuli in the environment that should be avoided and those that are safe to approach and/or consume. The sensory experience of ‘bitterness’ signals the potential presence of toxic substances and triggers rejection. Sourness signals the presence of acids and can be an indicator of spoiled food or unripe fruit, which may also lead to product rejection. Sweetness, on the other hand, indicates the presence of calories needed for survival and triggers ingestion. Saltiness indicates the presence of sodium and other ions needed for nervous function and fluid homeostasis and can also trigger ingestive responses. Lastly, umami taste (savoriness), which is exemplified by the taste of monosodium glutamate (MSG), chicken broth or aged cheddar cheese, suggests the presence of amino acids, the building blocks of life, and leads to ingestion. All of the taste qualities that lead to ingestion also trigger physiological or ‘cephalic phase’ responses that prepare the body to process food, including production of stomach acid and the release of insulin (see Mattes, 1987).

2.2. The mechanisms of taste experience Oral anatomy Although taste receptors are located throughout the oral cavity, many are clustered on taste papillae located on the dorsal surface of the tongue. The most plentiful of these are fungiform papillae, which are mushroom-shaped structures distributed over the anterior tongue. Foliate papillae consist of folds located on the margins of the tongue, while circumvallate papillae are large, circular structures forming a chevron near the back of the tongue. A fourth type of papilla, filiform, is widely distributed over the dorsal surface, but houses no taste buds. Taste buds are rosebud shaped groups of cells (Smith and Margolskee, 2001). Each bud contains about 100 cells, some of which are taste receptor cells. Tastants (molecules of products in solution) pass through a small pore at the top of the taste bud and contact finger-like projections from the taste receptor cells. These projections (microvilli) contain receptor-proteins. Receptors for sweet, savory (umami), and bitter (see below) are probably expressed in different subsets of taste receptor cells, although a given cell may express multiple types of bitter receptors (Zhao et al., 2003; Mueller et al., 2005; Breslin and Huang, 2006). Accordingly, a given taste receptor cell may respond to a number of different compounds, but those compounds will tend to signal the same taste quality (though this picture may change when sour and salty receptors are identified). Receptor mechanisms for experiencing taste Over the last decade, researchers have discovered receptor-proteins for sweet, umami (savory), and bitter tastes (Lindemann, 2001; Mombaerts, 2004; Breslin and Huang, 2006). When these proteins ‘recognize’ particular chemicals, they initiate a cascade of chemical reactions in the taste receptor cell that eventually causes the cell to respond. Receptors for salty and sour tastes have been harder to pinpoint (Lindemann, 2001;

94

Product Experience

Breslin and Huang, 2006). Candidate receptors include ion channels that allow ions to pass through cell membranes. The search for sour and salty receptors continues, and may lead to greater understanding soon (see Liu and Simon, 2001; Huang et al., 2006). Individual differences in the genes that encode receptor-proteins can lead to differences in taste experience (see Breslin and Huang, 2006). Many readers will remember classroom demonstrations using the bitter substance propylthiouracil (PROP). Those who express one form of a particular bitter receptor-protein can detect PROP at much lower concentrations than those who express other forms (see Bufe et al., 2005). The former individuals prove more sensitive to certain other bitter chemicals, including compounds present in a number of healthful vegetables, and are less likely to consume such vegetables (Bell and Tepper, 2006; Dinehart et al., 2006; but also see Yackinous and Guinard, 2002). Other studies suggest that PROP-sensitive people experience greater bitterness and burn from alcoholic beverages and that this sensory response, in turn, reduces consumption (Duffy et al., 2004; Lanier et al., 2005). Still other studies suggest that sensitive children prefer higher levels of sugar in their foods, perhaps to mask the more intense bitter sensations that they experience (Mennella et al., 2005). Differences in sweet preference are also related to genetic variation in the receptor-proteins for sweet taste (McDaniel and Reed, 2004; Li et al., 2005). In short, some aspects of everyday product experience can be predicted by variation in the genes that encode taste receptor-proteins. Knowledge of variations in genes may suggest specific product formulations targeted toward particular phenotypes. For example, as technology progresses, a consumer might someday scan a card with information about his/her genes for taste and smell receptors and receive a cup of coffee with just the right balance of taste and aroma compounds to optimize his/her personal flavor experience. Processing taste receptor information into taste experience The first relay station for taste signals coming from receptors in the oral cavity lies in the brain stem (Smith-Swintosky et al., 1991; Smith and Shepherd, 1999). Connections to other areas in the hindbrain mediate very basic motor and visceral responses connected to acceptance or rejection, including characteristic facial expressions in response to bitter and sweet tastes (see Mattes, 1987) and certain cephalic phase responses, like salivation and increased production of stomach acid. In primate models, and thus perhaps in humans, projections from the brainstem go to primary taste areas in the cortex via the thalamus (Small, 2006). Research suggests that the primary taste cortex plays a role in coding taste intensity (Smith-Swintosky et al., 1991). Next, connections pass to other nearby areas (Baylis et al., 1995; Carmichael and Price, 1995). Responses in the amygdala seem to be driven by interactions among intensity, pleasantness, and familiarity (Small et al., 1997, 2003). Thus, the amygdala might also play an important role in representing stimulus salience (Small, 2006). Responses in the orbitofrontal cortex (OFC) do not seem closely tied to intensity, but different patterns of activation emerge for pleasant versus unpleasant tastes (Small et al., 2003). As will be discussed elsewhere in this chapter, the OFC seems to represent reward value, and probably plays a key role in decisions about chemosensory stimuli. The neural basis of perceived taste quality remains unclear, but the experience of quality probably begins in higher (cortical) taste areas. In these areas, quality is represented in patterns of activity across neurons, and perhaps in timing of responses (Hallock and Di Lorenzo, 2006; Lemon and Smith, 2006). The primary taste cortex and the OFC also receive input from other sensory modalities, including somatosensation and smell (Rolls and Baylis, 1994; Cerf-Ducastel et al., 2001; Savic et al., 2002). In a brain imaging study, Small and colleagues (1997) found

Taste, smell and chemesthesis

95

that both odors and tastes (presented separately) caused activation in primary taste cortex and the OFC. However, when subjects received some taste-odor pairs simultaneously, total activation exceeded the sum of the activations from either stimulus alone. This suggests a special representation in these areas for flavor (the combined effect of taste and smell). Interestingly, both physiological and behavioral studies suggest that effective combinations between tastes and smells only occur for congruent pairs, e.g., vanilla odor plus sweet taste, but not vanilla odor plus salty taste (Small et al., 1997, 2004; Dalton et al., 2000). Learning plays an important role in what sensations yield a combined effect (Diamond et al., 2005).

3. TASTE: BASIC PHENOMENA OF TASTE EXPERIENCE 3.1. Taste adaptation To understand product experience, one must go beyond genetics and physiology to look at the fundamental perceptual phenomena that underlie taste experience. One such phenomenon that occurs in all sensory systems is adaptation, i.e. the sensory experience grows weaker during sustained stimulation. Figure 4.1 shows a typical adaptation curve for taste. When a taste stimulus comes into contact with receptors, the minimum concentration of that stimulus that can be detected (its absolute threshold) is elevated. With continued stimulation this threshold concentration continues to increase until it asymptotes after about one minute. When the stimulus is removed, the threshold concentration begins to fall dramatically, returning to its baseline level after about 30 seconds. This phenomenon is the reason why your first taste of a sour ball candy can be unbearably sour, but as you continue to suck on it, it becomes less sour. Taste ‘adaptation’ also occurs when eating one product reduces the perceived taste intensity of a different product that has the same or similar taste, such as tasting a bite of a Granny Smith apple after having the sour ball candy in your mouth. This latter

FIGURE 4.1 A typical taste adaptation curve (from Matlin and Foley, 1992).

96

Product Experience

phenomenon is known as cross-adaptation. When products that have different tastes are eaten one after another, an increase in the perceived intensity of the second product may occur. One of the most well studied examples of this cross-enhancement of taste experience reveals itself in the bitter taste of distilled water. The bitterness that results from drinking distilled water is due to the fact that, normally, the salt in saliva adapts taste receptors in the oral cavity. When distilled water is consumed, the adapting saliva is removed, producing a characteristic bitter ‘water taste’ (Bartoshuk et al., 1964). In fact, distilled water can take on a variety of different tastes depending upon the nature of the pre-adapting taste (McBurney and Schick, 1971). Thus, our taste product experiences depend not only on our genetic make-up, physiology, and the product being consumed, but also on other products that the user has recently consumed.

3.2. Repetitive taste experiences and liking A similar, but different, phenomenon to taste adaptation occurs with our affective (liking) experiences following repeated exposure to a food or beverage. Cabanac (1971) showed that the pleasantness of a stimulus increases as a function of how much the stimulus satisfies a biological need. Thus, liking for a food will increase the hungrier is the individual. He termed this effect ‘allesthesia’. However, when a food is consumed repeatedly over time, its pleasantness declines relative to foods that have not been eaten (Rolls et al., 1981, 1988). This change in the affective experience of a food or beverage after repeated exposure is known as ‘sensory specific satiety’. In simple terms, it is the reason why, if you eat the same meal for several days in a row, you start to desire other foods. It is the basis for our need for variety in taste experiences. Sensory specific satiety was first viewed to be related to physiological satiation. However, since the phenomenon can occur in response to the taste of foods and beverages that are not consumed (Murphy, 1982; Drewnowski et al., 1982) it seems clear that this is a perceptual effect. Although there is evidence that some sensory attributes and, therefore, some foods/beverages are less susceptible to sensory specific satiety (Drewnowski et al., 1982; Johnson and Vickers, 1991), the phenomenon is a powerful one that shapes our daily experience of foods, how much we like them, and whether and how much we eat of them (Swithers and Hall, 1994; Hetherington and Rolls, 1996).

3.3. Combined taste experiences Most foods and beverages are comprised of a complex combination of ingredients. Each of these contributes their own unique taste to the product. So what happens to our taste experiences when these separate ingredients are combined? Most commonly, the perceived intensity of the mixed tastes will be less than the total of the perceived intensities of the unmixed tastes (Lawless, 1987; Kroeze, 1990). This phenomenon is known as ‘taste suppression’. A number of studies, including those that have shown that the suppression occurs even when different tastes are presented to different sides of the tongue, support the fact that suppression occurs in the brain and is not due to adaptation of taste receptors in the mouth (Lawless, 1979; Kroeze and Bartoshuk, 1985). However, the suppression that occurs when different tastes are combined does not always affect each taste equally. For example, if you mix certain bitter tastants with salty ones, the bitterness is suppressed, but the salty experience typically remains unchanged (Breslin and Beauchamp, 1995). Similarly, if you mix certain bitter compounds with sour ones, you can get enhancement of the taste. Breslin (1996), in a review of the interactions that occur between bitter, salty and sour tastes, concluded that: (1) salty and sour tastes

Taste, smell and chemesthesis

97

suppress each other at high concentrations, but enhance each other at lower concentrations; (2) bitter and sour tastes either suppress or enhance each other, depending on concentration and other factors; and (3) salty and bitter tastes usually suppress the bitterness, but leave the salty taste unaffected. In many food formulations, ingredients are combined that have the same taste quality. Even here interactions can occur to alter the experienced taste intensity of the product. Thus, combining MSG and 5⬘ ribonucleotides can produce enhancement of the umami taste (Rifkin and Bartoshuk, 1980). Similarly, combining high intensity sweeteners will produce enhancement of their sweetness (Ayya and Lawless, 1992; Schiffman et al., 1995). Combining ingredients with either the same or different tastes can produce a variety of interactions, depending on the taste compounds that are mixed, their absolute and relative concentrations, and other factors. The art of the food product developer is to be aware of these mutually interactive effects and to compensate for their impact in order to optimize the sensory experience of the product for the consumer. But can the quality (rather than the intensity) of taste experience change when two or more tastes are combined? On this point most research supports the notion that the components of any taste mixture are still perceived separately and that taste is, therefore, an ‘analytic’ sense. However, the data are not singular in this regard. Not all taste experience comes from foods. One common ‘taste interaction’ that many people experience is how toothpaste ‘changes’ the taste of foods after brushing. At first glance, one might think that the ‘taste’ of the toothpaste is somehow responsible. However, brushing changes a variety of factors that control the general milieu of the mouth (e.g. amount and composition of saliva, presence of minute food particles, micro flora, etc.), so the change in the taste of foods may be due to multiple factors. Further, it must be remembered that much of what we experience as ‘taste’ is really smell. On this point, a recent study of the effects of toothpaste on the flavor of foods (Allison and Chambers, 2005) showed that toothpaste did not affect the perceived taste experience of food, rather it changed the perceived olfactory attributes of the food. For the flavor of orange juice, this effect was seen for up to an hour after brushing.

3.4. Modifying taste experiences Can other sensory properties of a product influence our experience of its taste, and is it possible that our taste experiences can alter perception of other sensory properties? The answer to both of these questions is ‘yes’. Perhaps the most ubiquitous interactions with taste experiences are those that occur with odors. Research by Murphy, e.g. Murphy and Cain (1980), has shown that the overall intensity of taste-odor mixtures presented orally are perceived to be a simple additive function of the components separately. However, when samples of the odorants in solution are presented alone to the mouth, subjects perceive the experience to be one of taste, not odor. The effect, which they termed ‘taste-referred olfaction’, is illusory, because pinching the nose to block retronasal passage of the volatile compounds eliminates the effect. Similar effects have been demonstrated by others with some evidence showing that the more qualitatively similar the tastes and odors, the more likely is the effect (Frank and Byram, 1988; Dalton et al., 2000; Delwiche and Heffelfinger, 2005). Figure 4.2 is taken from research by Labbe et al. (2006), in which the investigators added either cacao or vanilla flavorants (odorants) to a cocoa beverage. As can be seen, the addition of a cocoa odor (top panel) increased both the odor intensity and the bitter taste of the beverage, while depressing its sweetness. Adding vanilla odor (bottom panel) increased aroma and also elevated sweetness at the highest concentration. Thus, the odor

98

Product Experience

FIGURE 4.2 The impact of the addition of three different concentrations of cocoa (top panel) or vanilla flavorings (bottom panel) on taste and olfactory attributes (and body) of cocoa beverage. The stars indicate significant differences at the levels specified above them (from Labbe et al., 2006).

imparted by flavorants in food can differentially affect our experiences of the product’s taste qualities. In commercial practice, the phenomenon of taste-referred olfaction is now being used to ‘flavor’ bottled water by simply embedding odorants into the bottle caps (ScentSational Technologies, LLC, Jenkintown, PA), and quite surprisingly, even imagined odors can produce illusory taste experiences (Algom et al., 1993; Djordjevic et al., 2004). In the latter study, the investigators measured the accuracy of detecting sucrose in water versus plain water during simultaneous presentation of real or imagined ham and strawberry odors. Figure 4.3 shows the results of this study. The accuracy (% correct) of detecting sweetness in the forced choice trials was enhanced in the presence of actual strawberry odor and depressed in the presence of ham odor. Nearly identical effects were found when subjects merely imagined these odors. Chemesthesis, the tactile feel of chemical irritants, can also alter our taste experiences of food products. Several researchers have shown that a rinse of either piperine or capsaicin (black or red pepper solutions) reduces the perceived intensity of sweet, sour, salty and bitter tastes (Lawless and Stevens, 1984; Lawless et al., 1985) Similarly, Gilmore and Green (1993) examined the effect of a capsaicin rinse on the perceived saltiness, sourness, and irritation of high concentration NaCl and citric acid. Their results showed a significant desensitization effect of the capsaicin on both the experienced taste intensity and irritation of these compounds. Thus, it seems that chemesthesis has broad

99

Taste, smell and chemesthesis

100

Detection of sucrose (percentage)

90

80 Strawberry Ham No odor 70

60

50 Perception

Imagery

FIGURE 4.3 Accuracy of detecting sucrose as a function of real (perception) and imagined (imagery) strawberry odor, ham odor, or no odor (redrawn from Djordjevic et al., 2004).

interactive effects with basic tastes and that cross-adaptation can occur between the irritant effects of capsaicin and other taste-related oral irritants. As far as taste experience affecting other perceived aspects of a product, research by Christensen (1980) showed that increasing the sweet taste of solutions resulted in an increase in the experience of their thickness or viscosity. One explanation often given for this ‘cross-modal’ interaction is that, through learning, people come to associate high sugar levels with greater thickness in beverages, e.g. tea sweetened with honey. However, in today’s marketplace, the prevalence of high intensity sweeteners that impart little viscosity to a solution diminishes the likelihood of such a learned association. The taste of certain foods can also alter our experience of somesthetic sensations. A common example of this is how the sweetness of sugar can reduce the experience of burning that results from eating black or red pepper spices (Stevens and Lawless, 1986; Nasrawi and Pangborn, 1989). (See Section 6 for a broader discussion of chemesthetic effects.)

3.5. Innate and early taste experiences and preference How much of our experiences and product preferences is learned and how much is innate? This is a complex question because functioning taste receptors appear in the human fetus at around 14 weeks. Thus, by birth, infants have been exposed to a wide variety of taste compounds present in the amniotic fluid. However, the work cited earlier on the characteristic facial expressions of infants in response to bitter and sweet compounds provides support to the notion that some aspects of our adult taste experiences may be hard-wired. Other research also supports this view. Desor et al. (1973, 1975) measured the amount of solution that young infants consumed and found a strong ‘innate’ preference for sweetness, but little difference in their intake of sour, salty, and bitter solutions. Desor et al. (1975) also demonstrated that adding citric acid to a sweet

100

Product Experience

solution suppressed the infants intake of the mixture, suggesting that the sour experience produced by the citric acid was innately unpleasant. In a review of the area, Cowart and Beauchamp (1990) concluded that there is good evidence of an innate preference for sweet taste and reasonable evidence supporting an innate rejection of sour tastes. While it seems that many everyday products like sour milk, bitter coffee, and sweet pastry may produce pleasurable or unpleasurable experiences as a result of factors present at birth, many other product experiences are modifiable at an early age. For example, Beauchamp et al. (1986) found that infants, who are relatively insensitive to salty tastes at birth, develop a preference for it within the first four months of life. Cowart and Beauchamp (1990) have argued that this is the result of developmental changes, not learning. Other researchers have shown that exposure to certain tastes early in life can alter our taste experiences and preferences later in life (Birch and Marlin, 1982; Harris and Booth, 1987; Beauchamp and Cowart, 1993). The reader is referred to Ganchrow (1995) for a more detailed analysis of the development of human taste experience and preference.

3.6. The effects of learning on taste experience While the perceptual and hedonic natures of many taste experiences are innate, others are learned. An example of this is the development of taste aversions. Researchers have shown that a single pairing of food with the experience of illness is enough to create a lasting aversion to the food (Garb and Stunkard, 1974; Bernstein and Webster, 1980; Pelchat and Rozin, 1982). In many cases this aversion develops within minutes, although the delay between eating the suspect food and illness can be as long as 12 hours. In one survey of college students, 57% reported to have at least one learned food aversion (Midkiff and Bernstein, 1985). Not unexpectedly for that sample population, many of these aversions were to the taste of an alcoholic beverage that had been over-consumed, resulting in alcohol-induced illness. Bartoshuk and Wolfe (1990) have suggested that the aversions that form to foods result from an association with the food’s odor, not its taste. This argument makes sense from a survival perspective, because to be of future advantage to the organism, the aversion to illness-producing foods should occur as early as possible upon encountering the food. By all accounts, taste aversion learning is a primitive form of learning (Schafe and Bernstein, 1996) that can powerfully shape our product experiences. An opposite ‘medicine effect’ in which taste experiences associated with recuperation from illness become preferred has been shown in animals, but it has been difficult to demonstrate in humans. The reader is referred to Chambers and Bernstein (1995) and Schafe and Bernstein (1996) for reviews of this area. In the case of learned taste preferences, mere exposure to novel or disliked tastes (and odors) can increase one’s liking and intake of them (Cain and Johnson, 1978; Davis and Porter, 1991; Stein et al., 2003). This form of simple repetitive exposure to tastes and smells can serve as an important strategy for overcoming neophobia (the reluctance to try new foods or other products). This is why parents, usually unwittingly, continue to offer their children foods that they dislike, e.g. vegetables, in an attempt to ‘overcome’ their disliking with time and repeated exposure. The influence that repeated exposure can have on our product experiences has been powerfully demonstrated in research by Rozin and others on the development of liking for chili pepper. In one such study, Rozin and Schiller (1980) demonstrated that simple repeated exposure to increasing levels of chili in food is all that is needed to reduce the aversion to its burning sensation and to create a preference for it. Other studies have shown that a neutral or disliked taste paired with a pleasant taste experience will increase the preference for the neutral flavor (Zellner et al., 1983; Breslin

Taste, smell and chemesthesis

101

et al., 1990; Capaldi, 1996). This flavor–flavor conditioning occurs unknowingly among coffee and tea drinkers who often initiate their coffee or tea drinking by adding liberal amounts of sugar or non-nutritive sweeteners to offset the bitterness of the coffee/tea. With time, the pairing of the pleasantness of the sweet taste with that of the taste of the coffee/ tea produces a greater liking for the taste of them alone, resulting in the gradual reduction of the added sweetener. The opposite effect of associating a non-preferred taste, e.g. bitter, with a preferred taste, e.g. sweet, has been demonstrated to reduce the liking for the preferred taste (Capaldi and Hunter, 1994). Still other research has shown that flavor preferences can be increased by associating a neutral flavor or food with nutritional consequences (calories) or with feelings of satiety (fullness) (Booth, 1981; Capaldi et al., 1987). Although some taste and hedonic experiences in the real world may be ‘innately’ determined on the basis of our genetic make-up, our needs to avoid toxic substances and to obtain calories have created the ability for our taste experiences to be modified to achieve survival goals. In Section 8 we will discuss how taste and other product experiences can also be modified by such factors as context, information and expectations. The important point to take away from the data discussed here is that our taste experiences of products result from a combination of innate, genetic, cognitive and learned factors. Product experience is only partly to be found in the product. An equally important part is to be found in the consumer and his/her past experiences.

4. SMELL 4.1. The experience of smell Smell is the sensory experience that results from stimulation of receptors in the olfactory epithelium of the nose by airborne molecules. When these molecules interact with olfactory receptors, experiences such as ‘grassy’, ‘woody’ or ‘musky’ result. Odors in the environment warn us of dangers both near, like spoiled food, and remote, like the smell of smoke from a distant fire. Odors also alert us to more pleasant prospects, such as breakfast cooking downstairs, and they play an important role in social communication.

4.2. The mechanisms of olfactory experience Nasal anatomy Information gathering in the olfactory system begins with the passive (normal breathing) or active (sniffing) arrival of airborne molecules. Upon passage into the nares, odorant molecules interact with specialized olfactory receptor-neurons located in the olfactory epithelium, a small patch of receptor tissue located high in the nasal cavity (Rawson and Yee, 2006). Sniffing is more effective for stimulating smell than natural breathing because sniffs create eddy-currents that drive odorized air up to this epithelium. In fact, some authors believe that the act of sniffing is an important unit of olfactory processing and experience, with each sniff constituting a ‘snapshot’ of the olfactory information in the environment (Kepecs et al., 2006; Mainland and Sobel, 2006). While sniffing causes the passage of inhaled air through the nares (orthonasal), chewing of food causes air from the back of the mouth to be passed upward into the nasal chamber in the reverse direction, through the nasopharynx (retronasal). When one perceives the ‘flavor’ of a food in the mouth, it is its retronasal smell that is the major contributing factor to the experience. When a cold produces nasal congestion, it interferes with the flow of air both through the nares and

102

Product Experience

retronasally, which is why our flavor experience of foods eaten when we have a head cold is so diminished. Receptors for olfactory experience Olfactory receptor neurons extend hair-like cilia into the mucous layer that covers the olfactory epithelium (Lledo et al., 2005; Rawson and Yee, 2006). Like taste receptor cells, these cilia contain specialized receptor-proteins that recognize odor molecules. Recent research has identified many of the genes that encode receptor-proteins (Glusman et al., 2001). Humans have approximately 350 potentially functional genes, which implies that we may have as many functional receptor-proteins (Rawson and Yee, 2006). Research suggests that each type of receptor-protein is expressed by many olfactory receptor neurons, but that each individual receptor neuron expresses only a single type of receptor-protein (Buck, 1996). A given receptor-protein, and therefore the olfactory receptor neuron that expresses that protein, will respond to multiple odorants, and a given odorant will stimulate multiple receptor neurons (Hildebrand and Shepherd, 1997). The overall pattern, or mosaic, of stimulated receptor neurons signals the identity of the molecule at this level of the olfactory system. Recent research on odor receptors provides new insights into consumer olfactory experience. It is clear that odor perception differs substantially among people with a ‘normal’ sense of smell. For example, the compound androstenone, which is found in products like celery, pork, and truffles, is experienced quite differently among individuals. It is described as being either urinous, musky, or floral by different people. Yet, upwards of 50% of the population cannot even smell it (Labows and Wysocki, 1984). Wysocki and Beauchamp (1984) have shown a strong genetic basis for this difference and have demonstrated the involvement of genetic, developmental, and experiential factors (Wysocki and Beauchamp, 1991). Researchers are now attempting to link differences in olfactory experiences to specific mutations in genes that encode receptors. Those who develop consumer olfactory products, like perfume, know that a single formulation will rarely please all consumers. That we differ in our mosaic of smell receptors helps explain why we each seem to live in our own world of scents. As with taste experience, it may soon prove possible to infer much about the olfactory experiences and preferences of an individual by knowing the genes that he or she expresses. Receptors for pheromones Pheromones are chemical signals released by one individual that influence the physiology or behavior of another individual of the same species (see Wysocki and Preti, 2004). Many animals can sense pheromones both with the olfactory system and with a specialized chemosensory organ called the VNO, or vomeronasal organ (see Buck, 2000). Humans lack a functional VNO, but we may express some receptors in our olfactory epithelium very similar to receptors found in the VNO of other animals (Rodriguez et al., 2000). Further, members of a newly-discovered family of receptors respond to volatile amines, including stress-related amines in urine and amines that differ in concentration between males and females (Liberles and Buck, 2006). Genes for some of these receptors can be found in human olfactory epithelium. In short, between regular olfactory receptors and other receptors that may exist, humans have adequate capabilities for sensing pheromones. It is clear that humans respond to pheromones (Wysocki and Preti, 2004), although not with the automatic, stereotyped behavior of moths that fly miles upwind to mate. A magic love potion that irresistibly draws the opposite sex seems unlikely. However, pheromones may influence mood and physiology. Male underarm extract (odor masked) placed on the upper lip makes women ‘less tense’ and ‘more relaxed’ relative to the

Taste, smell and chemesthesis

103

masking vehicle alone (Preti et al., 2003). Underarm extract from a female donor can influence release of hormones to either accelerate or delay ovulation, depending on the stage of the donor (Stern and McClintock, 1998; Shinohara et al., 2001). This finding helps explain menstrual synchrony, i.e., the tendency for women who live together to fall into register over time. In addition, underarm extract from male donors can make women more regular and accelerates release of hormones associated with ovulation (Cutler et al., 1986; Preti et al., 2003). More research is needed, but data suggest that a properly formulated olfactory product could alter both our mood and our physiology. Processing olfactory receptor information into smell experiences Signals from the olfactory receptor neurons pass to the brain for further processing (Rolls, 2001; Lledo et al., 2005; Gottfried, 2006). First, signals are sharpened in specialized structures called the olfactory bulbs. Then, signals diverge into several pathways (Price, 1973). The direct cortical targets of the signals are situated in the primary olfactory cortex. Activity in parts of the primary olfactory cortex is modulated by sniffing, again reflecting its importance in olfactory processing (Sobel et al., 1998). Other work suggests that primary olfactory cortex plays a role in olfactory learning and memory, and may be involved in separating odors of interest from steady (background) odors (Gottfried et al., 2003; Kadohisa and Wilson, 2006). Interestingly, there are direct or nearly direct connections to other evolutionarily old parts of the brain, including the hypothalamus. The hypothalamus regulates hormonal release via the pituitary gland and plays a key role in the regulation of hunger, thirst, sexual behavior, and other basic functions, The hypothalamus may play a roll in the olfactory circuits that help regulate these functions, and may modulate sensory response based on both internal and external signals (Rolls, 2001). There are also direct connections to the amygdala, which responds to interactions between intensity and pleasantness (Anderson et al., 2003). Thus, similar to conclusions regarding taste processing (see p. 94), olfactory activity in the amygdala seems to play a role in representing stimulus salience. Interestingly, personally meaningful odors are described with strong emotional language and are associated with relatively strong activation in the amygdala (Herz et al., 2004a). Contrary to popular perception, studies suggest that odor evoked memories are no more accurate than those evoked by auditory, visual, or tactile cues, but they do entail stronger emotional reactions (Herz, 1997). Thus, people may confuse emotional vividness with accuracy. From the primary cortical areas, odor signals pass to higher brain areas, either directly or through the thalamus. The orbitofrontal cortex (OFC) is one important target (see Rolls, 2001; Gottfried, 2006). As with taste, the OFC combines input from multiple sensory modalities to make evaluative ‘judgments’. Also like taste, odors that people find pleasant evoke different patterns of activity in the OFC than do odors they find unpleasant (Anderson et al., 2003). Other studies suggest that some cells in the OFC respond according to the reward-value of a stimulus (reviewed in Rolls, 2001). In other words, a given cell might respond differently to a particular odor, depending on whether or not the odor was associated with a pleasant reward. This result suggests that evaluative judgments about odors are subject to learning. Other work has shown that, relative to a vanilla odor control, OFC activation to banana odor declines after subjects have eaten bananas to satiety (O’Doherty et al., 2000). This effect is not simple adaptation, since both neural response at lower levels of processing and rated intensity might change very little. Rather, this is a case of sensory specific satiety and reflects a change in the rewardvalue of the stimulus.

104

Product Experience

5. SMELL: BASIC PHENOMENA OF EXPERIENCE 5.1. Olfactory adaptation Continued exposure to the odor of a product decreases our ability to detect it, reduces its perceived intensity, and increases our reaction time to its presence (see Cometto-Muñiz and Cain, 1995; Lawless, 1997). A typical pattern consists of rapid loss of odor intensity within 5–10 seconds, and a more gradual decline thereafter, reaching a new steady state within a minute or two. However, the rate and degree to which odors adapt varies. Intense odors are more resistant to adaptation than weak ones (Cain, 1974). Further, unpleasant odors adapt more rapidly than pleasant ones (Jacob et al., 2003). The time course of adaptation could also depend on physical properties of the molecules, related to diffusion into the mucus and eventual clearance (Dalton, 2000). Adaptation is strongest for odors identical to the adapting odor (self-adaptation), but exposure to one odor may reduce the perceived experience of another odor to some lesser degree (cross-adaptation). One might expect similar smelling odors to stimulate overlapping sets of physiological mechanisms and thus cross-adapt effectively, and dissimilar odors to cross-adapt less effectively. Some data are consistent with this notion (e.g. Todrank et al., 1991; Cain and Polak, 1992). However, molecular structure, apart from the perceived quality, seems to play a roll. For example, 3-methyl-2-hexenoic acid (3M2H), a sweaty-smelling compound important in human underarm odor, can be crossadapted by the fruity-smelling ethyl ester of 3M2H (Pierce et al., 1995). Other research suggests it may even be possible to cross-adapt an odor using an odorless molecule with similar structure (Pierce et al., 1996). Wysocki et al. (2003) identified common fragrance ingredients that cross-adapt human underarm odor. This work showed that for males, many odors cross-adapt underarm odor, but far fewer cross-adapt underarm odor for females. Thus, properties of the observer, as well as properties of the molecule, can play a roll in efficacy of cross-adaptation. Adaptation probably occurs at various levels of the olfactory system (Cometto-Muñiz and Cain, 1995; Dalton, 2000). Receptor cells adapt, but neither the magnitude nor the duration of peripheral adaptation fully accounts for human olfactory responses (Zufall and Leinders-Zufall, 1997). Further, long-term reductions in sensitivity observed with repeated exposure on the job (Dalton et al., 2003) or in the home (Dalton and Wysocki, 1996) may have quite different underlying causes than short-term adaptation seen while sampling multiple perfumes at the department store. At some level, cognitive influences play a role. We may cease to pay attention to an odor that is always present, a general phenomenon many researchers call habituation (Lawless, 1997). Interestingly, if subjects believe an odor is safe and beneficial, rated intensity will wane more over time than if subjects believe the same odor is potentially harmful (Dalton, 1996). Thus, cognitive and emotional reactions to odors may modulate their experienced intensity (see Distel and Hudson, 2001, for a discussion of these influences on perceived odor intensity).

5.2. Combined olfactory experiences The odors of most consumer products come from mixtures of odorants. Products that counteract malodors often seek to mask offending smells by creating a mixture of the masker and the offending odor. Understanding the relationship between the experience of unmixed components and the experience of their mixtures will aid in product design, optimization, and quality control.

Taste, smell and chemesthesis

105

Most research on odor mixtures has been limited to two-component mixtures. The following general rules have emerged with respect to odor strength: (1) a mixture of two odorants will generally smell less intense than the sum of the intensities of the unmixed components, but stronger than the weaker of the two components; (2) the components, when they can be individually perceived, will smell weaker than when unmixed; and (3) the stronger of the two mixture-components will reduce the intensity of the weaker component more than the weaker will reduce the stronger (Laing et al., 1984; Cain et al., 1995; Lawless, 1997). Less work has been done on more complex mixtures, but some research suggests that beyond three or four components, adding additional odors has little impact on overall intensity (Moskowitz and Barbe, 1977). In addition, interactions depend to some extent on the overall intensity of the odors. Studies have found more complete summation when the unmixed components are relatively weak (Laing et al., 1984; Cain et al., 1995). With weak odors, even synergy has been observed (Engen, 1982; Laska et al., 1990). This general trend of concentration-dependence in mixture interactions also holds true at threshold levels (Cometto-Muñiz et al., 2005). Like taste mixtures, simple mixtures of odorants can be analyzed into their components. However, while taste qualities can be discerned in five to six component mixtures (Marshall et al., 2006), complex odor mixtures often form holistic sensations, and even expert perfumers fail to correctly identify all the constituents of a mixture that consists of more than about three to five chemicals (Livermore and Laing, 1996). This phenomenon is commonly assumed to support the fact that olfaction is a synthetic sense (Burgard and Kuznicki, 1990). According to most sensory work, the perceived qualities of mixtures generally fall between those of their unmixed components, with no radically new notes emerging, though some research leaves the possibility of emergent qualities open (Moskowitz and Barbe, 1977; Jinks and Laing, 2001; Zou and Buck, 2006). In addition, some odors tend to have an impact on the quality of mixtures disproportionate to their individual perceived intensities. For example, woody odors tend to dominate fruity ones, consistent with the laments of wine makers who find that tannins have a large impact on flavor (Atanasova et al., 2005). Unfortunately, science cannot precisely predict how a mixture will smell based on the properties of the molecules that comprise it. There is no substitute for the flavorist and perfumer, who create flavors and scents using a combination of personal experience and experimentation. However, systematic studies of how molecular properties influence mixture-interactions could eventually lead to perceptual models that greatly streamline the process of formulating novel flavors and aromas.

5.3. Modification of olfactory experiences Olfactory experiences can often be modified by taste. Von Sydow et al. (1974) showed that the experience of the intensity of the pleasant odors of fruit juice increases, while the experience of unpleasant odors decreases, in response to added sugar. McBride and Johnson (1987) found enhancement of lemon flavor experience with increasing levels of sucrose and citric acid in lemon juices, while Perng and McDaniel (1989) showed that sucrose enhanced the fruit flavor intensity of blackberry juice. However, in contrast to McBride and Johnson, the latter authors found that increasing the sourness of the juice lowered the experience of its fruit flavor. Other investigators have shown that aspartame enhances and prolongs the fruitiness of fruit-flavored solutions more than sucrose (e.g. Bonnans and Noble, 1993). A number of additional studies have also shown the strong interactions of taste and smell experience in foods (Frank et al., 1993; Stevenson et al., 1995, 1999). When odorant compounds are mixed with irritant compounds, mutual suppression occurs, much like with mixtures of odors. Cain and Murphy (1980) demonstrated this

106

Product Experience

effect by mixing the nasal irritant, carbon dioxide, with concentrations of amyl butyrate. The result was a mutual suppressive interaction, i.e. increasing concentrations of carbon dioxide resulted in decreasing judgments of the odor of amyl butyrate and vice versa. Moreover, by presenting the odorant compound to one nostril and the irritant compound to the other, it was possible to show that the locus of interaction occurred centrally in the nervous system (Cain and Murphy, 1980).

5.4. Innate versus learned odor experiences and preferences Some research has found that infants and young children have odor preferences that differ from those of adults, including a relative indifference to some odors that adults find offensive (Engen, 1982, 1988). These data suggest that odor preferences are learned over time. However, other research has found support for innate odor preferences. Among these latter studies are those showing that infants, like adults, display more negative facial responses to fishy and rotten egg odors than to the odors of vanilla, banana or butter (Steiner, 1977; see also Schmidt and Beauchamp, 1988) and that very young infants display preferences for certain biologically relevant odors, like the odor of breast milk (Porter et al., 1992). However, the fact that we begin learning odor preferences in the womb (Schaal et al., 1998; Winberg and Porter, 1998), e.g. infants of mothers who consume garlic are born with a preference for garlic odor (Mennella et al., 1995), makes it difficult to dissociate innate odor preferences from early learning. Regardless of whether there are innately preferred or rejected odors, learning and context certainly play important roles later in life. One line of evidence comes from intercultural differences in odor preferences (Schleidt et al., 1981; Wysocki and Gilbert, 1989; AyabeKanamuura et al., 1998). An odor that appeals to one culture does not necessarily appeal to another (see Section 9.3). Also consider a study sponsored by the US military that sought a universally offensive odor for non-lethal crowd control (Dilks et al., 1999). No single odor in the study was considered equally unpleasant across all ethnic groups. Another line of evidence comes from studies of odors paired with either positive or negative emotional experiences (Baeyens and Wrzesniewski, 1996; Herz et al., 2004b). The results suggest that, through associative learning, the experience of an odor takes on the emotional tone of the context in which the odor was originally experienced. As suggested elsewhere in this chapter, beliefs and expectations play a critical role. In one study, panelists rated a given odor as less pleasant if they believed it consisted of synthetic rather than natural materials (Herz, 2003), while in another, people expressed more liking for a mixture of fatty acids when the stimulus was labeled ‘parmesan cheese’ rather than ‘vomit’ (Herz and von Clef, 2001). Perfumers and flavorists know the benefits of selecting odors that have positive associations in the culture in which the product will be marketed. The above research suggests that the consumer product industry might actively influence the acceptability of new scents through proper advertising, packaging, and labeling that creates associations with positive images and concepts. Retailers can further help by ensuring that consumers experience new odors in an emotionally positive context. Such techniques might prove particularly effective in influencing the pleasantness of newly developed odors and scents.

6. CHEMESTHESIS 6.1. The experience of chemesthesis We feel chemicals when they stimulate nerve endings in our skin (see Doty and ComettoMuñiz, 2003; Wysocki and Wise, 2003). Chemical feel is called ‘chemesthesis,’ a

Taste, smell and chemesthesis

107

contraction of ‘chemical somesthesis’. When chemicals reach nerve endings in sufficient concentration, we experience sensations such as the ‘cooling’ of menthol, the ‘bite’ of carbonation, the ‘burn’ of distilled spirits, the ‘heat’ of chili peppers, and the ‘sting’ of smelling salts. Chemesthesis warns us against high chemical concentrations, and helps protect tissue through reactions such as tearing, increased mucus production, sneezing, and coughing. However, mild to moderate chemesthesis plays an important role in our everyday experience of foods, beverages, and many personal products.

6.2. Peripheral anatomy and receptor mechanisms for experiencing chemesthesis Mucous membranes, like those that cover the eyes, nose, and mouth, are more sensitive to chemicals because they offer easier access to nerve endings than the keratinized skin on the rest of the body. As with taste and smell, some irritants stimulate specific receptor proteins in nerve endings (Caterina and Julius, 2001; Peier et al., 2002; McKemy, 2005). For example, one protein responds to capsaicin, the active principle in hot peppers, whereas another responds to cooling compounds like menthol. Interestingly, these and other chemically sensitive receptors also respond to temperature changes, so the terms ‘hot pepper’ and ‘cool mint’ make perfect sense. In addition, some compounds may act through non-specific effects on cell membranes rather than on specific receptor proteins. The action of carbon dioxide (CO2), the gas that gives bite to carbonated beverages, depends on enzyme activity (see Tarun et al., 2003). CO2 diffuses into tissue, where enzymes facilitate formation of carbonic acid (the effective stimulus). Recent work has found individual differences in the enzyme genes expressed in the nose (Tarun et al., 2003) and sensory receptors for nicotine may also vary among individuals (Keiger et al. 2003). Furthermore, PROP tasters and PROP non-tasters (p. 94) differ in sensitivity to oral irritation, probably due to differences in nerve density in the tongue (Prescott and SwainCampbell, 2000; Duffy et al., 2004). As with smell and taste, we may eventually predict some chemesthetic product experiences and preferences based on gene make-up.

6.3. Processing peripheral nerve signals into chemesthetic experiences Nerve fibers that carry somatosensory information project first to the brain stem (for signals from the eyes, nose, and mouth) (Hendry et al., 1999). From there, signals pass to the thalamus, then to the primary somatosensory cortex, then to higher cortical areas. Data on central processing of chemesthesis are sparse. An imaging study using an irritating concentration of acetone found activation in the brainstem and somatosensory cortex (Savic et al., 2002). In addition, research measuring electrical potentials has found differences in patterns of activity across the scalp between pure odors and irritants (reviewed in Shusterman, 2002). However, little research bears on quality coding past the periphery.

6.4. Chemesthesis: Basic phenomena At both supra-threshold (Cometto-Muñiz and Hernádez, 1990) and threshold levels (Cometto-Muñiz et al., 2004), there is greater cooperation (better additivity) among mixture-components for nasal chemesthesis than for smell. Better cooperation might indicate that detection-mechanisms for chemesthesis are more broadly tuned (Cometto-Muñiz et al., 2004). Practically speaking, good cooperation means a mixture might irritate, even if none of the individual compounds irritate, having implications for the experience of multi-odorous indoor air and odor-containing personal products used in indoor spaces.

108

Product Experience

In terms of compounds that have both chemesthetic and olfactory components, Figure 4.4 shows data from an experiment in which ammonia was presented to subjects in two concentrations and for two durations. As can be seen, the perceived irritation (and thus, the total intensity) increased more dramatically as a function of both concentration and stimulus duration than did the perceived odor. Thus, when one inhales ammonia, its odor reaches a peak rapidly while irritation continues to grow over time, perhaps even suppressing its odor. Although strong chemesthesis is innately aversive, many individuals develop a preference for mild to moderate irritation with repeated exposure (see Rozin and Schiller, 1980; Stevenson and Yeomans, 1993). Also, people in warmer climates tend to consume more hot spices. Since compounds in many spice plants have anti-microbial activity, societies in which food spoilage poses greater problems might develop a taste for spices, in part, for practical reasons. Further, as with odor, emotional reactions will depend on beliefs about the stimulus. A pleasantly pungent fragrance will be received quite differently if subjects believe the aroma comes from a potentially dangerous chemical (see Dalton and Wysocki, 1996). Relative to taste and smell, the dynamics of chemesthesis are slow (Cain et al., 1986; Green, 1990; Prescott and Swain-Campbell, 2000). Depending on the chemical, concentration, method of presentation, and location of stimulation (mouth, nose, or eyes), irritation may build over many minutes and fade slowly once a peak is reached. Build-up, seen with steady presentation or with a sufficiently brief inter-stimulus interval (ISI), is often referred to as sensitization (Green, 1990; Prescott and Swain-Campbell, 2000). The waning of irritation, seen after some time with steady presentation or with a sufficiently long ISI, is often called desensitization (Green, 1990; Prescott and Swain-Campbell, 2000; Jacquot et al., 2005). However, the mechanisms underlying these phenomena are not completely understood.

FIGURE 4.4 Perceived odor, pungency (irritation) and total intensity for two concentrations of ammonia inhaled for two different durations (from Cometto-Muniz and Cain, 1984).

Taste, smell and chemesthesis

109

7. MEASURING CHEMOSENSORY PRODUCT EXPERIENCE 7.1. Chemosensory experiences: Consumers versus experts Quantifying the taste, smell and chemesthetic experiences produced by products is critical to many product development efforts. In many commodity areas, e.g. wine, tea, coffee, dairy products, and perfumes, expert ‘palates’ or ‘noses’ are used to characterize sensory properties, to create new formulations, and to delineate what constitutes a ‘good’ or ‘bad’ product. However, by virtue of their extensive experience and their acquired abilities to distinguish subtle product differences, the product experiences of experts are not the same as those of consumers. A reason commonly cited by product experts for not using consumers to judge product quality is that consumers differ from one another in their experiences and preferences. Yet, while this is true, it has been shown that the expert’s experience of wine and beer quality differs significantly from one expert to another (Langron et al., 1983; Guinard et al., 1999), and that even the vocabulary they use to describe these products differs dramatically among them (Sauvageot et al., 2006). In the perfume industry, ‘experts’ also disagree. Witness a leading perfumist’s review of Dark Chocolate perfume by Coty: ‘The name suggests a beguilingly sinful, high calorie confection. Instead, the fragrance has all the charm and mystery of a neon-lit shopping mall, and smells of cheap chocolates with a fruit heart. A case of pastel clad obesity’ (Turin, 2006). The notion that experts have special sensory abilities that enable them to detect product differences that consumers cannot is also often overstated. For example, it has been shown that while expert tea tasters can discriminate the experience produced from tasting tea in which the milk has been poured into the tea versus the tea having been poured into the milk, it has been shown that consumers have the same ability (Powers, 1988). Of the many purported abilities of experts, the one that most differentiates expert product experiences from those of consumers is their ability to identify and characterize differences in taste and olfactory product experiences using a well-defined vocabulary. This ability to provide a detailed description of product attributes, while beyond the capacity of most consumers, can often be taught using standard descriptive sensory methods, as discussed below.

7.2. Methods for quantifying chemosensory product experiences: Trained panels Although the use of experts is still common in some specialized product areas, most commercial products are now characterized by trained ‘descriptive’ panels. Descriptive panels are groups of individuals, screened to ensure lack of defects in taste or smell. These individuals, usually 10–20 in number, are trained in one of a variety of descriptive sensory methods that enable them to articulate and describe their product experiences using standard terminology and operational definitions. This process produces a product profile that identifies all of the taste, smell, and chemesthetic attributes of the product and the intensities of each. There are a wide variety of methods for conducting descriptive analysis. The oldest is the flavor profile method developed by Arthur D. Little Co. (Cairncross and Sjostrom, 1950). This method utilized a panel of six to eight individuals who received training in taste and smell physiology, sensory testing, and flavor vocabulary. Physical reference standards for specific taste and smell attributes, e.g. ‘salty’, ‘rancid’ or ‘piney’ were employed to be used during training and product evaluation. A panel leader was chosen

110

Product Experience

to lead panel discussions and to arrive at a consensus on the experienced aroma, taste, flavor, mouthfeel, and aftertaste attributes of the product, the order in which they were perceived, their intensity, and the ‘amplitude’ or overall quality of the product. Although the A. D. Little Flavor Profile method was the first popular descriptive method, it has largely been replaced by more contemporary methods that have improved upon various aspects of the methodology. Among these more contemporary approaches is Quantitative Descriptive Analysis (QDA) (Stone et al., 1974), a method that replaced consensus agreement with the use of quantitative, statistical analysis of panel data. The perceived intensity ratings were improved in QDA by using a linear graphic rating scale, resulting in a method with reportedly good quantitative reliability (Stone and Sidel, 2004). This technique also spurred the use of ‘spider web’ charts for displaying product attributes. Figure 4.5 shows a spider web chart of descriptive data for two personal care products (bar soap). The lengths of the spines reflect the maximum intensity for each attribute and the mean product ratings for that attribute are plotted along the spine. By overlaying the data for two or more products on a single spider web (as in Figure 4.5), the differences in the sensory profiles of the products are easily visualized. A third technique, known as Spectrum Descriptive Analysis, has also been popularized (Meilgaard et al., 1999). This latter technique adapts approaches from earlier methods, while utilizing ‘universal’ rating scales with physical reference anchors in order to quantify the product experiences of panelists. While each of these methods differ to some degree, comparisons of the methods for the same products typically produce similar results.

7.3. Methods for quantifying chemosensory product experiences: Consumers Quantifying sensory experience After training and extensive panel work, it can be argued that the product experiences of descriptive panelists are also unlike those of the average user. In an effort to avoid the use of either experts or trained panelists to describe taste and olfactory product

FIGURE 4.5 A typical spider web plot of descriptive data for two bar soaps (from Lawless and Heymann, 1998).

Taste, smell and chemesthesis

111

experience, the method known as ‘free-choice profiling’ was developed (Williams and Langron, 1984; Steenkamp and van Trijp, 1988). This approach relies on naive consumers who use their own vocabulary to describe their experience of the product or who select terminology from a pre-prepared list. Terminology may also be generated using the repertory grid method (e.g. Jaeger et al., 2005). The data are then analyzed using multivariate techniques in order to develop a user-based perceptual space of products, the dimensions of which can be defined in terms of the actual experiential vocabulary that consumers use. Other consumer-friendly techniques have also been developed. Free-sorting procedures, in which products are sorted into groups and a common vocabulary is used to describe the groups, have been used to analyze the perceptual product space for tastants (Yang and Lawless, 2005), odorants (Rawcliffe et al., 1992), and a variety of foods and beverages (Lawless et al., 1995; Popper and Heymann, 1996; Tang and Heymann, 2002). In a recent study comparing sorting by consumers and QDA trained panelists, the perceptual product space for the two groups was found to be quite similar. Moreover, the sorting procedure for trained panelists provided similar results to traditional QDA methods (Cartier et al., 2006). The authors concluded that ‘sorting combined with verbalization led to meaningful and consistent product sensory mapping, whatever the panelists’ level of training’ (Cartier et al., 2006, p. 562). Similar results also have been reported by Faye et al. (2006) for non-food products. Quantifying affective experience As noted in other sections, most chemosensory products produce a strong affective or ‘hedonic’ experience in the user. When an individual smells a new cologne or tastes a new chewing gum, there is an emotional experience of its pleasantness or unpleasantness. These hedonic experiences of products are the primary drivers of the consumer’s choice, purchase, consumption, and/or utilization of the product. The first studies of affective experience were undertaken by Beebe-Center in the 1930s, but the systematic study of food hedonics (from the Greek word for ‘pleasure’) and its measurement began after World War II when the US Army Quartermaster Food and Container Institute in Chicago began to study why soldiers liked and/or disliked certain foods. This research program ultimately produced the now well known, ‘ninepoint hedonic scale’ (Peryam and Pilgrim, 1957) (see left side of Figure 4.6). This simple method for quantifying the intensity of a person’s affective experience is now the most commonly used affective measure in the food, cosmetics, perfumery, and personal care products industries. In spite of its simplicity, the nine-point hedonic scale does have problems. Since it is a form of ‘category’ scale there is a tendency for people to avoid the extreme categories (Stevens and Galanter, 1957). In addition, the intervals between scale points are not perceptually equal (Jones and Thurstone, 1955; Moskowitz and Sidel, 1971), so the level of measurement it provides is, at best, ordered metric. Other methods that have been used over the years include magnitude estimation, linear graphic rating scales and, most recently, a Labeled Affective Magnitude (LAM) scale (Schutz and Cardello, 2001; Cardello and Schutz, 2004). The latter scale was developed to provide a simple-to-use, ratio-like scale of product liking experience that is modeled after other contemporary labeled magnitude scales (Green et al., 1993; Bartoshuk, 2000). As shown on the right side of Figure 4.6, this scale uses a vertical line with the end points labeled ‘greatest imaginable liking’ and ‘greatest imaginable disliking’. Between these end-point anchors the nine labels used in the nine-point hedonic scale are distributed along the liking/disliking dimension according to the perceived ratios in their semantic meaning, as determined

112

Product Experience

Scales for rating product liking experience Nine-point Hedonic Scalea

Labeled Affective Magnitude Scaleb,c Greatest imaginable liking

Like extremely Like very much Like moderately Like slightly Neither like nor dislike Dislike slightly Dislike moderately Dislike very much Dislike extremely

Like extremely Like very much Like moderately Like slightly Neither like nor dislike Dislike slightly Dislike moderately Dislike very much Dislike extremely Greatest imaginable disliking

a

Peryam and Pilgrim (1957)

bSchutz

and Cardello (2001) and Schutz (2004)

cCardello

FIGURE 4.6 Scales for rating product liking/disliking (affective) experiences.

using a ratio scaling procedure. Using this scale it is possible to estimate whether the liking experience for a particular product is twice, one-third, etc. as large as the liking experience for a different product (Schutz and Cardello, 2001). In addition, the scale has been shown to produce better sensitivity, equal reliability, and equal ease of use to the nine-point hedonic scale by consumers (Schutz and Cardello, 2001).

8. CONTEXT, INFORMATION AND EXPECTATIONS IN CHEMOSENSORY AND PRODUCT EXPERIENCE 8.1. Non-sensory influences on product experience The taste and olfactory experiences that arise from the intrinsic properties of a product do not constitute the sole determinants of product experience. There are a host of other ‘extrinsic’ factors, e.g. product names, packaging, labeling, pricing, context and promotional information that influence product experience through cognitive and psychological processes. In this section we will explore the effect of some of these factors.

8.2. The effects of context Both the sequential and situational contexts in which a product is experienced can significantly alter our experiences of it. Sequential effects are commonplace. When you are shopping for fragrances in a department store, you might first smell a perfume on one wrist and then its eau de toilette on your other wrist. You would likely conclude that the eau de toilette was quite weak. However, if you tried them in the reverse order, you would

Taste, smell and chemesthesis

113

likely feel differently about the strength of the eau de toilette, even if you waited a relatively long time between trials to ensure that no adaptation to the scent was occurring. In fact, in a recent study of sequential context effects, the perceived intensity of the odor of perfumery grade California Orange Oil was affected by presentation of either a weaker or stronger sample as much as 25 minutes before the target odorant was presented (Pol et al., 1998). A number of other studies using tastants and odorants have shown that the intensity of a target product presented within a context of products of either higher or lower taste/smell intensity or liking will be ‘contrasted’ with the intensity of the contextual products (Lawless, 1983; McBride, 1985; Schifferstein, 1995). These results are commonly explained in terms of a ‘perceptual’ adaptation level (Helson, 1964) or in terms of range-frequency theory (Parducci, 1974). Recently, Schifferstein has addressed this phenomenon in terms of the nature and characteristics of the contextual stimuli and their impact on the magnitude of these contrast effects (Schifferstein, 1995; Schifferstein and Frijters, 1992; Schifferstein and Oudejans, 1996). His results support the notion that the more similar the contextual and target stimuli, the larger will be the contrast effect. In addition to sequential context effects, the situational and environmental context in which products are encountered can alter our experiences of them. For example, the product experience that results from eating ice cream on a hot summer day is quite different from eating that same ice cream on a cold, winter day. Likewise, the experience of encountering a fishy odor in the hallway of your home is quite different from the experience of encountering that same odor in a seafood market. In the world of taste and smell, context can make all the difference. The number of different environmental, situational, and other contextual factors that can alter product experiences is quite large. The reader is referred to articles and book chapters by Rozin and Tuorila, 1993; Cardello, 1994, 2007; Deliza and MacFie, 1996; Schifferstein, 2001; Wansink et al., 2005; Jaeger, 2006 and Chapter 24 by Meiselman in this volume for discussions of the contextual influences on product experience. Perhaps the most common contextual or ‘framing’ influences in the commercial products industry are those involving product information. Whether written, verbal, or non-verbal, information influences consumer experience and behavior by altering the attitudes and beliefs that the consumer holds toward the product. In turn, these attitudes and beliefs can give rise to product expectations, i.e. psychological anticipations that the product will possess certain attributes or have certain anticipated effects. Expectations can arise from previous experiences with the product, with other products in the category, or from any one of a variety of information sources about the product. Two major types of expectations are those that relate to expected sensory experiences, e.g. a belief that a skin cream will have a floral scent, and those that relate to expected affective experiences, e.g. a belief that you will like the smell of a new automobile’s leather interior.

8.3. The effects of information and expectations Effects on olfactory product experiences Olfactory experiences seem to be especially prone to the influence of expectations. As early as 1899, Slosson performed a classroom demonstration in which he told students that an odorant was to be dispersed into the back of the classroom and that he wanted to see how long it took for the odor to reach the front of the room. He asked his students to raise their hand when they smelled the odor. Although no odor was actually dispersed, a majority of students reported smelling the odor, some even becoming upset by its unpleasant quality. Much more recently, O’Mahony (1978) performed a similar but more dramatic experiment. Using a British television broadcast of a documentary on

114

Product Experience

the chemical senses, viewers were informed that smells could be produced by transmitting sound waves of the same vibrational frequency as the frequency of vibration of the molecules of the odorant. The documentary then went on to ‘demonstrate’ the phenomenon by ‘broadcasting’ a specific frequency of sound alleged to correspond to a pleasant country odor. Viewers were asked to phone or write the station if and when they smelled the odor. Of 130 responses received, 114 viewers said that they experienced a smell when the sound was emitted. The most common smell experiences reported were ‘hay’ or ‘grass’. Using more traditional paradigms, Engen (1972) performed experiments in which an odorant was presented to subjects at regular intervals prior to the presentation of a ‘blank odorant’ in order to establish an expectation for the odor. The mere expectation that the odorant was to be presented resulted in an odor experience. In another experiment, he flowed odorants through a tube that was visible to the subjects. The odorants were either colored yellow or left transparent. A much higher rate of odor experience resulted on trials in which the odorant was colored yellow than when it was transparent. The color appeared to have induced an expectation that an odor was being presented and this expectation led to a greater likelihood of the olfactory experience. Dalton (1996) exposed subjects to the odorant ‘isobornyl acetate’ under three different information conditions. One information group was told that they would be exposed to a natural extract of balsam trees. A second group was told they would be exposed to an industrial solvent that could produce health and cognitive problems after longterm use. A control group was simply told that they would be exposed to a standard odorant approved for smell research. The subjects were exposed to the odorant over the course of 20 minutes and were asked to judge its intensity every minute. The results of this study are depicted in Figure 4.7. Although the positive information group (balsam trees) showed a typical adaptation curve to the odorant over time, the negative bias group (industrial solvent) first began to adapt, but then their experience of the odor began to increase in intensity, reaching strong levels by the end of the test. The neutral

50

Very strong Positive bias Negative bias Neutral bias

Perceived intensity

40

Strong 30

20 Moderate 10 Weak Barely detectable 0

5

10

15

20

Exposure duration (minutes)

FIGURE 4.7 Perceived odor intensity of isobornyl acetate for subjects in the positive, negative, and neutral bias conditions during a 20-minute exposure (from Dalton, 1996).

Taste, smell and chemesthesis

115

(control) group produced an intermediate level of adaptation. This study underscores the relevance of survival mechanisms and how our olfactory experiences can be shaped by perceived dangers. Another study showing the importance of information about the source of experienced odors was conducted by Distel and Hudson (2001). In this study odors of everyday products were experienced as more pleasant and more intense when subjects were told the name of the product than when the source of the odor was left unstated. Cognitive elements, perhaps related to the reduction in uncertainty about the odors, appear to have induced changes in the olfactory experiences of them. In a study on perfumes, Scharf and Volkmer (2000) examined the olfactory expectations produced by brand information. They tested two different perfumes at two different concentrations. Consumers first evaluated the fragrances in a blind test for their olfactory experiences of ‘intensity’, ‘sweetness’, ‘freshness’ and ‘floweriness’. In a second session, consumers evaluated images of four different package designs for the perfumes that differed in color (dark red vs. pastel green) and brand (‘Christian Dior elegance’ vs. ‘TIP woman’). Each design was evaluated for the olfactory experiences that would be expected of the product using the same attributes used in the blind test. Lastly, consumers rated combinations of the four fragrances and the four package images on the attributes. Results showed that the olfactory expectations created by the packages influenced the actual olfactory experiences of the perfumes. In some cases, the olfactory experience was altered to be more consistent with the expectation that was created by the packaging. In other cases, the experience was altered to become less like that which was expected. The former finding is commonly referred to as ‘assimilation’, i.e. the experience assimilates (incorporates) the expectation of it. The latter finding is referred to as ‘contrast’. As we shall see in our discussion of taste and liking expectations, ‘assimilation’ and ‘contrast’ are but two of several possible outcomes, yet they seem to form the basis of what appears to be an evolving cognitive model to explain how expectations can influence our product experiences. Effects on taste product experiences One early study of the effects of information on taste experience was conducted by Carlsmith and Aronson (1963). These researchers presented a series of sweet or bitter solutions to consumers who evaluated them for their intensity. Before presenting each, they gave a signal to indicate that either a bitter or sweet stimulus would be presented. In some cases the signal accurately cued the taste solution being presented, but in other cases the cue was misleading, e.g. a signal for a bitter solution was given, but a sweet solution was presented. Their results showed that sweet solutions that disconfirmed a taste expectation were rated less intense than sweet solutions that confirmed a taste expectancy. However, bitter solutions that disconfirmed an expectation were rated more intense. In both cases, disconfirmed expectations resulted in a more negative experience (less sweet or more bitter). They concluded that the data supported Cognitive Dissonance Theory (Festinger, 1957), a psychological theory that proposes that when cognitions (beliefs, values, attitudes, perceptions) or behaviors are inconsistent with one another, a negative psychological state of ‘dissonance’ occurs that makes the individual evaluate the stimulus more negatively. Olson and Dover (1976) performed a similar study on the taste of coffee. These researchers assigned consumers to an information or control condition. In the information condition, consumers were given written information that the coffee had ‘no bitterness’. After reading the information, consumers rated the strength of their belief that the coffee would be either ‘very bitter’, ‘fairly bitter’, ‘somewhat bitter’, or ‘not at all bitter’. Several days later, they returned to taste and evaluate the actual coffee using the same procedure to assess their perceptions of its bitterness. The consumers in the

116

Product Experience

Effect of expectations on product experience (coffee bitterness) 8 7

Control group E

Expectation group

6

Expectation level (E)

5 4 3 2

E

1 E

E

0 Not at all bitter Somewhat bitter Fairly bitter

Very bitter

Bitterness cateogries

FIGURE 4.8 The effects of product expectations on product experience (plot of the data from Olson and Dover, 1976). The bars indicate the mean ratings of the reported ‘belief strength’ of the bitterness of coffee. The arrows indicate the level of pre-trial expectation in the information condition.

control condition merely came to the taste test session and evaluated the coffee. Figure 4.8 shows the results of the study. The arrows indicate the expected belief strength of the information group before tasting the coffee. As can be seen, when compared to the control group, the information group’s experience of the bitterness of the tasted coffee moved in the direction of their expectations of it, supporting an ‘assimilation’ of the expectation into the product experience. Cardello and Sawyer (1992) conducted a study in which consumers were given one of four different types of information about pomegranate juice, i.e. that the juice was ‘very bitter’, had ‘average bitterness’, was ‘not bitter at all’, or no information. The consumers rated their expected experience of the bitterness and other sensory and hedonic attributes of the juice prior to tasting it. The results showed that the information established different expectations for the bitterness of the juice that were consistent with the information. However, the ratings of the bitterness experienced in the tasted product showed both assimilation and contrast effects, similar to the findings of Scharf and Volkmer (2000). Lastly, in a recent study examining the role of information about the area of origin of olive oils (Carporale et al., 2006), users of Italian olive oils were led to believe that a particular oil had been produced in one of three different olive oil producing regions of Italy. Expectations of the bitterness and pungency of the oil were found to differ depending upon the regional information provided about the oil. Moreover, these expectations were assimilated into the ratings of the actual experience of the bitterness and pungency of the oil. Effects on affective product experiences Over the years, a number of studies have also examined the effects of product information on expectations of the liking for foods. Among these have been studies that have examined the effects of nutritional or health content labeling of foods (Tuorila et al., 1994, 1998; Kahkonen and Tuorila, 1998; Kahkonen, et al., 1999; Wansink and Park, 2002). In these studies, product labels such as ‘low-fat’ or ‘contains soy’ lowered user expectations for both the taste and other desirable sensory attributes of the product and, in many cases, resulted in an assimilation of the lowered expectation into the actual liking

Taste, smell and chemesthesis

117

experience (Tuorila et al., 1994; Kahkonen et al., 1996, 1999). Other studies have examined the effects of information about the geographic origins of the product (Stefani et al., 2006; Caporale et al., 2006) or the methods of processing and preserving them (Siret and Issanchou, 2000; Cardello, 2003; Caporale and Monteleone, 2004; Iaccarino et al., 2006). In the majority of these studies product liking has been shown to assimilate the expectation. However, as with sensory expectations, contrast effects have also been observed. Researchers in the area of consumer expectations are now shifting to a hybrid ‘assimilation-contrast model’ to account for the disparate data, with factors related to the magnitude of the discrepancy between expectations and actual product features and the individual’s confidence in his/her sensory abilities determining whether assimilation or contrast effects will occur. The reader is referred to Cardello (2007) for a recent review of this general topic area and for a more detailed discussion of predictive models of the effects of expectations on product liking experiences. Brain mechanisms, expectations and product experience In recent years progress has been made in understanding the neural bases of cognitive effects on taste, smell and other sensory experiences. Much of this progress has been made through the use of fMRI technology that measures blood flow in various regions of the brain. McClure et al. (2004) measured fMRI brain activity in response to the taste of cola beverages (Coke and Pepsi). When a simple visual stimulus of a Coke can (versus a circle of light) was presented before the test stimulus, increased brain activity was observed in several brain areas. The authors concluded that these latter brain areas are involved in the process of ‘biasing perception based on prior affective behavior’. In a more recent study, remarkably similar to the early expectation studies on bitter taste, fMRI was used to investigate the neural mechanisms by which taste expectations can attenuate actual taste perception (Sarinopolous et al., 2006). These investigators provided misleading information to subjects that a taste solution would be mildly bitter. This false information produced neural activity in specific areas of the cortex, including the orbitofrontal cortex (OFC) that predicted decreases in other brain activity in response to the presentation of a very bitter taste solution. The authors concluded that the (neural) network processing aversive taste was suppressed by ‘expectancy-related brain activity’ in specific regions of the cortex, including the OFC. Since the OFC encodes the ‘reward value’ of olfactory and taste stimuli (Rolls, 2000; Hornack et al., 2004), it appears that mere expectations about products activate reward areas in the brain, enabling cognitive-based stimuli to alter, at a neural level, the hedonic experience of the products we taste, eat, smell and drink.

9. AGE, GENDER, CULTURAL AND SOCIAL FACTORS IN CHEMOSENSORY AND PRODUCT EXPERIENCE 9.1. The effects of age As life expectancy rises, the elderly comprise an increasingly larger group of consumers. With advanced age comes declines in sensitivity to taste, smell and chemesthesis (Murphy 1993; Hummel et al., 2003; Boyce and Shone, 2006) and concomitant changes in our experiences of products. Aging takes the largest toll on olfactory experience, with substantial losses in sensitivity to a wide range of odorants. Figure 4.9 shows data from a study by Murphy and Cain (in Murphy, 1986) showing the percent correct identification for a battery of 80 odorants as a function of age. As age increases, there is a steady and progressive decline in olfactory identification. In terms of sensory thresholds, some

118

Product Experience

90

80

Percent correct

70

60

50

40

30

20

10

r ⫽ ⫺0.75

20

30

40

50

60

70

Age (yrs)

FIGURE 4.9 Percent correct identification for a battery of 80 odorants as a function of age (from Murphy and Cain [unpublished] in Murphy, 1986).

evidence suggests that the minimum concentration that subjects can smell increases by two orders of magnitude between the 2nd and 8th decades of life (Wysocki et al., 2003). Declines in taste sensitivity are more modest than those for smell, with age-associated increases in thresholds ranging from 1.3- to 9-fold, depending upon the taste compound studied (Stevens and Cain, 1993; Mojet et al., 2001). Results vary, but in several studies sensitivity to bitter and salty stimuli tends to decrease more than sensitivity to other tastes (reviewed in Mojet et al., 2001). Figure 4.10 from Mojet et al. (2003) shows data on age and gender differences for the five basic tastes. Each taste quality is represented by two taste compounds. Age effects are largest for umami taste, with other differences varying by both taste compound and gender. What is clear, though, is that in all cases, taste responsiveness declines with age. For chemesthesis, declines in sensitivity are also modest, with an age-associated increase in threshold of less than 10-fold for one study (Wysocki et al., 2003), and no significant effect of age in another study (Stuck et al., 2006). Declining chemosensory function poses risks. Declining taste sensitivity could increase the risk of consuming bitter food contaminants and of chronically over-consuming sodium (Stevens and Cain, 1993). Declining olfactory function also might render product warning odors less effective. In one study, 45% of elderly subjects failed to detect the warning odor in natural gas at standard levels (Murphy et al., 2002). Another study found that, relative to normal controls, those with olfactory dysfunction were more likely to miss fires and gas leaks, and more likely to ingest spoiled foods or toxic substances (Santos et al., 2004). These results suggest that concentrations of warning odors in products could require adjustment for elderly consumers. In addition, reduced flavor perception may lead to reduced food intake, reduced dietary variety, and less healthful food choices (Murphy, 1993; Duffy et al., 1995; Rolls 1999). Research suggests that appropriate flavor enhancement can improve consumption and nutritional status among the elderly (Schiffman, 2000; Mathey et al., 2001), but not all manipulations of this kind are effective

Taste, smell and chemesthesis

119

FIGURE 4.10 Perceived intensity of ten taste compounds in water for young (solid lines) and elderly (dotted lines) males (triangles) and females (circles) (from Mojet et al., 2003).

(Koskinen et al., 2003; Essed et al., 2006). Thus, formulation of optimal taste and smell product experiences to improve consumption, nutrition, and safety for the elderly represents both an opportunity and a challenge for the consumer products industry.

9.2. The effects of gender When gender differences occur in simple laboratory tests of sensitivity to taste, smell, and chemesthesis, the differences tend to favor women (Dalton, 2000; Shusterman, 2002; Mojet et al., 2001, 2003; Stuck et al., 2006). However, differences in taste and smell experiences are usually small and often fail to reach statistical significance (Dalton, 2000; Mojet et al., 2001, 2003) (see Figure 4.10 as an example). Differences are somewhat more consistent for chemesthetic experience, but are still moderate and still often fail to

120

Product Experience

reach significance (Wise et al., 2003; Wysocki et al., 2003; Hummel et al., 2003; Stuck et al., 2006). Some evidence suggests that gender differences may increase with age (e.g. Wysocki and Gilbert, 1989; Mojet et al., 2001), but on the whole, individual differences within the sexes are much larger than average differences between them. Although simple tests of sensitivity reveal relatively little difference between men and women, some interesting higher-order effects occur, particularly in olfaction. For example, with repeated testing, women can become progressively more sensitive to some odors (Dalton et al., 2002). Over the course of 30 sessions, sensitivity can increase by several orders of magnitude. However, beyond an initial practice period, men fail to show this sensitization. Interestingly, neither pre-pubescent girls nor post-menopausal women show this sensitization, which suggests that the effect may depend on hormonal function. Fluctuations across the menstrual cycle also suggest that hormonal function might influence olfactory sensitivity in women, though the exact role that hormones play remains unclear (Pause et al., 1996; Navarrete-Palacios et al., 2003). Other studies suggest that women are more likely than men to develop intolerance to certain environmental odors (Kreutzer, 1999; Caress and Steinman, 2004; Johansson et al., 2005). Yet another study found that a pleasant odor could reduce rated pain intensity in women, but not in men (Marchand and Arsenault, 2002). Sex differences in emotional responses to odors appear to play a role here. Regardless, it is clear that chemosensory experiences do differ between men and women. Although it may not always make sense to create gender-specific product formulations, gender differences are worth considering in product development and evaluation.

9.3. The effects of cultural and social factors Cultural factors are, perhaps, the most powerful of factors that can control product experience. In fact, one set of authors has concluded ‘If one were interested in determining as much as possible about an adult’s food preferences and could only ask one question, the question should undoubtedly be: “What is your culture or ethnic group?” ’ (Rozin and Vollmecke, 1986). Insofar as odors are concerned, there are large intercultural differences in odor preferences (Schleidt et al., 1981; Wysocki and Gilbert, 1989; Ayabe-Kanamuura et al., 1998). For example, wintergreen is experienced as a ‘candy’ odor in the US (Cain and Johnson, 1978), but as a ‘medicine’ odor in the UK (Moncrieff, 1966). Also the study cited in Section 5.4 that sought a universally offensive odor for non-lethal crowd control (Dilks et al., 1999) found no single odor to be unpleasant across all ethnic groups. In a recent study on the pleasantness of ‘Japanese’, ‘international’, and ‘European’ odors as experienced by Japanese and German consumers, ‘Japanese’ odors were perceived as much less pleasant by the German consumers than by the Japanese consumers. The ‘international’ odors were generally perceived as pleasant by both groups, and the ‘European’ odors elicited mixed responses (Ayabe-Kanamura et al., 1998). For taste, ethnic, and cultural differences abound. Kimchi is the national food of Korea and is eaten ubiquitously there. However, its strong salty/sour taste is not well accepted in most European and North American countries. Nor are the super-tart umeboshi pickled plums that are eaten in Japan. These and other exotic taste and olfactory experiences that one encounters in Asian, Indian and other ethnic markets are product experiences that few Americans or Europeans fully appreciate. Thus, it appears that the food habits and olfactory experiences of different ethnic groups can influence future sensory and affective preferences for chemosensory stimuli and products. Even within the same country, differences in product experiences are commonplace. Take, for example, the preference for brown eggs over white eggs in the Northeastern

Taste, smell and chemesthesis

121

United States and the preference for ‘grits’ (coarsely ground, hulled grain) in the Southern US, preferences not seen elsewhere in the U.S. Rozin (1996) has argued that some product avoidances, e.g. those for bean curd or pork kidneys among Midwesterners in the US, are due to people not viewing them as food. Thus, cultural factors can even define what items are considered to be foods and which are not. Religious and moral taboos provide the most dramatic examples of cultural effects on product experience. The dietary avoidance of pork by Jews and Muslims, beef in India, and eggs among certain African tribes (Simoons, 1961) are powerful examples of religious and cultural beliefs influencing taste, smell, and product preferences. The moral and cultural differences that make horse, cat, and dog consumption unheard of in North America, but a common practice in certain other countries is another example. Interestingly, the peoples who disdain these and other food products for cultural or religious reasons often consider them ‘unclean’ or ‘disgusting’; yet those who are not part of that culture view them to be quite normal for consumption. Disgust has also been found to be a major reason for morally motivated vegetarians to avoid all meat products (Rozin et al., 1996). Socioeconomic factors, such as status in society and income level can also affect our product experiences. In an early study of smell preferences, Brill (1932) reported that among 43 common odors the most disliked was that of perspiration, which he said was due to its ‘association with people of the lower class’. Perfumes, on the other hand, are often associated with royalty and wealth, while incense is associated with religious piety and sacredness (Largey and Watson, 2006). In the case of taste, witness the expensive, gourmet and health food tastes of today’s urban socialites that makes drinking a non-specialty shop coffee an experience to be avoided. And compare that to the oftenreported consumption of back alley restaurant waste by the homeless in our society. Clearly, the different socioeconomic situations in which these individuals find themselves drive their product experiences, defining for them what is a good or acceptable product experience and what is a bad or unacceptable product experience. Lastly, when talking about product experiences in today’s marketplace, one must, of course, mention the important social influences of role models. Whether it is a famous Hollywood actress pitching her own brand of perfume, a handsome actor drinking the latest introduction into the beer market, or the soccer superstar eating his favorite candy bar, role models are the essence of product advertising. A variety of role model effects on taste and smell have been reported in children, where elders, teachers, and peers have been used to increase liking for specific foods (see Rozin, 1988, for a review). It has also been shown that father–son and mother–daughter food preferences are more similar than father–daughter or mother–son preferences (Pliner, 1983). However while taste preferences begin early in life, systematic data on the role of parental influences in forming taste preferences in children is weak (Rozin, 1996). In addition to experimental studies, there is a large case study literature in sociology and anthropology describing the variety of social and cultural differences in taste and smell. The reader is referred to recent anthologies and books by Korsmeyer, (1999, 2005), Watson and Caldwell (2005) and Drobnick (2006) for fascinating discussions of the role of culture in chemosensory experience.

10. CONCLUSION In this chapter, we have discussed the physiological, psychological, cognitive and sociocultural factors that affect the taste, smell, chemesthetic, and liking experiences of products in today’s market. Although our treatment of topics has relied heavily on the

122

Product Experience

understanding of the fundamental mechanisms of human taste and smell experience, we feel that only through a thorough understanding of these underlying principles can we begin to understand the complex problem of product experience. It is our opinion that the future of product development in the food, perfume, and personal care industries lies in a combination of breakthroughs in the genetics of taste, smell, and chemesthesis and in a better understanding of the psychological, cultural, and societal factors that form the bases of our everyday product experiences.

REFERENCES Algom, D., Marks, L. E. and Cain, W. (1993). Memory psychophysics for chemosensation: perceptual and mental mixtures of odor and taste. Chemical Senses, 18, 151–160. Allison, A. and Chambers, D. H. (2005). Effects of residual toothpaste flavor on flavor profiles of common foods and beverages. Journal of Sensory Studies, 20, 167–186. Anderson, A. K., Christoff, K., Stappen, I., et al. (2003). Dissociated neural representations of intensity and valence in human olfaction. Nature Neuroscience, 6(2), 196–202. Atanasova, B., Thomas-Danguin T., Chabanet, C., Langlois, D., Nicklaus, S. and Etiévant, P. (2005). Perceptual interactions in odour mixtures: Odour quality in binary mixtures of woody and fruity wine odorants. Chemical Senses, 30, 209–217. Ayabe-Kanamuura, S., Schicker, I., Laska, M., et al. (1998). Differences in perception of everyday odors: A Japanese–German cross-cultural study. Chemical Senses, 23, 31–38. Ayya, N. and Lawless, H. T. (1992). Quantitative and qualitative evaluation of high-intensity sweeteners and sweetener mixtures. Chemical Senses, 17, 245–259. Baeyens, F. and Wrzesniewksi, A. (1996). Toilet rooms, body massages, and smells: Two field studies on human evaluative odor conditioning. Current Psychology: Developmental, Learning, Personality, Social, 15, 77–96. Bartoshuk, L. M. (2000). Comparing sensory experiences across individuals: Recent psychophysical advances illuminate genetic variation in taste perception. Chemical Senses, 25, 447–460. Bartoshuk, L. M., McBurney, D. H. and Pfaffmann, C. (1964). Taste of sodium chloride solutions after adaptation to sodium chloride: Implications for the ‘water taste’. Science, 143(3609), 967–968. Bartoshuk, L. M. and Wolfe, J. M. (1990). Conditioned taste aversions in humans: Are they olfactory aversions? Chemical Senses, 15, 551. Baylis, L. L., Rolls, E. T. and Baylis, G. C. (1995). Afferent connections of the caudolateral orbitofrontal cortex taste area of the primate. Neuroscience, 64, 801–812. Beauchamp, G. K. and Cowart, B. J. (1993). Development of salt taste responses in human newborns, infants and children. In: NIH Publication No. 93-3483, 3. Development, Growth and Senescence in the Chemical Senses, pp. 61–68. Washington, DC: U.S. Department of Health and Human Services. Beauchamp, G. K., Cowart, B. J. and Moran, M. (1986). Developmental changes in salt acceptability in human infants. Developmental Psychology, 19, 75–83. Bell, K. I. and Tepper, B. J. (2006). Short-term vegetable intake by young children classified by 6-n-propylthiouracil bitter-taste phenotype. American Journal of Clinical Nutrition, 84(1), 245–251. Bernstein, I. L. and Webster, M. M. (1980). Learned taste aversions in humans. Physiology and Behavior, 25, 363–366. Birch, L. L. and Marlin, D. W. (1982). I don’t like it; I never tried it: Effects of exposure on two-year old children’s food preferences. Appetite, 3, 353–360. Bonnans, S. and Noble, A. C. (1993). Effect of sweetener type and of sweetener and acid levels on temporal perception of sweetness, sourness and fruitiness. Chemical Senses, 18(3), 273–283. Booth, D. A. (1981). The physiology of appetite. British Medical Bulletin, 37, 135–140. Boyce, J. M. and Shone, G. R. (2006). Effects of ageing on smell and taste. Postgraduate Medical Journal, 82(966), 239–241. Breslin, P. A. S. (1996). Interactions among salty, sour and bitter compounds. Trends in Food Science and Technology, 7, 390–399. Breslin, P. A. S., Davidson, T. L. and Grill, H. J. (1990). Conditioned reversal of reactions to normally avoided tastes. Physiology and Behavior, 47, 535–538. Breslin, P. A. S. and Beauchamp, G. K. (1995). Suppression of bitterness by sodium: variation among bitter taste stimuli. Chemical Senses, 20, 609–623.

Taste, smell and chemesthesis

123

Breslin, P. A. and Huang, L. Q. (2006). Human taste: Peripheral anatomy, taste transduction, and coding. In: T. Hummel and A. Welge-Lüssen (Eds.) Taste and smell: An update (Advances in Oto-rhino-laryngology) Vol. 63, pp. 152–190. Basel, Germany: Karger. Brill, A. A. (1932). The sense of smell in the neuroses and psychoses. Psychoanalytic Quarterly, 1 (Spring), 7–42. Buck, L. B. (1996). Information coding in the vertebrate olfactory system. Annual Review of Neuroscience, 19, 517–544. Buck, L. B. (2000). The molecular architecture of odor and pheromone sensing in mammals. Cell, 100, 611–618. Bufe, B., Breslin, P. A., Kuhn, C., et al. (2005). The molecular basis of individual differences in phenylthiocarbamide and propylthiouracil bitterness perception. Current Biology, 15, 322–327. Burgard, D. R. and Kuznicki, J. T. (1990). Chemometrics: chemical and sensory data. Boca Raton, FL: CRC Press. Cabanac, M. (1971). Physiological role of pleasure. Science, 173, 1103–1107. Cain, W. S. (1974). Perception of odor intensity and the time-course of olfactory adaptation. ASHRAE Transactions, 80, 53–75. Cain, W. S. and Johnson, F. (1978). Lability of odor pleasantness: influence of mere exposure. Perception, 7, 459–465. Cain, W. S. and Murphy, C. (1980). Interaction between chemoreceptive modalities of odour and irritation. Nature, 284, 255–257. Cain, W. S. and Polak, E. H. (1992). Olfactory adaptation as an aspect of odor similarity. Chemical Senses, 17, 481–491. Cain, W. S., Schiet, F. T., Olsson, M. J. and De Wijk, R. A. (1995). Comparison of models of odor interaction. Chemical Senses, 20, 625–637. Cain, W. S., See, L. C. and Tosun, T. (1986). Irritation and odor from formaldehyde: Chamber studies. In: IAQ ’86: Managing the Indoor Air for Health and Energy Conservation, pp. 126–137. The American Society of Heating and Refrigerating and Air-Conditioning Engineers, Atlanta. Cairncross, S. E. and Sjostrom, L. B. (1950). Flavor profiles – a new approach to flavor problems. Food Technology, 4, 308–311. Capaldi, E. D. (1996). Conditioned food preferences. In: E. D. Capaldi (Ed.) Why we eat what we eat: the psychology of eating, pp. 53–80. Washington, DC: American Psychological Association. Capaldi, E. D., Campbell, D. H., Sheffer, J. D. and Bradford, J. P. (1987). Conditioned flavor preferences based on delayed caloric consequences. Journal of Experimental Psychology: Animal Behavior Processes, 13, 150–155. Capaldi, E. D. and Hunter, M. J. (1994). Taste and odor in conditioned flavor preference learning. Animal Learning and Behavior, 22, 355–365. Caporale, G. and Monteleone, E. (2004). Influence of information about manufacturing process on beer acceptability. Food Quality and Preference, 15, 271–278. Caporale, G., Policastro, S., Carlucci, A. and Monteleone, E. (2006). Consumer expectations for sensory properties in virgin olive oils. Food Quality and Preference, 17, 116–12. Cardello, A. V. (1994). Consumer expectations and their role in food acceptance. In: H. J. MacFie and D. M. H. Thomson (Eds.) Measurement of food preferences, pp. 253–297. London: Blackie Academic. Cardello, A. V. (2003). Consumer concerns and expectations about novel food processing technologies: effects on product liking. Appetite, 40, 217–233. Cardello, A. V. (2007). Measuring consumer expectations to improve product development. In: H. MacFie (Ed.) Consumer-led Food Product Development. Cambridge: Woodhead, 223–261. Cardello, A. V. and Sawyer, F. M. (1992). Effects of disconfirmed consumer expectations on food acceptability. Journal of Sensory Studies, 7, 253–277. Cardello, A. V. and Schutz, H. G. (2004). Numerical scale-point locations for constructing the LAM (labeled affective magnitude) scale. Journal of Sensory Studies 19, 341–346. Caress, S. M. and Steinemann, A. C. (2004). Prevalence of multiple chemical sensitivities: A population-based study in the southeastern United States. American Journal of Public Health, 94(5), 746–747. Carlsmith, J. M. and Aronson E. (1963). Some hedonic consequences of the confirmation and disconfirmation of expectancies. Journal of Abnormal and Social Psychology, 66, 151–156. Carmichael, S. T. and Price, J. L. (1995). Limbic connections of the orbital and medial prefrontal cortex in macaque monkeys. Journal of Comparative Neurology, 363, 615–641. Caterina, M. J. and Julius, D. (2001). The vanilloid receptor: A molecular gateway to the pain pathway. Annual Reviews in Neuroscience, 24, 487–517. Cartier, R., Rytz, A., Lecomte, A., Poblete, F., Krystlik, J., Belin, E. and Martin, N. (2006). Sorting procedure as an alternative to quantitative descriptive analysis to obtain a product sensory map. Food Quality and Preference, 17, 562–571. Cerf-Ducastel, B., Van de Moortele, P. F., MacLeod, P., Le Bihan, D. and Faurion, A. (2001). Interaction of gustatory and lingual somatosensory perceptions at the cortical level in the human: a functional magnetic resonance imaging study. Chemical Senses, 26, 371–383.

124

Product Experience

Chambers, K. C. and Bernstein, I. L. (1995). Conditioned flavor aversions. In: R. L. Doty (Ed.) Handbook of olfaction and gestation, pp. 745–773. New York: Dekker. Christensen, C. M. (1980). Effects of taste quality and intensity on oral perception of viscosity. Perception and Psychophysics, 28(4), 315–320. Cometto-Muñiz, J. E. and Cain, W. S. (1984). Temporal integration of pungency. Chemical Senses, 8, 315–327. Cometto-Muñiz, J. E. and Cain, W. S. (1995). Olfactory adaptation. In: R. L. Doty (Ed.) Handbook of olfaction and gestation, pp. 257–282. New York: Marcel Dekker. Cometto-Muñiz, J. E., Cain, W. S. and Abraham, M. H. (2004). Chemosensory additivity in trigeminal chemoreception as reflected by detection of mixtures. Experimental Brain Research, 158, 196–206. Cometto-Muñiz, J. E., Cain, W. S. and Abraham, M. H. (2005). Odor detection of single chemicals and binary mixtures. Behavioural Brain Research, 156, 115–123. Cometto-Muñiz, J. E. and Hernandez, S. M. (1990). Odorous and pungent attributes of mixed and unmixed odorants. Perception and Psychophysics, 47, 391–399. Cowart, B. J. and Beauchamp, G. K. (1990). Early development of taste perception. In: R. L. McBride and H. J. H. MacFie (Eds.) Psychological Basis of Sensory Evaluation, pp. 1–17. London: Elsevier Applied Science. Cutler, W. B., Preti, G., Krieger, A. M., et al. (1986). Human axillary secretions influence women’s menstrual cycles: The role of donor extract from men. Hormones and Behavior, 20, 463–473. Dalton, P. (1996). Odor perception and beliefs about risk. Chemical Senses, 21, 447–458. Dalton, P. (2000). Psychophysical and behavioral characteristics of olfactory adaptation. Chemical Senses, 25, 487–492. Dalton, P., Cowart, B., Dilks, D., et al. (2003). Olfactory function in workers exposed to styrene in the reinforced-plastics industry. American Journal of Industrial Medicine, 44(1), 1–11. Dalton, P., Doolittle, N. and Breslin, P. A. (2002). Gender-specific induction of enhanced sensitivity to odors. Nature Neuroscience, 5(3), 199–200. Dalton, P., Doolittle, N., Nagata, H. and Breslin, P. A. (2000). The merging of the senses: Integration of subthreshold taste and smell. Nature Neuroscience, 3, 431–432. Dalton, P. and Wysocki, C. J. (1996). The nature and duration of adaptation following long-term odor exposure. Perception and Psychophysics, 58(5), 781–792. Davis, L. B. and Porter, R. H. (1991). Persistent effects of early odor exposure on human neonates. Chemical Senses, 16(2), 169–174. Deliza R. and MacFie, H. J. H. (1996). The generation of sensory expectations by external cues and its effect on sensory perception. Journal of Sensory Studies, 11, 103–128. Delwiche, J. F. and Heffelfinger, A.L. (2005). Cross-modal additivity of taste and smell. Journal of Sensory Studies, 20, 512–525. Desor, J. A., Maller, O. and Andrews, K. (1975). Ingestive responses of human newborns to salts, sour and bitter stimuli. Journal of Comparative and Physiological Psychology, 89, 966–970. Desor, J. A., Maller, O. and Turner, R. E. (1973). Taste in acceptance of sugars by human infants. Journal of Comparative and Physiological Psychology, 84, 496–501. Diamond, J., Breslin, P. A. S., Doolittle, N., Nagata, H. and Dalton, P. (2005). Flavor processing: Perceptual and cognitive factors in multi-modal integration. Chemical Senses, 30 (Suppl. 1), i232–i233. Dilks, D. D., Dalton, P. and Beauchamp, G. K. (1999). Cross-cultural variation in responses to malodors. Chemical Senses, 24, 599. Dinehart, M. E., Hayes, J. E., Bartoshuk, L. M., Lanier, S. L. and Duffy, V. B. (2006). Bitter taste markers explain variability in vegetable sweetness, bitterness, and intake. Physiology and Behavior, 87(2), 304–313. Distel, H. and Hudson, R. (2001). Judgement of odor intensity is influenced by subjects’ knowledge of the odor source. Chemical Senses, 26, 247–251. Djordjevic, J., Zatorre, R. J. and Jones-Gotman, M. (2004). Effects of perceived and imagined odors on taste detection. Chemical Senses, 29, 199–208. Doty, R. L. and Cometto-Muñiz, J. E. (2003). Trigeminal chemosensation. In: R. L. Doty (Ed.) Handbook of olfaction and gustation (2nd Ed.), pp. 981–999. New York: Marcel Decker. Drewnoski, A., Grinker, J. A. and Hirsh, J. (1982). Obesity and flavor perception: Multidimensional scaling of soft drinks. Appetite, 3, 361–368. Drobnick, J. (2006). The smell culture reader. New York: Oxford. Duffy, V. B., Backstrand, J. R. and Ferris, A. M. (1995). Olfactory dysfunction and related nutritional risk in free-living, elderly women. Journal of the American Dietetic Association, 95(8), 879–884. Duffy, V. B., Peterson, J. M. and Bartoshuk, L. M. (2004). Associations between taste genetics, oral sensation and alcohol intake. Physiology and Behavior, 82(2–3), 435–445. Engen, T. (1972). The effect of expectation on judgments of odor. Acta Psychologica, 35, 450–458.

Taste, smell and chemesthesis

125

Engen, T. (1982). The perception of odors. New York: Academic. Engen, T. (1988) The acquisition of odour hedonics. In: S. Van Toller and G. H. Dodd (Eds.) Perfumery: The psychology and biology of fragrance, pp. 79–90. London: Chapman and Hall. Essed, N. H., Van Staveren, W. A., Kok, F. J. and De Graaf, C. (2007). No effect of 16 weeks flavor enhancement on dietary intake and nutritional status of nursing home elderly. Appetite, 48, 29–36. Faye, P., Bremaud, D., Teillet, E., Courcoux, P., Giboreau, A. and Nicod, H. (2006). An alternative to external preference mapping based on consumer perceptive mapping. Food Quality and Preference, 17, 604–614. Festinger, L. (1957). A theory of cognitive dissonance. Evanston, IL: Row and Peterson. Frank, R. A. (2002). Response context affects judgments of flavor components in foods and beverages. Food Quality and Preference, 14, 139–145. Frank, R. A. and Byram, J. (1988). Taste–smell interactions are tastant and odorant dependent. Chemical Senses, 13, 445–455. Frank, R. A., Van der Klaauw, N. J. and Schifferstein, H. N. J. (1993). Both perceptual and conceptual factors influence taste–odor and taste–taste interactions. Perception and Psychophysics, 54, 343–350. Ganchrow, J. R. (1995). Ontogeny of human taste perception. In: R. L. Doty (Ed.) Handbook of olfaction and gustation, pp. 715–772. New York: Dekker. Garb, J. L. and Stunkard, A. (1974). Taste aversions in man. American Journal of Psychiatry, 131, 1204–1207. Gibson, J. J. (1966). The senses considered as perceptual systems. Boston: Houghton Mifflin. Gilmore, M. M. and Green, B. G. (1993). Sensory irritation and taste produced by NaCl and citric acid: effects of capsaicin desensitization. Chemical Senses, 18(3), 257–272. Glusman, G., Yanai, I., Rubin, I. and Lancet, D. (2001). The complete human olfactory subgenome. Genome Research, 11, 685–702. Gottfried, J. A. (2006). Smell: Central nervous processing. In: T. Hummel and A. Welge-Lüssen (Eds.) Taste and smell: An update (Advances in Oto-rhino-laryngology), pp. 44–69. Basel, Germany: Karger. Gottfried, J. A., O’Doherty, J. and Dolan, R. J. (2003). Encoding predictive reward value in human amygdala and orbitofrontal cortex. Science, 301, 1104–1107. Green, B. G. (1990). Effects of thermal, mechanical, and chemical stimulation on the perception of oral irritation. In: Green, B. G., Mason, J. R. and Kare, M. R. (Eds.) Chemical senses, Vol. 2, Irritation, pp. 171–175. New York: Marcel Dekker. Green, B. G., Shaffer, G. S. and Gilmore, M. M. (1993). Derivation and evaluation of a semantic scale of oral sensation magnitude with apparent ratio properties. Chemical Senses, 18, 683–702. Guinard, J., Yip, D., Cubero, E. and Mazzucchelli, R. (1999). Quality ratings by experts, and relation with descriptive analysis ratings: a case study with beer. Food Quality and Preference, 10, 59–67. Hallock, R. M. and Di Lorenzo, P. M. (2006). Temporal coding in the gustatory system. Neuroscience and Biobehavioral Reviews, 30(8), 1145–1160. Harris, G. and Booth, D. A. (1987). Infants’ preference for salt in food: Its dependence upon recent dietary experience. Journal of Reproductive and Infant Psychology, 5, 97–104. Helson, H. (1964). Adaptation-level theory: An experimental and systematic approach to behavior. New York: Harper. Hendry, S. H. C., Hsiao, S. S. and Bushnell, M. C. (1999). Somatic sensation. In: M. J. Zigmond, F. E. Bloom, S.C., et al. (Eds.) Fundamental neuroscience, pp. 761–789. San Diego: Academic Press. Herz, R. S. (1997). Emotion experienced during encoding enhances odor retrieval cue effectiveness. American Journal of Psychology, 110, 489–505. Herz, R. S. (2003). The effect of verbal context in olfactory perception. Journal of Experimental Psychology: General, 132, 595–606. Herz, R. S., Eliassen J., Beland S. and Souza T. (2004a). Neuroimaging evidence for the emotional potency of odor-evoked memory. Neuropsychologia, 42, 371–378. Herz, R. S., Beland, S. and Hellerstein, M. (2004b). Changing odor hedonic perception through emotional associations in humans. International Journal of Comparative Psychology, 17, 315–339. Herz, R. S. and Von Clef, J. (2001). The influence of verbal labeling on the perception of odors: Evidence for olfactory illusions? Perception, 30, 381–391. Hetherington, M. M. and Rolls, B. J. (1996). Sensory-specific satiety: theoretical frameworks and central characteristics. In: E. Capaldi (Ed.) Why we eat what we eat: The psychology of eating, pp. 267–290. Washington, DC: American Psychological Association. Hildebrand, J. G. and Shepherd, G. M. (1997). Mechanisms of olfactory discrimination: Converging evidence for common principles across phyla. Annual Review of Neuroscience, 20, 595–631. Hornak J., O’Doherty J., Bramham J., Rolls E.T., et al. (2004). Reward-related reversal learning after surgical excisions in orbito-frontal or dorsolateral prefrontal cortex in humans. Journal of Cognitive Neuroscience, 16(3), 463–478.

126

Product Experience

Huang, A. L., Chen, X., Hoon, M. A., et al. (2006). The cells and logic for mammalian sour taste detection. Nature, 442, 934–938. Hummel, T., Futschik, T., Frasnelli, J. and Huttenbrink, K. B. (2003). Effects of olfactory function, age, and gender on trigeminally mediated sensations: A study based on the lateralization of chemosensory stimuli. Toxicology Letters, 140–141, 273–280. Iaccarino T., DiMonaco R., Mincione A., Cavella S. and Masi P. (2006). Influence of information on origin and technology on the consumer response: The case of soppressata salami. Food Quality and Preference, 17, 76–84. Jacob, T. J., Fraser, C., Wang, L., Walker, V. and O’Connor, S. (2003). Psychophysical evaluation of responses to pleasant and mal-odour stimulation in human subjects; adaptation, dose response and gender differences. International Journal of Psychophysiology, 48, 67–80. Jacquot, L., Monnin, J., Lucarz A. and Brand, G. (2005). Trigeminal sensitization and desensitization in the nasal cavity: a study of cross interactions. Rhinology, 43, 93–98. Jaeger, S. R. (2006). Non-sensory factors in sensory science research. Food Quality and Preference, 17, 132–144. Jaeger, S. R., Rossiter, K. L. and Lau, K. (2005). Consumer perceptions of novel fruit and familiar fruit: A repertory of grid application. Journal of the Science of Food and Agriculture, 85(3), 480–488. Jinks, A. and Laing, D. G. (2001). The analysis of odor mixtures by humans: evidence for a configurational process. Physiology Behavior, 72, 51–63. Johansson, A., Bramerson, A., Millqvist, E., Nordin, S. and Bende, M. (2005). Prevalence and risk factors for self-reported odour intolerance: the Skovde population-based study. International Archives of Occupational and Environmental Health, 78(7), 559–564. Johnson, J. L. and Vickers, Z. (1991). Sensory-specific satiety for selected bread products. Journal of Sensory Studies, 6(2), 65–79. Jones, L. V. and Thurstone, L. L. (1955). The psychophysics of semantics: an experimental investigation. Journal of Applied Psychology, 39(1), 31–36. Kadohisa, M. and Wilson, D. A. (2006). Olfactory cortical adaptation facilitates detection of odors against background. Journal of Neurophysiology, 95(3), 1888–1896. Kahkonen P., Hakanpaa P. and Tourila H. (1999). The effects of information related to fat content and taste on consumer responses to a reduced-fat frankfurter and a reduced-fat chocolate bar. Journal of Sensory Studies, 14, 35–46. Kahkonen P. and Tuorila H. (1998). Effect of reduced-fat information on expected and actual hedonic and sensory ratings of sausage. Appetite, 30, 13–23. Kahkonen P., Tuorila H. and Rita, H. (1996). How information enhances the acceptability of low-fat spread. Food Quality Preference, 7, 87–94. Keiger, C. J., Case, L. D., Kendal-Reed, M., et al. (2003). Nicotinic cholinergic receptor expression in the human nasal mucosa. Annals of Otology, Rhinology and Laryngology, 112(1), 77–84. Kepecs, A., Uchida, N. and Mainen, Z. F. (2006). The sniff as a unit of olfactory processing. Chemical Senses, 31, 167–179. Korsmeyer, C. (1999). Making sense of taste, food and philosophy. Ithaca: Cornell University Press. Korsmeyer, C. (2005). The taste culture reader, experiencing food and drink. Oxford: Berg. Koskinen, S., Kalviainen, N. and Tuorila, H. (2003). Flavor enhancement as a tool for increasing pleasantness and intake of a snack product among the elderly. Appetite, 41(1), 87–96. Kreutzer, R., Neutra, R. R., and Lashuay, N. (1999). Prevalence of people reporting sensitivities to chemicals in a population-based survey. American Journal of Epidemiology, 150, 1–12. Kroeze, J. H. A. (1990). The perception of complex taste stimuli. In: R. L. McBride and H. J. H. MacFie (Eds.) Psychological Basis of Sensory Evaluation, pp. 41–68. London: Elsevier Applied Science. Kroeze, J. H. A. and Bartoshuk, L. M. (1985). Bitterness suppression as revealed by split-tongue taste stimulation in humans. Physiology and Behavior, 35, 779–783. Labbe, D., Damevin, L., Vaccher, C., Morgenegg, C. and Martin, N. (2006). Modulation of perceived taste by olfaction in familiar and unfamiliar beverages. Food Quality and Preference, 17, 582–589. Labows, J. N. and Wysocki, C. J. (1984). Individual differences in odor perception. Perfumer and Flavorist, 9, 21–26. Laing, D. G., Panhuber, H., Wilcox, M. E. and Pittman, E. A. (1984). Quality and intensity of binary odor mixtures. Physiology and Behavior, 33, 309–319. Langron, S. P., Noble, A. C. and Williams, A. A. (1983). Measurement and Control. In: R. K. Atkin and A. A. Williams (Eds.) Sensory quality in foods and beverages: Definition, p. 325. Chichester: Ellis Horwood. Lanier, S. A., Hayes, J. E. and Duffy, V. B. (2005). Sweet and bitter tastes of alcoholic beverages mediate alcohol intake in of-age undergraduates. Physiology and Behavior, 83(5), 821–831. Largey, G. and Watson, R. (2006). The sociology of odors. In: J. Drobnick (Ed.) The smell culture reader, pp. 29–40. Oxford: Berg.

Taste, smell and chemesthesis

127

Laska, M., Hudson, R. and Distel, H. (1990). Olfactory sensitivity to biologically relevant odors may exceed the sum of component thresholds. Chemoecology, 1, 139–141. Lawless, H. T. (1979). Evidence for neural inhibition in bittersweet taste mixtures. Journal of Comparative and Physiological Psychology, 93, 538–547. Lawless, H. T. (1983). Contextual effects in category ratings. Journal of Testing and Evaluation, 11, 346–349. Lawless, H. T. (1987). Gustatory psychophysics. In: T. E. Finger and W. L. Silver (Eds.) Neurobiology of taste and smell, pp. 401–420. New York: John Wiley. Lawless, H. T. (1997). Olfactory psychophysics. In: G. Beauchamp and L. Bartoshuk (Eds.) Tasting and smelling, pp. 125–168. San Diego, CA: Academic Press. Lawless, H. T. and Heymann, H. (1998). Sensory evaluation of food: Principles and practices. New York: Chapman and Hall. Lawless, H. T., Rozin, P. and Shenker, J. (1985). Effects of oral capsaicin on gustatory, olfactory and irritant sensations and flavor identification in humans who regularly or rarely consume chilli pepper. Chemical Senses, 10, 579–589. Lawless, H. T. and Stevens, D. A. (1984). Effects of oral chemical irritation on taste. Physiology and Behavior, 32, 995–998. Lawless, H. T., Sheng, N. and Knoops, S. S. C. P. (1995). Multidimensional scaling of sorting data applied to cheese perception. Food Quality and Preference, 6, 91–98. Lemon, C. H. and Smith, D. V. (2006). Influence of response variability on the coding performance of central gustatory neurons. Journal of Neuroscience, 26(28), 7433–7443. Li, X., Li, W., Wang, H., et al. (2005). Pseudogenization of a sweet-receptor gene accounts for cats’ indifference toward sugar. PLoS Genetics, 1, 27–35. Liberles, S. D. and Buck, L. B. (2006). A second class of chemosensory receptors in the olfactory epithelium. Nature, 442(7103), 645–650. Lindemann, B. (2001). Receptors and transduction in taste. Nature, 413, 219–225. Liu, L. and Simon, S. A. (2001). Acidic stimuli activate two distinct pathways in taste receptor cells from rat fungiform papillae. Brain Research, 923, 58–70. Livermore, A. and Laing, D. G. (1996). Influence of training and experience on the perception of multicomponent odor mixtures. Journal of Experimental Psychology: Human Perception and Performance, 22, 267–277. Lledo, P. M., Gheusi, G. and Vincent, J. D. (2005). Information processing in the mammalian olfactory system. Physiological Reviews, 85, 281–317. Mainland, J. and Sobel, N. (2006). The sniff is part of the olfactory percept. Chemical Senses, 31, 181–196. Marchand, S. and Arsenault, P. (2002). Odors modulate pain perception: A gender-specific effect. Physiology and Behavior, 76(2), 251–256. Marshall, K., Laing, D.G., Jinks, A. L. and Hutchinson, I. (2006). The capacity of humans to identify components in complex odor-taste mixtures. Chemical Senses, 31, 539–545. Mathey, M. F., Siebelink, E., De Graaf, C. and Van Staveren, W. A. (2001). Flavor enhancement of food improves dietary intake and nutritional status of elderly nursing home residents. Journals of Gerontology Series A: Biological Sciences and Medical Sciences, 56(4), M200–M205. Matlin, M. W. and Foley, H. J. (1992). Sensation and perception. Boston: Allyn and Bacon. Mattes, R. D. (1987). Sensory influences on food intake and utilization in humans. Human Nutrition: Applied Nutrition, 41A, 77–95. McBride, R. L. (1985). Stimulus range influences intensity and hedonic ratings of flavour. Appetite, 6, 125–131. McBride, R. L. and Johnson, R. L. (1987). Perception of sugar-acid mixtures in lemon juice drink. International Journal of Food Science and Technology, 22, 299–408. McBurney, D. H. and Shick, T. R. (1971). Taste and water taste of twenty-six compounds for man. Perception and Psychophysics, 10, 249–252. McClure, S. M., Li, J., Tomli, D., Cypert, K. S., Montagu, L. M. and Montague, P. R. (2004). Neural correlates of behavioral preference for culturally familiar drinks. Neuron, 44, 379–387. McDaniel, A. H. and Reed, D. R. (2004). The human sweet tooth and its relationship to obesity. In: N. Moustaid-Moussa and C. Berdanier (Eds.) Genomics and proteomics in nutrition, pp. 49–67. New York: Marcel Dekker. McKemy, D. D. (2005). How cold is it? TRPM8 and TRPA1 in the molecular logic of cold sensation. Molecular Pain, 1, 1–7. Meilgaard, M., Civille, G. V. and Carr, B. T. (1999). Sensory evaluation techniques. Boca Raton, Fl: CRC Press. Mennella, J. A., Johnson, A. and Beauchamp, G. K. (1995). Garlic ingestion by pregnant women alters the odor of amniotic fluid. Chemical Senses, 20, 207–209. Mennella, J. A., Pepino, M. Y. and Reed, D. R. (2005). Genetic and environmental determinants of bitter perception and sweet preferences. Pediatrics, 115(2), e216–e222.

128

Product Experience

Midkiff, E. E. and Bernstein, I. L. (1985). Targets of learned food aversions in humans. Physiology and Behavior, 34, 839–841. Mojet, J., Christ-Hazelhof, E. and Heidema, J. (2001). Taste perception with age: Generic or specific losses in threshold sensitivity to the five basic tastes? Chemical Senses, 26, 845–860. Mojet, J., Heidema, J. and Christ-Hazelhof, E. (2003). Taste perception with age: Generic or specific losses in supra-threshold intensities to the five taste qualities? Chemical Senses, 28, 397–413. Mombaerts, P. (2004). Genes and ligands for odorant, vomeronasal, and taste receptors. Nature Reviews: Neuroscience, 5, 263–278. Moncreiff, R. W. (1966). Odour preferences. New York: Wiley. Moskowitz, H. R. and Barbe, C. D. (1977). Profiling of odor components and their mixtures. Sensory Processes, 1, 212–226. Moskowitz, H. R. and Sidel, J. L. (1971). Magnitude and hedonic scales of food acceptability. Journal of Food Science, 36, 677–680. Mueller, K. L., Hoon, M. A., Erlenbach, I., et al. (2005). The receptors and coding logic for bitter taste. Nature, 434, 225–229. Murphy, C. (1982). Effects of exposure and context on hedonics of olfactory–taste mixtures. Sensory Processes, 1, 212–226. Murphy, C. (1986). Taste and smell in the elderly. In: H. L. Meiselman and R. S. Rivlin (Eds.) Clinical measurement of taste and smell, pp. 343–375. New York: Macmillan. Murphy, C. (1993). Nutrition and chemosensory perception in the elderly. Critical Reviews in Food Science and Nutrition, 33(1), 3–15. Murphy, C. and Cain, W. S. (1980). Taste and olfaction: Independence versus interaction. Physiology Behavior, 24, 601–605. Murphy, C., Schubert, C. R., Cruickshanks, K. J., et al. (2002). Prevalence of olfactory impairment in older adults. Journal of the American Medical Association, 288, 2307–2312. Nasrawi, C.W. and Pangborn, R.M. (1989). The influence of tastants on oral irritation by capsaicin. Journal of Sensory Studies, 3, 287–294. Navarrete-Palacios, E., Hudson, R., Reyes-Guerrero, G. and Guevara-Guzman, R. (2003). Lower olfactory threshold during the ovulatory phase of the menstrual cycle. Biological Psychology, 63(3), 269–279. O’Doherty, J., Rolls, E. T., Francis, S., et al. (2000). Sensory-specific satiety-related olfactory activation of the human orbitofrontal cortex. Neuroreport, 11, 399–403. Olson J.C. and Dover P. (1976). Effects of expectation creation and disconfirmation on belief elements of cognitive structure. In: B. B. Anderson (Ed.) Advances in Consumer Research, 3, 168–175. Association for Consumer Research. O’Mahoney, M. (1978). Smell illusions and suggestion: reports of smells contingent on tones played on radio and television. Chemical Senses and Flavor, 3, 183–189. Parducci, A. (1974). Contextual effects: a range-frequency analysis. In: E. C. Carterette and M. P. Friedman (Eds.) Handbook of perception, psychophysical judgment and measurement, pp. 127–141. New York: Academic. Pause, B. M., Sojka, B., Krauel, K., Fehm-Wolfsdorf, G. and Ferstl, R. (1996). Olfactory information processing during the course of the menstrual cycle. Biological Psychology, 44(1), 31–54. Peier, A. M., Moqrich, A., Hergarden, A. C., Reeve, A. J., Andersson, D. A., Story, G. M., et al. (2002). A TRP channel that senses cold stimuli and menthol. Cell, 108, 705–715. Pelchat, M. L. and Rozin, P. (1982). The special role of nausea in the acquisition of food dislikes by humans. Appetite, 3, 341–351. Perng, C. M. and McDaniel, M. R. (1989). Optimization of a blackberry juice drink using response surface methodology. Presented at the 49th Annual Meeting of the Institute of Food Technologists. Chicago, IL. Peryam, D. R. and Pilgrim, F. J. (1957). Hedonic scale method of measuring food preferences. Food Technology, 11, 9–14. Pierce, J. D., Jr., Wysocki, C. J., Aronov, E. V., Webb, J. B. and Boden, R. M. (1996). The role of perceptual and structural similarity in cross-adaptation. Chemical Senses, 21, 223–237. Pierce, J. D., Jr., Zeng, X. N., Aronov, E. V., Preti, G. and Wysocki, C. J. (1995). Cross-adaptation of sweatysmelling 3-methyl-2-hexenoic acid by a structurally-similar, pleasant-smelling odorant. Chemical Senses, 20, 401–411. Pliner, P. (1983). Family resemblance in food preferences. Journal of Nutrition Education, 15, 137–140. Pol, H. E. H., Hijman, R., Baare, W. F. C. and van Ree, J. M. (1998). Effects of context on judgements of odor intensities in humans. Chemical Senses 23, 131–135. Popper, R. and Heymann, H. (1996). Analyzing differences among products and panelists by multidimensional scaling. In: T. Naes and E. Risvik (Eds.) Multivariate analysis of data in sensory science, pp. 159–184. Amsterdam: Elsevier.

Taste, smell and chemesthesis

129

Porter, R. H., Makin, J. W., Davis, L. B. and Christensen, K. M. (1992). Breast-fed infants respond to olfactory cues from their own mother and unfamiliar lactating females. Infant Behavior and Development, 15, 85–93. Powers, J. J. (1988). Mathematics of a lady tasting tea revisited. Journal of Sensory Studies, 3, 151–158. Prescott, J. and Swain-Campbell, N. (2000). Responses to repeated oral irritation by capsaicin, cinnamaldehyde and ethanol in PROP tasters and non-tasters. Chemical Senses, 25, 239–246. Preti, G., Wysocki, C. J., Barnhart, K. T., Sondheimer, S. J. and Leyden, J. J. (2003). Male axillary extracts contain pheromones that affect pulsatile secretion of luteinizing hormone and mood in women recipients. Biology of Reproduction, 68, 2107–2113. Price, J. L. (1973). An autoradiographic study of complementary laminar patterns of termination of afferent fibers to the olfactory cortex. Journal of Comparative Neurology, 150, 87–108. Rawcliffe, T., Howgate, P. and Geelhoed, E. N. (1992). Patterns of odours similarity among carbonyls and their mixture. Chemical Senses, 17, 119–125. Rawson, N. E. and Yee, K. K. (2006). Transduction and coding. Advances in Oto-Rhino-Laryngology, 63, 23–43. Rifkin, B. and Bartoshuk, L. M. (1980). Taste synergism between monosodium glutamate and disodium 5⬘-guaylate’. Physiology and Behavior, 24, 1169–1172. Rodriguez, I., Greer, C. A., Mok, M. Y. and Mombaerts, P. (2000). A putative pheromone receptor gene expressed in human olfactory mucosa. Nature Genetics, 26, 18–19. Rolls, B. J. (1999). Do chemosensory changes influence food intake in the elderly? Physiology and Behavior, 66(2), 193–197. Rolls, E. T. (2000). The orbitofrontal cortex and reward. Cerebral Cortex, 10, 284–294. Rolls, E. T. (2001). The rules of formation of the olfactory representations found in the orbitofrontal cortex olfactory areas in primates. Chemical Senses, 26, 595–604. Rolls, E. T. and Baylis, L. L. (1994). Gustatory, olfactory, and visual convergence within the primate orbitofrontal cortex. Journal of Neuroscience, 14, 5437–5452. Rolls, B. J., Hetherington, M. and Burley, V. J. (1988). The specificity of satiety: the influence of foods of different macronutrient content on the development of satiety. Physiology and Behavior, 43, 145–153. Rolls, B. J., Rolls, E. T., Rowe, E. A. and Sweeney, K. (1981). Sensory specific satiety in man. Physiology and Behavior, 27, 137–142. Rozin, P. (1988). Social learning about foods by humans. In: T. Zental and B. G. Galef, Jr. (Eds.) Social learning: A comparative approach, pp. 165–187. Hillsdale, NJ: Erlbaum. Rozin, P. (1996). Sociocultural influences on human food selection. In: E. Capaldi (Ed.) Why we eat what we eat: The psychology of eating, pp. 233–263. Washington, DC: American Psychological Association. Rozin, P., Markwith, M. and Stoess, C. (1996). Moralization and becoming a vegetarian: the transformation of preferences into values and the recruitment of disgust. Psychological Science, 8, 67–73. Rozin, P. and Schiller, D. (1980). The nature and acquisition of a preference for chili pepper by humans. Motivation and Emotion, 4, 77–100. Rozin, P. and Tuorila, H. (1993). Simultaneous and temporal contextual influences on food acceptance. Food Quality and Preference, 4, 11–20. Rozin, P. and Vollmecke, T. A. (1986). Food likes and dislikes. Annual Review of Nutrition, 6, 433–456. Santos, D. V., Reiter, E. R., DiNardo, L. J. and Costanzo, R. M. (2004). Hazardous events associated with impaired olfactory function. Archives of Otolaryngology – Head and Neck Surgery, 130, 317–319. Sarinopoulos, I., Dixon, G. E., Short, S. J., Davidson, R. J. and Nitschke, J. B. (2006). Brain mechanisms of expectation associated with insula and amygdala response to aversive taste: implications for placebo. Brain, Behavior, and Immunity, 20, 120–132. Sauvageot, F., Urdapilleta, I. and Peyron, D. (2006). Within and between variations of texts elicited from nine experts in wines. Food Quality and Preference, 17(6), 429–444. Savic, I., Gulyas, B. and Berglund, H. (2002). Odorant differentiated pattern of cerebral activation: comparison of acetone and vanillin. Human Brain Mapping, 17, 17–27. Schaal, B., Marlier, L. and Soussignan, R. (1998). Olfactory function in the human fetus: Evidence from selective neonatal responsiveness to the odor of amniotic fluid. Behavioral Neuroscience, 112, 1438–1449. Schafe, G. E. and Bernstein, I. L. (1996). Taste aversion learning. In: E. Capaldi (Ed.) Why we eat what we eat: the psychology of eating, pp. 31–52. Washington, D.C.: American Psychological Association. Scharf, A. and Volkmer, H. P. (2000). The impact of olfactory product expectations on the olfactory product experience. Food Quality and Preference, 11, 497–504. Schifferstein, H. N. J. (1995). Contextual shifts in hedonic judgments. Journal of Sensory Studies, 10, 381–392. Schifferstein, H. N. J. (2001). Effects of product beliefs on product perception and liking. In: L. Frewer, E. Risvik. and H. Schifferstein. Food, people and society: A European perspective of consumers’ food choices, London: Springer Verlag. Schifferstein, H. N. J. and Frijters, J. E. R. (1992). Contextual and sequential effects on judgments of sweetness intensity. Perception and Psychophysics, 52(3), 243–255.

130

Product Experience

Schifferstein, H. N. J. and Oudejans, I. M. (1996). Determinants of cumulative successive contrast in saltiness intensity judgments. Perception and Psychophysics, 58(5), 713–724. Schiffman, S.S. et al. (1995). Investigation of synergism in binary mixtures of sweeteners. Brain Resource Bulletin, 38, 105–120. Schiffman, S. S. (2000). Intensification of sensory properties of foods for the elderly. Journal of Nutrition, 130(Suppl. 4S), 927S–930S. Schleidt, M., Hold, B. and Attila, G. (1981). A cross-cultural study on the attitude towards personal odors. Journal of Chemical Ecology, 7, 19–31. Schmidt, H. J. and Beauchamp, G. K. (1988). Adult-like preferences and aversions in three-year-old children. Child Development, 59, 1136–1143. Schutz, H. G. and Cardello, A. V. (2001). A labeled affective magnitude (LAM) scale for assessing food liking/ disliking. Journal of Sensory Studies, 16, 117–159. Shinohara, K., Morofushi, M., Funabashi, T. and Kimura, F. (2001). Axillary pheromones modulate pulsatile LH secretion in humans. Neuroreport, 12, 893–895. Shusterman, D. (2002). Individual factors in nasal chemesthesis. Chemical Senses, 27, 551–564. Simoons, F. J. (1961). Eat not this flesh. Madison: University of Wisconsin Press. Siret, F. and Issanchou, S. (2000). Traditional process: influence on sensory properties and on consumers’ expectation and liking. Application to pate’ de campagne. Food Quality and Preference, 11, 217–228. Slosson, E. E. (1899). A lecture experiment in hallucinations. Psychology Review, 6, 407–408. Small, D. M. (2006). Central gustatory processing in humans. In: T. Hummel and A. Welge-Lüssen (Eds.) Taste and smell: An update (Advances in Oto-rhino-laryngology), pp. 191–220. Basel, Germany: Karger. Small, D. M., Gregory, M. D., Mak, Y. E., et al. (2003). Dissociation of neural representation of intensity and affective valuation in human gustation. Neuron, 39, 701–711. Small, D. M., Jones-Gotman, M., Zatorre, R. J., Petrides, M. and Evans, A. C. (1997). Flavor processing: More than the sum of its parts. Neuroreport, 8, 3913–3917. Small, D. M., Voss, J., Mak, Y. E., Simmons, K., Parrish, T. and Gitelman, D. (2004). Experience-dependent neural integration of taste and smell in the human brain. Journal of Neurophysiology, 92, 1892–1903. Smith, D. V. and Margolskee, R. F. (2001). Making sense of taste. Scientific American, 284, 32–39. Smith, D. V. and Shepherd, G. M. (1999). Chemical senses: Taste and olfaction. In: M. J. Zigmond, F. E. Bloom, S. C. Landis, et al. (Eds.) Fundamentals of neuroscience, pp. 719–760. San Diego, CA: Academic Press. Smith-Swintosky, V. L., Plata-Salaman, C. R. and Scott, T. R. (1991). Gustatory neural coding in the monkey cortex: Stimulus quality. Journal of Neurophysiology, 66, 1156–1165. Sobel, N., Prabhakaran, V., Desmond, J. E., et al. (1998). Sniffing and smelling: Separate subsystems in the human olfactory cortex. Nature, 392, 282–286. Steenkamp, J. B. E. M. and van Trijp, H. C. M. (1988). Free-choice profiling in cognitive food acceptance research. In: D. M. H. Thomson (Ed.) Food acceptability, pp. 363–376. London: Elsevier Applied Science. Stefani G., Romano D. and Cavicchi A. (2006). Consumer expectations, liking and willingness to pay for specialty foods: Do sensory characteristics tell the whole story? Food Quality and Preference, 17, 53–62. Stein, L. J., Nagai, H., Nakagawa, M. and Beauchamp, G. K. (2003). ‘Effects of repeated exposure and healthrelated information on hedonic evaluation and acceptance of a bitter beverage.’ Appetite 40(2),119–129. Steiner, J. E. (1977). Facial expressions of the neonate infant indicating the hedonics of food-related chemical stimuli. In: J. M. Weiffenbach (Ed.) Taste and development: The genesis of sweet preference, pp. 173–189. Bethesda, MD: DHEW Publication No. (NIH) 77-1068, US Department of Health, Education, and Welfare. Stern, K. and McClintock, M. K. (1998). Regulation of ovulation by human pheromones. Nature, 392, 177–179. Stevens, D. A. and Lawless, H. T. (1986). Putting out the fire: Effects of tastants on oral chemical irritation. Perception and Psycholophysics, 39, 346–350. Stevens, J. C. and Cain, W. S. (1993). Changes in taste and flavor in aging. Critical Reviews in Food Science and Nutrition, 33, 27–37. Stevens, S. S. and Galanter, E. H. (1957). Ratio scales and category scales for a dozen perceptual continua. Journal of Experimental Psychology, 54(6), 377–411. Stevenson, R. J., Prescott, J. and Boakes, R. A. (1995). The acquisition of taste properties by odors. Learning and Motivation, 26, 433–455. Stevenson, R. J., Prescott, J. and Boakes, R. A. (1999). Confusing tastes and smells: how odours can influence the perception of sweet and sour tastes. Chemical Senses, 24, 627–635. Stevenson, R. J. and Yeomans, M. R. (1993). Differences in ratings of intensity and pleasantness for the capsaicin burn between chili likers and non-likers; implications for liking development. Chemical Senses, 18, 471–482. Stone, H. and Sidel, J. L. (2004). Sensory evaluation practices (3rd Ed.) New York: Academic Press. Stone, H., Sidel, J. L., Oliver, S. Woolsey, A. and Singleton, R. C. (1974). Sensory evaluation by qualitative descriptive analysis. Food Technology, 28, 24–34.

Taste, smell and chemesthesis

131

Stuck, B. A., Frey, S., Freiburg, C., et al. (2006). Chemosensory event-related potentials in relation to side of stimulation, age, sex, and stimulus concentration. Clinical Neurophysiology, 117(6), 1367–1375. Swithers, S. E. and Hall, W. G. (1994). Does oral experience terminate ingestion? Appetite, 23(2), 113–138. Tang, C. and Heymann, H. (2002). Multidimensional sorting, similarity scaling and free-choice profiling of grape jellies. Journal of Sensory Studies, 17(6), 493–510. Tarun, A., Bryant, B., Zhai, W., Soloman, C. and Shusterman, D. (2003). Gene expression for carbonic anhydrase isoenzymes in human nasal mucosa. Chemical Senses, 28, 621–629. Todrank, J., Wysocki, C. J. and Beauchamp, G. K. (1991). The effects of adaptation on the perception of similar and dissimilar odors. Chemical Senses, 16, 467–482. Tuorila H., Andersson A., Martikainen A. and Solovaara H. (1998). Effect of product formula, information and consumer characteristics on the acceptance of a new snack food. Food Quality and Preference, 9, 313–320. Tuorila, H., Cardello, A. V. and Leshe, L. L. (1994). Antecedents and consequences of expectations related to fat-free and regular-fat foods. Appetite, 23, 247–263. Turin, L. (2006). Accords and discords: perfume reviews. In: J. Drobnick (Ed.) The Smell Culture Reader, pp. 216–227. New York; Oxford: Berg. Von Sydow, E., Moskowitz, H., Jacobs, H. and Meiselman, H. (1974). Odor–taste interaction in fruit juices. Lebensmittel Wissenschaft und Technologie, 7, 9–16. Wansink, B. and Park, S. (2002). Sensory suggestiveness and labeling: Do soy labels bias taste? Journal of Sensory Studies, 17, 483–491. Wansink, B., van Ittersum, K. and Painter, J. E. (2005). How descriptive food names bias sensory perceptions in restaurants. Food Quality and Preference, 16, 393–400. Watson, J. L. and Caldwell, M. L. (2005). The cultural politics of food and eating: A reader. Oxford: Blackwell Publishing. Williams, A. A. and Langron, S. P. (1984). The use of free-choice profiling for the evaluation of commercial ports. Journal of the Science of Food and Agriculture, 35, 558–568. Winberg, J. and Porter, R. H. (1998). Olfaction and human neonatal behaviour: Clinical implications. Acta Paediatrica, 87, 6–10. Wise, P. M., Radil, T. and Wysocki, C. J. (2003). Time-intensity ratings of nasal irritation from carbon dioxide. Chemical Senses, 28, 751–760. Wysocki, C. J. and Beauchamp, G. K. (1984). Ability to smell androstenone is genetically determined. Proceedings of the National Academy of Sciences of the United States of America, 81, 4899–4902. Wysocki, C. J. and Beauchamp, G. K. (1991). Individual differences in human olfaction. In: C. J. Wysocki and M. R. Kare (Eds.) Chemical senses: genetics of perception and communication, pp. 353–373. New York: Marcel Dekker. Wysocki, C. J., Cowart, B. J. and Radil, T. (2003) Nasal-trigeminal chemosensitivity across the adult lifespan. Perception and Psychophysics, 65, 115–122. Wysocki, C. J. and Gilbert, A. N. (1989). The National Geographic Smell Survey: The effects of age are heterogeneous. In: C. L. Murphy, W. S. Cain, and D. M. Hegsted (Eds.) Chemical senses and nutrition in aging, pp. 12–28. New York: New York Academy of Sciences. Wysocki, C. J. and Preti, G. (2004). Facts, fallacies, fears, and frustrations with human pheromones. The Anatomical Record Part A: Discoveries in Molecular, Cellular, and Evolutionary Biology, 281(1), 1201–1211. Wysocki, C. J. and Wise, P. (2003). Methods, approaches, and caveats for functionally evaluating olfaction and chemesthesis. In: K. Deibler and J. F. Delwiche (Eds.) Handbook of flavor characterization: Sensory, chemical and physiological, pp. 1–40. New York: Marcel Dekker. Yackinous, C. A. and Guinard, J. X. (2002). Relation between PROP (6-n-propylthiouracil) taster status, taste anatomy and dietary intake measures for young men and women. Appetite, 38(3), 201–209. Yang, H. H. and Lawless, H. T. (2005). Descriptive analysis of divalent salts. Journal of Sensory Studies, 20, 97–113. Zellner, D. A., Rozin, P., Aron, M. and Kulish, C. (1983). Conditioned enhancement of humans’ liking for flavor by pairing with sweetness. Learning and Motivation, 14, 338–350. Zhao, G. Q., Zhang, Y., Hoon, M. A., et al. (2003). The receptors for mammalian sweet and umami taste. Cell, 115, 255–266. Zou, Z. and Buck, L. B. (2006). Combinatorial effects of odorant mixes in olfactory cortex. Science, 311, 1477–1481. Zufall, F. and Leinders-Zufall, T. (1997). Identification of long-lasting form of odor adaptation that depends on the carbon monoxide/cGMP second-messenger system. Journal of Neuroscience, 8, 2703–2712.

5

MULTISENSORY PRODUCT EXPERIENCE HENDRIK N.J. SCHIFFERSTEIN Delft University of Technology, Delft, The Netherlands

CHARLES SPENCE University of Oxford, Oxford, UK

1. INTRODUCTION People use all of their senses in order to explore the world around them. Under normal everyday conditions the senses all work together to create the overall product experiences that fill a typical person’s life. For example, a man who drives a car will determine its position on the road by means of the objects that he sees through the windshield. He can ascertain the car’s speed from the numbers he reads on the dashboard, the speed at which he sees the trees passing him by on the side of the road, the sounds made by the car’s engine, and/or by means of the vibrations he feels through his body. He can judge the stability of the car from the way in which the vehicle responds to his actions: The changes in velocity he feels when he changes gear, the forces he experiences when changing lane. In general, whenever we use a product, we perform actions on (or with) that product and our senses provide us with feedback regarding how the product, or the environment, reacts to those actions. By now, many different studies have suggested that the greater the number of sensory modalities that are stimulated at any one time, the richer our experiences will be (e.g. Bahrick and Lickliter, 2000; Spence, 2002; Stein and Meredith, 1993). As a consequence, increasing the number of modalities of sensory input presented in a virtual environment can help to increase people’s sense of presence and also increase their memory for objects placed within the virtual environment (e.g. Dinh et al., 1999; Hoffman et al., 1998; Washburn et al., 2003). An interesting application of this was recently described by Vlahos (2006), who reported that the addition of a scent collar to standard virtual-reality equipment (e.g. goggles offering a stereoscopic view, headphones providing binaural sounds, and movement sensors) can help to create an immersive multisensory Product Experience Copyright © 2008 Elsevier Ltd.

133

134

Product Experience

environment in which soldiers can be prepared for the kinds of situations that they may subsequently encounter in a war zone. Soldiers on the ground generally find themselves in an unknown area under circumstances that are unfamiliar and often highly stressful. On the one hand, the unusual smells produced by exploding bombs and mines, burning trucks and buildings, and decaying corpses may overwhelm the newly-deployed soldiers and may interfere with the completion of their mission. On the other hand, the olfactory cues may provide information that is vital for the soldier’s survival: Burning wires may, for example, indicate an electrical problem in an aircraft, while the smell of cigarettes may signal the presence of the enemy. Despite the important role that the ensemble of sensory modalities plays in a consumer’s interactions with the environment, very few industrial companies have attempted to make full use of the multisensory potential of the products that they market/produce (see Hine, 1995; Lindstrom, 2005). For a product designer, it is important to know what kind of actions people will perform with a product, how they will perceive it during these interactions, and how the consumer’s senses work together to deliver rich and varied multisensory experiences. This knowledge can then be applied to the design of new products that will more effectively stimulate the senses of the consumer, and hence lead to more pleasurable and memorable multisensory product interactions. In this chapter, we highlight the roles of the various senses, and their interplay when people interact with different products. Besides highlighting the key theoretical debates in this area, our discussion will center on empirical data gathered in well-controlled experimental studies, as well as on survey data. This chapter also highlights a number of examples where the theoretical principles of multisensory perception have actually been incorporated and tested in the design of everyday products. In each of the seven sections that follow, we start out by describing a particular topic and then critically discuss a number of scientific studies that have investigated this topic. Typically, each topic can be characterized not only by its content, but also by the specific approaches and research methods that are used. Additionally, where possible we outline how this knowledge has been used or can be used in the development of new products. In the first three sections we consider the modalities as if they were separate systems. First, we evaluate the roles of the various sensory modalities in people’s interactions with products during everyday life (Section 2) and in mental imagery (Section 3). Next, we review what happens when people switch their attention between different sensory systems (Section 4). Given that the senses typically do not work in isolation, but rather operate as an integrated whole, the next sections discuss the links that people experience intuitively between phenomena occurring in different sensory modalities (Section 5), and the ways in which sensory information from the different modalities is integrated into a holistic product experience. While Section 6 discusses a number of interactions that have been observed between various forms of sensory stimulation, Section 7 focuses specifically on the role of sensory (in)congruity in these interactions. Section 8 highlights what is currently known about the factors that determine the relative importance people attach to each of their senses in various situations. Finally, at the end of the chapter, we highlight some of the important emerging challenges currently facing researchers and designers in the area of multisensory product design.

2. COMPARING THE DIFFERENT SENSORY MODALITIES Each of our senses is most sensitive to a different type of stimulation (energy). Because each sensory modality may be considered as a separate information channel, not all of

135

Multisensory product experience

the incoming sensory information will necessarily communicate the same message to a person. For instance, whereas some information is only used to perform a task with a product, other information may also evoke memories or associations to other objects, or may trigger feelings and emotions. It is important to note that product experiences are based on all of the incoming sensory information, no matter whether it is perceived consciously by a person or not. The discussion outlined here therefore raises the question of what roles the different senses play in people’s everyday lives. In order to obtain a more detailed insight into the similarities and differences between the various senses in modulating our multisensory product experiences, Schifferstein and Cleiren (2005) developed a split-modality approach. The participants in their study were presented with six simple products (a marker pen, a spray deodorant, a tennis ball, a bag of crisps, a boiled egg, and a can of soft drink) via only a single sensory modality at a time. The active interaction between user and product was choreographed in equivalent ways for the four modalities: A prerecorded video (vision) or sound (audition); a product presented in a sound-isolated box (touch); or a smell contained in a glass jar (olfaction). For the particular range of products studied by Schifferstein and Cleiren (2005), vision and touch turned out to be approximately equally successful in providing participants with detailed information concerning a product; audition proved somewhat less useful, and olfaction provided the least detailed information. Furthermore, products perceived by vision and touch were found to be the easiest to identify, and yielded the clearest memories of previous events and associations to persons and other products (see Table 5.1). In a complementary experimental study, Schifferstein and Desmet (2007) assessed the roles of the various senses on people’s perception of different everyday products by comparing the effects of blocking one sense. They found that preventing people from seeing the products had the most detrimental effect on the amount of functional product information that they perceived. Because fewer product features are perceived simultaneously, and a considerable proportion of the product-related information is lost when one cannot see a product, task difficulty and task duration typically increase, up to the point at which simple tasks can no longer be completed without help from another person. Interestingly, when products cannot be seen, people report that their experiences become more intense and that they start to use their other senses more. When tactual perception was blocked to some degree (in this case by wearing very thick gloves), a substantial amount of product information was lost as well. Similar to vision, perceived task difficulty and task duration increased significantly. Tasks requiring subtle coordinated movements (such as composing an SMS message on a mobile phone) became almost impossible to perform. In addition, an emotional dimension of tactual product experiences was revealed: Familiar products felt strange; they did not feel familiar anymore. It seems as if through blocking tactual perception one becomes somewhat alienated from one’s own surroundings. TABLE 5.1 Mean evaluation responses for the products experienced by participants in Schifferstein and Cleiren’s (2005) study. Higher values indicate that participants found a particular aspect more applicable

Detailed information Difficulty of identification Clarity of memory associated with the event

Scale type

Vision

Touch

Sound

Smell

7-point 7-point 4-point

6.2 1.3 3.2

6.0 1.7 3.3

4.7 3.7 2.9

3.0 4.6 2.8

136

Product Experience

Furthermore, Schifferstein and Desmet (2007) found that blocking auditory perception primarily resulted in communication problems: People felt cut-off from the outside world (see also Ramsdall, 1978). Blocking olfaction only led to a reported decrease in appetite for foods (see also Blomqvist et al., 2004). Hence, blocking the ears or the nostrils did not interfere much with functional product usage. However, it did decrease how good, how stimulating, and how intense the products were experienced as being. Therefore, consumers’ emotional product experiences nevertheless seem to suffer when audition or olfaction is blocked. When taken together with the existing literature, the two studies discussed above indicate that visual information is primarily important in user-product interactions, because it seems to be used mainly for the functional interaction, that is for executing tasks. In general, vision provides the largest amount of information on a product within the shortest time frame. Furthermore, visual input seems to be linked most directly to stored knowledge, such as information regarding the method of production, region of origin, and product safety (e.g. see Burns et al., 1995; Hinton and Henley, 1993). This large quantity of information most likely attracts the majority of a consumer’s attentional resources, which leaves fewer resources available for the processing of any other sensory experiences. This may explain why people claim that they use their other senses more after vision has been blocked. Given that olfaction and touch are often regarded as being our more ‘emotional senses’ (see Spence, 2002), this may help to explain the somewhat counterintuitive finding that products are experienced ‘more intensely’ when vision is denied. Similar to visual exploration, through touching people gather a lot of information about a product. On the one hand, this makes it relatively easy for people to identify many common objects by touch alone (e.g. Klatzky, Lederman and Metzger, 1985). On the other hand, this information is very helpful during functional product usage. Furthermore, tactual experiences are likely to possess a substantial emotional component, given the key role that touch plays in mediating interpersonal intimacy (e.g. Bolanowski, Verrillo and McGlone, 1999; Field, 2001). Given that audition is critical in verbal communication by means of which factual information may be effectively distributed to others, one might perhaps expect that sounds play an analogous role in communicating factual information about products. However, the studies discussed above indicate that product sounds play only a very limited role in our functional interaction with products. Because audition plays an important part in the expression of emotion, for example in the non-verbal aspects in human speech (e.g. Scherer, 2003) and in music (e.g. Herz, 1998; Krumhansl, 2002), product sounds may nevertheless affect the emotional product expression. The functional role of taste and smell is evident in people’s everyday interactions with food products, such as those occurring when people prepare and eat a meal. In addition, olfactory cues play a functional role in judging whether or not a person, object, or space is clean and safe. This is illustrated by the importance people attach to the smell of personal care and cleaning products (Schifferstein, 2006). As far as consumers’ interactions with non-food products are concerned, however, Schifferstein and Desmet’s (2007) study suggests that blocking olfaction does not have a deleterious effect on the functional execution of many everyday tasks. Olfactory cues may nevertheless have a major effect on a consumer’s emotional reaction to these products. In general, olfactory cognition seems to be dominated by the affective dimension (e.g. Engen, 1982). What’s more, memories elicited by odors tend to be more emotional in character than memories elicited by other types of stimuli (e.g. Herz and Schooler, 2002). Odor pleasantness even plays an important role in the way people categorize odors. It is, however, important to note that many of our associations with smells are

Multisensory product experience

137

idiosyncratic in nature (Ayabe-Kanamura et al., 1998; Dubois, 2000). The consequent variation in a consumer’s associations with a particular olfactory experience/attribute may, therefore, be expected to show a much more profound variation across individuals/ cultures than that found for visual, auditory, or tactual stimuli. In conclusion, all of the human senses contribute to how a product is experienced and are, therefore, important to consider in the design process. Insight into the roles that the different senses play in people’s interactions with products can help designers to choose the optimum sensory channel by which to communicate a certain message to a consumer (Lindstrom, 2005). For example, because the olfactory modality seems to be most closely linked to emotional experiences, designers who want to influence the emotions elicited by their products may want to consider altering the product’s smell. Although changes in appearance may be linked more explicitly and more consistently to shifts in specific emotional associations (Desmet, 2002) than smells, changes in the smell of a product may have a more direct impact on the emotional experience evoked. As a result, a specially designed smell could prove to be a powerful reinforcer of the emotions activated by the product’s appearance. Furthermore, because touch plays such an important role in perceiving a product as one’s own versus ‘foreign’, the use of distinctive tactual details may provide an effective way to generate differential advantage that operates on a more personal level than that obtained with visual features. In this way, manufacturers’ consistent use of particular tactual features could generate distinctive and appealing tactual brand images for products such as hand tools, mobile phones, or shavers.

3. SENSORY IMAGERY The way in which people experience multisensory products is not determined solely by their physical interactions with the products themselves. Part of the experience is often formed in a consumer’s mind prior to their actually buying or using a given product. That is, they may already imagine what it would feel like to use the product or to show it off to friends. Consumers may fantasize about how soft a new mattress will feel, how dazzling the smell of a new bath oil would be, how great they would look in that new pair of jeans, how delicious a piece of cheesecake will taste, or how magnificent that new stereo will sound. While for most people mental imagery may be most vivid for the case of visual images (Kosslyn, 1994), it is important to note that people can generate rich mental images for the other senses as well (e.g. Klatzky, Lederman and Matula, 1991; Reisberg, 1992; Stevenson and Case, 2005). That said, it nevertheless seems legitimate to ask whether all sensory images can be formed with the same ease and, if so, whether they are all equally vivid? Any test of mental imagery that prespecifies a number of products or events (e.g. Betts, 1909) is unlikely to provide an unbiased test of whether, for example, olfactory images are any less clear and vivid than visual and tactual images, because the clarity of a mental image may be at least partly product-dependent. With this in mind, Schifferstein (2007) instructed the participants in his study to imagine a product with a conspicuous (or specific) sensory characteristic (i.e., appearance, sound, feel, smell, or taste), without specifying a particular event or product category. By using this instruction, the products chosen were likely to vary between sensory modalities (see Table 5.2), but the roles of the sensory stimuli in everyday life would, as far as possible, be matched. The vividness ratings obtained under such conditions were only slightly, but significantly, lower for smell and taste than for the other sensory modalities: Whereas mean vividness ratings for

138

Product Experience

TABLE 5.2 Frequency with which objects in various categories were imagined for five sensory modalities (N ⴝ 50 participants) (adapted from Schifferstein, 2007)

Vehicles Electrical apparatuses Interior product Toys and sports gear Tools Musical instruments Care products Foods and beverages Surroundings (industry, nature) Art, culture, music Person Other

Vision

Touch

Audition

Smell

Taste

13 9 4 5 0 1 0 2 10 5 1 0

4 9 4 6 6 2 6 3 3 1 5 1

12 8 7 0 8 0 0 0 2 9 4 0

1 0 1 0 2 1 7 15 20 0 2 1

0 0 0 0 1 0 3 46 0 0 0 0

vision, touch, and audition ranged from 5.4 to 5.7, the ratings for taste and smell were both 4.9 on a seven-point scale. In a follow-up experiment, Schifferstein (2007) instructed his participants to imagine interacting with eight different products. He asked them to imagine what the product would smell like, what it would sound like, what it would look like, and finally how it would feel to interact with the particular product. When the mean imagery vividness ratings were correlated with the mean importance ratings for these sensory modalities during product usage (see Schifferstein, 2006), significant positive correlations were observed for touch, audition, and olfaction. This suggests that for three out of the four sensory modalities under investigation in Schifferstein’s study, the vividness of a characteristic part of a mental image was related to how important people found that aspect of the product when they interacted with it. The lack of any significant correlation in the case of vision may be due to the fact that visual attributes were important for the perception of nearly all products. The role of sensory imagery in product experience is particularly important in situations in which products cannot be experienced physically, such as when products are advertised or when they are sold through catalogs or through virtual stores. In some cases, providing a particular kind of sensory input in an available modality (typically vision or audition) may be used to create a cross-modal illusion that can compensate for the absent sensory stimulation. For example, Li, Daugherty and Biocca (2002, 2003) suggested that interactive visual images can compensate for a lack of tactual input in product advertising in a virtual environment. Enhancing the sense of presence can facilitate the occurrence of cross-modal illusions, such as feeling physical resistance from virtual (visual) objects that are moved (Biocca, Kim and Choi, 2001). Furthermore, auditory cues can be used to give an impression of the tactile attributes of a product or surface (Kitagawa and Spence, 2006; Spence and Zampini, 2006). For instance, Lederman (1979) describes a television commercial for a shaving cream in which the edge of a credit card was drawn across both sides of a model’s face shown in close-up. One side of his face had been shaved using a proprietary shaving cream, the other with a competitor product. Although both sides of the model’s face looked identical after shaving in the

Multisensory product experience

139

advert, the difference in the closeness of the shave was effectively illustrated by the difference in the sound made by the credit card as it was drawn across the skin on the two sides of the model’s face.

4. ATTENTION SWITCHING BETWEEN THE SENSES Mechanisms of selective attention allow people to concentrate on particular events of interest in their environments, and to avoid sensory overload. Researchers have shown that people can covertly (i.e. in the absence of any change in posture) direct their selective attention to a particular sense (e.g. Spence, Nicholls and Driver, 2001). In fact, attending to one sensory modality can facilitate the perception of stimuli in that modality relative to when attention is diverted to another sensory modality (or else divided simultaneously between several senses). For example, the participants in a study reported by Spence and Driver (1997b) had to make speeded spatial discrimination responses to an unpredictable sequence of auditory and visual target stimuli presented to the left or right of a central fixation point. At the beginning of each trial, a symbolic visual cue indicated the likely modality of the upcoming target. This cue was valid on the majority (80%) of the trials and invalid on the remaining 20% of the trials. The participants responded significantly faster (and somewhat more accurately) when the target occurred in the expected modality than when the very same target stimulus was presented in the unexpected modality. Using similar experimental protocols, it has subsequently been shown that people can also direct their attention selectively toward the sense of touch (Spence et al., 2001), to the sense of smell (Spence et al., 2001), to the gustatory modality (Marks, 2002; Marks and Wheeler, 1998), and even to thermal pain (Miron, Duncan and Bushnell, 1989; Spence et al., 2002). Switching one’s attention from one sensory modality to another takes time; that is, it comes at a cost. Spence, Nicholls and Driver (2001) determined the reaction time costs associated with people shifting their attention between their eyes, their ears, and their skin. They found that the reaction time costs associated with shifting attention away from the sense of touch were larger than the costs associated with shifting attention away from either vision or audition. The costs associated with shifting attention toward touch from either the eyes or ears were larger than the costs associated with shifting attention to the eyes or ears. These results suggest that touch is, in a sense, ‘sticky’: Once our attention has been directed to what we are feeling, we find it harder to shift our attention toward what we are seeing or hearing instead. Spence et al. argued that this may be because when attending to touch people can either adopt an internal or an external perspective (see Chapter 2), whereas sounds and sights are always experienced as distal properties of the external world (Martin, 1995). It may then simply take people longer to shift between internal and external perspectives than to shift their attention between sensory modalities that both represent a more external perspective. Interestingly, differences in the reaction time costs associated with switching attention between the various senses have not only been reported in perceptual tasks, but also in the cognitive processing of words and concepts. For instance, Pecher, Zeelenberg and Barsalou (2003) used a property verification task in which their participants had to indicate whether a particular concept (e.g. blender) usually had a certain sensory property (e.g. loud) or not. Pecher et al. found that the average reaction times of participants increased when the sensory modality of the property switched between consecutive trials of the experiment: For example, responses to ‘blender-loud’ would be faster after ‘leavesrustling’ (both auditory), than after ‘toast-warm’ (tactual).

140

Product Experience

Perceived location is an extremely important factor governing multisensory integration and perception. This is because stimuli that appear to originate from the same spatial location are more likely to be attributed to a single multisensory source rather than to separate sources (Spence, 2007a). Consequently, it is far more likely that sensory stimuli will be integrated if they appear to come from the same spatial location than if they appear to come from different locations. The ventriloquist illusion (e.g. Bertelson, 1999), however, represents an important exception to this rule. Typically, the sounds that emanate from a ventriloquist’s mouth appear to originate from the mouth of his dummy. In order for the illusion to work well, the voice and lip-movements should co-vary in time; otherwise people have no particular reason to believe that the two streams of stimuli belong together. Similarly, in a movie theatre, the words uttered by an actor seem to originate from his mouth on the screen, whereas the sound is actually reproduced by the loudspeakers located elsewhere in the auditorium. In this case, given the temporal correspondence and the overall congruence between the auditory and visual streams, our brain assumes that the two sources of information ought to go together and we ventriloquize the actor’s speech sounds into spatial alignment with the seen lip movements. It should be noted that spatial and temporal correspondence are, in fact, just two of the many rules used by the human brain to group (bind, or unify) multisensory inputs. The interested reader is referred to Spence (2007a) for a recent discussion of the various other higher order variables (such as common number and semantic congruency) that have been shown to modulate the degree to which multisensory integration takes place. One of the key application areas for these research findings on attentional capture is in the design of warning and alarm signals. Many modern electronic devices, such as microwave ovens and telephones, make use of synthesized sounds to capture a consumer’s attention or to provide audible warning signals. In situations where multiple devices are used simultaneously, such as in kitchens, aircraft cockpits, or hospital operating rooms, several different alarms may go off at more or less the same time. Under such conditions, the source of a particular alarm can be hard to identify. For example, Momtahan, Hétu and Tansley (1993) reported that medical staff in operating rooms and intensive care units recognized only around 40% of the warning sounds correctly. An additional complication in many practical settings is that many interface operators have been known to disable or silence alarms if they consider them to be too irritating (e.g. Meredith, Edworthy and Rose, 1999; Sorkin, 1988). This problem is particularly pronounced under conditions where the false alarm rate is high. One possible strategy to overcome these limitations involves the inclusion of more sensory modalities in the design of alarm signals. In general, the chance of detecting an important warning signal and the speed with which it can be detected increases with the number of sensory modalities that redundantly communicate the same message. For instance, Selcon, Taylor and McKenna (1995) have shown that the verbal warning signal ‘left’ can elicit a left response more effectively if it is also presented on an interface operator’s left. Similarly, introducing a visual signal on the left can also help. Hence, gains in performance are most likely to occur if all of the relevant sensory signals originate from the same apparent spatial location, and if all the signals are congruent in terms of their perceived meaning (see Ho and Spence, 2005a; Spence and Driver, 1999). It is also important that the multiple sensory signals constituting a multisensory warning signal appear to be synchronized in time. One potential problem for interface designers here relates to the fact that auditory, visual, and tactile stimuli are all processed at different speeds (Spence and Squire, 2003). Even though light travels through air far more rapidly than sound, it takes much longer to process visual stimuli (chemically) in the eye than it does to mechanically transduce sound waves in the inner ear. It has been

Multisensory product experience

141

estimated that these two factors cancel each other out at a distance of approximately 10 m from an observer (known as the ‘horizon of simultaneity’, Pöppel, 1988). However, given that the majority of interfaces are located much closer to the typical interface operator (Spence and Driver, 1999), the visual component of an audiovisual warning signal should be presented slightly earlier in time than the auditory component in order to ensure that both stimuli reach the relevant parts of the brain at the same time (see also Chan and Chan, 2006). Ho and Spence (2005a) showed that warning signals presented from the direction in which a potentially dangerous driving event was about to occur (‘front’ or ‘back’), yielded faster and more accurate acceleration or braking responses than when they were presented from below the car driver’s seat. Auditory icons (the car horn sound) and verbal warning signals were found to be significantly more effective as warning signals than the pure tone cues typically favored by many designers. For application in real-world driving situations, Ho and Spence favored the implementation of auditory icons, because non-verbal signals are less likely to be susceptible to the influence of other concurrent linguistic elements in the driving environment, such as human speech from other passengers or from the car radio. In addition, their meaning is language-independent and understood intuitively (i.e. without the need for explicit training, Edworthy and Hellier, 2006). In subsequent studies, Ho and her colleagues (Ho, Reed and Spence, 2006; Ho, Tan and Spence, 2005) have shown that directional vibrotactile cues can also be used to effectively capture a driver’s attention. Vibrotactile warning signals have the additional advantage that they are private to the driver and that they remain effective, no matter how loud the radio may be turned up. However, depending on the body location where the vibrotactile warning signal is to be perceived, its utility may be limited by the thickness of the driver’s clothing. Although odors may also function as alarms, e.g. in indicating spoilage of foods or in indicating a fire burning, the deliberate use of odors in designed warning signals has, until now, mainly been confined to adding odorants to gas, so that a gas leakage will be noticed. Nonetheless, several other possible applications of using odors to attract attention or to alarm people have been suggested. For example, designers have considered the use of odorants (e.g. the smell of brewing coffee) as the wake-up signal in alarm clocks. However, by themselves olfactory stimuli are not capable of reliably awakening people who are asleep (Carskadon and Herz, 2004) and, consequently, an odor is unlikely to yield an effective wake-up signal. Others have attempted to isolate smells that are repulsive to as many people as possible to be used by the police in crowd control. However, no single odor has as yet been found to be suitably repulsive to all ethnic groups (Dilks, Dalton and Beauchamp, 1999). Elsewhere, it has been suggested that odors might be used to alert or relax people (e.g. Spence, 2002; Warm, Dember and Parasuraman, 1991). Such smells might find an application in office buildings, to energize employees during their post-lunch dip, or to help them to relax after a stressful meeting. Analogously, odorants might potentially be used in car interface design, either to alert drowsy drivers or to relax stressed drivers (see Baron and Kalsher, 1998; Ho and Spence, 2005b). However, more research is needed in order to find out which odorants can be used effectively to achieve the desired effects, and to specify the exact conditions under which they will be most effective. One other area where the spatial co-location of sensory signals has been shown to be important relates to the use of mobile phones in vehicles. Mobile phone conversations tend to be more demanding than conversations with other car passengers (see Kames, 1978), possibly because car passengers have less demanding/emotional conversations with the driver. In addition, car passengers tend to be aware of road conditions and

142

Product Experience

TABLE 5.3 Performance (% of words correctly recalled in the target speech stream) on a verbal shadowing task in a driving simulator, as a function of the location of the relevant auditory stream (adapted from Spence and Read, 2003)

Single task (Shadowing) Dual task (Shadowing while driving)

Front

Side

Difference

90.3 79.2

63.9 33.3

26.4 45.9

hence normally fall silent when the road conditions become more taxing. Using a mobile phone typically results in a four-fold increase in the risk of having an accident and, contrary to intuition, drivers are just as likely to have an accident using a hand-held device as when using a hands-free kit (Redelmeier and Tibshirani, 1997). This suggests that the key problem is related with attentional distraction (Horrey and Wickens, 2006; Spence and Read, 2003; Strayer and Johnston, 2001). Although certain influential ergonomics researchers (e.g. Wickens, 1980; 1992) have argued that people have separate pools of attentional resources for the processing of auditory and visual information, contemporary research has shown that there are in fact extensive crossmodal links in our spatial attention (Spence and Driver, 1997a, 2004). Spence and Read (2003) reported a driving simulator study showing that one of the important difficulties faced by drivers is in terms of trying to direct their visual attention spatially onto the road ahead, while at the same time trying to direct their auditory attention to the location from which the person’s voice is coming. Car drivers found it significantly easier to drive and to listen to a speech stream when both sources of information were presented from the same direction (i.e. from the front), rather than different directions (see Table 5.3). Spence and Read suggested that the introduction of ‘talking windscreens’ (where the sound appeared to come from directly in front of the driver) might make it easier to drive while simultaneously maintaining a mobile phone conversation.

5. CROSS-MODAL CORRESPONDENCES Certain product properties may be perceived by multiple sensory modalities. In particular, vision and touch show considerable overlap in terms of the information that they can provide: For example, people can both see and feel the shape of an object, its size, and the roughness of its surfaces (Lederman and Klatzky, 2004). Although the other senses are less well equipped to perceive these product properties, the shape and size of an object also affect the sound it will make when you tap it, and the surface properties affect the sound it makes when you run your fingers over its surface (see Spence and Zampini, 2006). Through their experiences with real-life objects, people will have some intuitive idea about which types of stimuli tend to co-occur and which do not. These patterns of covariation may form the basis of the correspondences that people perceive across different sensory modalities. If certain pairs of sensory stimuli are more likely to occur together (or seem to fit together better than others), presenting one stimulus of the pair may increase a consumer’s expectation that the other stimulus should also be present (see Garber, Hyatt and Starr, 2001; Schifferstein, 2001).

Multisensory product experience

143

Besides these associations that seem to have formed through prior experience, people might perceive some resemblances among colors, sounds, tastes, smells, and other sensory stimuli directly, because certain dimensions of sensory experience are shared across the different sensory modalities. For example, perception in all of the senses seems to share the dimensions of intensity (weak-strong), duration, and spatial location (Boring, 1942), which suggests that ‘the several senses display a fundamental unity in part because a class of suprasensory attributes pertains to sensations on all modalities’ (Marks, 1978, p. 51). Consistent with this line of thought, von Hornbostel (1931) collected evidence that brightness was a universal sensory dimension. The fact that people perceive correspondences across sensory modalities bears some resemblance to the phenomenon of synesthesia (Cytowic, 1989). However, we prefer to reserve the term ‘synesthesia’ for the phenomenon experienced by some special individuals, who experience a definite and reliable sensation in one sensory modality after being stimulated in a different sensory modality (e.g. Grossenbacher and Lovelace, 2001), and to use the term ‘cross-modal correspondence’ to refer to the connections that most people make between various sensory attributes in different modalities. One of the meaningful distinctions between these two groups is that individual reports of correspondences from synesthetic individuals have been shown to be more consistent over time than those from non-synesthetes (e.g. Baron-Cohen et al., 1993). However, given that these reports differ considerably between individual synesthetes, it is difficult to draw any generalizations from single reports that will be pertinent to design practice. Nevertheless, aggregate data from groups of synesthetes seem to show certain patterns of correspondence that are highly similar to those observed in non-synesthetes (Ward, Huckstep and Tsakanikos, 2006). We now present an example of how designers can make use of the associations that people perceive between different sensory domains. Even though people may find it hard to predict exactly how a product will smell if they have only had a chance to look at it, consistent crossmodal correspondences nevertheless do seem to exist between olfaction and vision. For example, Kemp and Gilbert (1997) have shown that colors that are matched to various odors generally show greater variation in hue across different odor than between various intensive levels of a single odor, suggesting that odor quality is primarily related to hue. Kemp and Gilbert also reported that perceived odor intensity, but not stimulus concentration, was related to visually-perceived brightness. People typically find it difficult to describe their olfactory experiences verbally (see Engen, 1982). Therefore, fragrance companies who want to inform potential consumers of the properties of their fragrances often try to communicate this information nonverbally through the color of the fragrance’s liquid itself, the bottle, the packaging, the advertisements, and other promotional materials. Packaging color has been shown to significantly affect people’s expectations with regard to perfume intensity (dark red is more intense than pastel green), sweetness (dark red is sweeter than pastel green), and freshness (pastel green is fresher than dark red) (Scharf and Volkmer, 2000). To determine which colors will match with a specific fragrance, Schifferstein and Tanudjaja (2004) asked 69 female participants to judge the degree to which each of 17 different colors fitted the smells of each of 14 fine fragrances. Figure 5.1 presents the color profiles for 5 of the fragrances that differ widely in their character. DKNY rated particularly high on saturated orange and yellow, whereas Wish rated high on all red colors, and Kouros fitted best with saturated blue. In addition, Kouros and Miss Dior gave rise to relatively high ratings for gray, black, and brown, whereas most other fragrances gave rise to very low ratings for these colors. Although individual degree-of-fit ratings for odor-color associations showed considerable variation over time in Schifferstein and Tanudjaja’s (2004) study, the aggregated

Pastel

9 8 7 6 5 4 3 2 1

Neutral DKNY Kouros Indecence Miss Dior Wish

i te gre y bla ck bro wn

9 8 7 6 5 4 3 2 1

wh

Saturated

pu rpl e red pin ora k n ye ge llo gre w en blu e

9 8 7 6 5 4 3 2 1

Product Experience

pu rpl e red ora ng ye e llo gre w en blu e

Degree-of-Fit

144

FIGURE 5.1 Color profiles for five fragrances (adapted from Schifferstein and Tanudjaja, 2004; reprinted with permission from Pion Limited, London).

data exhibited a relatively consistent pattern. In addition, it has recently been shown that the associations people report can be replicated using non-verbal psychophysical techniques, thus supporting the claim that they are a consistent, crossmodal phenomenon. In fact, Demattè, Sanabria, and Spence (2006) conducted the first crossmodal study of the Implicit Association Test (Greenwald, McGhee and Schwartz, 1998). The participants in their study had to make speeded discrimination responses to a random sequence of odors (strawberry vs. spearmint) and color patches (pink vs. turquoise) presented via computer. A color patch or an odor was presented as a target stimulus on each trial. The assignment of these targets onto the two response keys was manipulated in order to generate compatible (e.g. responding to the pink color and to the strawberry odor with the same response key) and incompatible (e.g. responding to the pink color and to the spearmint odor with the same response key) blocks of trials. Participants responded significantly more rapidly and more accurately to odor-color pairings having a stronger crossmodal association than to those having a weaker (or no) crossmodal association (see Figure 5.2). Designers can make use of these specific, individually matching colors in order to design packages that communicate a product’s olfactory properties to a consumer (Smets and Overbeeke, 1989). Given a limited set of matching colors, the designer can then pick those colors to create a package that will stand out against the competitors on the store shelves. In Schifferstein and Tanudjaja’s (2004) fragrance study, degree-of-fit ratings were, on average, highest for pastel colors, somewhat lower for saturated colors, and lowest for the neutral colors (see Table 5.4). Some colors showed many significant differences between odors, whereas others showed almost none. Saturated purple, red, and orange, and pastel purple, blue, and green obtained high degree-of-fit ratings and showed virtually no significant differences between odors. Consequently, these colors can be used in almost any fragrance package or advertisement. On the other hand, colors that show more differences between fragrances such as pastel red, pink and pastel yellow might better be reserved for those fragrances to which they best match. Although neutral colors (white, black, gray, brown) do not seem to match particularly well with fragrances in general, they do provide opportunities for some fragrances to create very distinctive packages that will stand out on the perfume shelves, and that still provide a reasonable fit (e.g. Miss Dior – brown; see Figure 5.1). The design strategy to adapt a package to match a fragrance’s olfactory characteristics assumes that consumers do not perceive the fragrance and the package separately, but may perceive them as a unitary whole. Indeed, a number of anecdotal reports suggest

145

Multisensory product experience

FIGURE 5.2 Percentage of correct responses to smells (olfactory) and colors (visual) when responses were coupled in a compatible (strawberry-pink and spearmint-turquoise) or incompatible (strawberry-turquoise and spearmint-pink) manner (adapted from Demattè et al., 2006; reprinted with permission from Oxford University Press).

TABLE 5.4 Mean ratings of degree-of-fit and the number of significant differences between odors for each color (from Schifferstein and Tanudjaja, 2004; reprinted with permission from Pion Limited, London)

Mean degree-of-fit

Number of different odor pairs

Saturated colors Purple Red Orange Yellow Green Blue

4.8 4.5 4.6 4.6 4.0 3.8

2 1 6 15 12 13

Pastel colors Purple Red Pink Orange Yellow Green Blue

5.1 5.1 4.9 5.3 4.9 4.6 4.8

9 22 17 12 19 7 2

Neutral colors White Gray Black Brown

3.3 2.8 2.2 2.6

0 7 19 10

that consumers experience the smell of spray-on deodorants somewhat differently if the coloring of their canisters is changed. Similarly, Thomas Hine (1995) cites the example of when 15% more yellow was added to the green on 7-Up cans, consumers found the taste to be a lot more limey/lemony. In fact, some consumers were so upset by the

146

Product Experience

FIGURE 5.3 MDS similarity spaces for graphic designs (visual – left) and sound characteristics of musical fragments (auditory – right), in both cases represented by CD covers (adapted from van Egmond, 2004; reprinted with permission from Taylor and Francis).

apparent change in taste as to say ‘You’re changing my Seven-Up! Don’t do a “New Coke” on me.’ Because the product formulation in these cases had not changed, these examples show that a product’s packaging can dramatically impact on a consumer’s perception of the product within. Another example of how designers can make use of the correspondences that people perceive across different sensory modalities was provided by van Egmond (2004). He asked participants to group existing CD covers (visual) and representative samples of music taken from those CDs (auditory) on the basis of either their visual or auditory similarity, respectively. Multi-Dimensional Scaling analyses of the similarity data revealed highly similar clusterings for the two types of stimuli (see Figure 5.3). This suggests that graphic designers are, to some extent, able to communicate the characteristics of the music through the designs they create for CD covers.

6. INTERACTIONS BETWEEN VARIOUS SENSORY DOMAINS Much of the research on multisensory integration has focused on the question of how perception in one sensory modality affects perception in a different sensory modality. In some cases, multisensory integration results in multisensory illusions, some of which may find an application in product design. Below, we give a number of examples. In the food realm, multiple interactions have been described between taste, smell, vision, and touch. For example, adding a colorant to an odorous solution generally increases the perceived intensity of its smell; darker solutions are often judged as smelling more intense than lighter solutions. The appropriateness of the color has little or no effect on the perceived intensity of the odor, but it can nevertheless facilitate odor identification and increases people’s liking for a particular odor (Zellner, Bartoli and Eckard, 1991; Zellner and Whitten, 1999). Research on odor-texture interactions has shown that increasing the hardness of a gel leads to a decrease in perceived intensity of its flavor,

Multisensory product experience

147

while flavor release from the gel is hardly affected (Bult, de Wijk and Hummel, 2007; Weel et al., 2002). Furthermore, essentially tasteless odorants may enhance the perceived intensity of a specific taste (Frank, van der Klaauw and Schifferstein, 1993). Examples of odor-texture interactions for non-food products are also available. In a classic study performed more than 70 years ago, Laird (1932) found that women evaluated the quality of silk stockings more positively when they were scented with a narcissus scent compared to their natural (slightly rancid) scent. When asked for the reasons behind their preference, the majority of the housewives pointed to differences in durability, sheen, or weave (i.e., to the tactile and/or visual properties of the stockings), rather than to differences in their olfactory properties. Demattè et al. (2006) recently followed up on Laird’s (1932) classic study. In their first experiment, fabric swatches were presented from a carousel under computer-control, and olfactory stimuli were delivered through a custom-built olfactometer. The participants were asked to rate the perceived softness of each fabric sample, while an odor of clean air was delivered directly to their nostrils. Demattè et al. observed that the fabric swatches were rated as significantly softer when presented with a lemon or lavender odor than when presented with an animal-like odor. While their experimental procedure allowed the authors to test a large number of possible combinations of fragrance and tactile feel within a short time frame, the ecological validity of the testing situation was obviously somewhat limited. Therefore, in a subsequent experiment, Demattè et al. went on to show that when participants evaluated fabric swatches that had actually been impregnated with one or the other fragrance, the fragrance affected the fabric’s softness in much the same way. In fact, if anything, the increase in fabric softness induced by the fragrance when emanating from the fabric itself was somewhat larger than when it was presented via the olfactometer (presumably due to the increased unity of the two signals in the former case). In another study looking at the nature of crossmodal interactions between hearing and touch, Zampini, Guest, and Spence (2003) asked their participants to evaluate the pleasantness and roughness of an electric toothbrush that they had to brush repeatedly across their front teeth. The toothbrush was judged to be rougher and less pleasant when the overall sound level increased and when the high-frequency sounds (above 2 kHz) made by the toothbrush were amplified. The same type of auditory manipulation has also been found to affect the perceived forcefulness of aerosol spray sounds (Spence and Zampini, 2007); the level of perceived carbonation in sparkling water drinks (Zampini and Spence, 2005); and even the perceived crispness of potato chips (Zampini and Spence, 2004). These examples show that people’s perception of the attributes of a product in a given sensory modality is frequently affected by the sensations that are simultaneously being perceived by another modality. It is crucial to note that participants/consumers are typically completely unaware of the occurrence (or nature) of these crossmodal effects. Thus, introspective consumer reports often fail to provide any useful insight into such crossmodal interactions. Therefore, experimental research on individual cases of multisensory perception is necessary in order to unravel the mechanisms behind each of these interactions. The belief among many researchers in the field of cognitive neuroscience is that many of the same rules will govern multisensory integration, no matter what combination of stimulus modalities (and sensory attributes) one looks at. However, it is becoming increasingly clear that some of the rules governing multisensory integration are more important for certain combinations of sensory modalities than for others. Thus, while common spatial origin and temporal alignment play a particularly important role in the integration of auditory, visual, and tactual stimuli (e.g. Congedo, Lécuyer and Gentaz,

148

Product Experience

2006; Kallinen and Ravaja, 2007; Spence, 2007a; Stein and Meredith, 1993), precise temporal synchronization would seem to be less important for the integration of olfactory and gustatory cues with information from the other sensory modalities. Because stimulus processing occurs more slowly for these modalities, the time frame for integration is much wider (Spence and Squire, 2003). Furthermore, taste–smell interactions occur in large part because people tend to confuse smell and taste sensations, probably because they cannot correctly localize the stimuli (see Rozin, 1982). Due to the fact that taste and smell seem to originate from the same location when eating, people may integrate some of the odorant intensity with the intensity of the taste (Frank et al., 1993). On the other hand, the robust crossmodal interactions that have been documented between vision and both taste and smell seem to rely mainly on learned associations (see Demattè, Sanabria and Spence, 2006; Maga, 1974). In such cases, the cognitive associations evoked by sensing something may generate expectations that affect people’s subsequent perception (Cardello and Sawyer, 1992; Schifferstein, 2001).

7. SENSORY (IN)CONGRUITY Knowledge of the roles of the different senses is important for designers who want to communicate a coherent message to the potential customers for their products. In particular, designers should consider which messages an existing product currently communicates through each of the senses (Lindstrom, 2005). The contents of certain of these messages may conflict, and thus the designer may want to check the possibility of changing certain product features in order to improve the coherence of the consumer’s overall multisensory product experience. The congruence of sensory messages in product design is also desirable from an ergonomic perspective, where coherence helps to clarify what a product is about and what it can do. In addition, perceived unity in visual stimuli has been shown to correlate with ratings of both aesthetic appeal and liking (Bell, Holbrook and Solomon, 1991; Veryzer and Hutchinson, 1998). Therefore, multisensory coherence is likely to be positively related to consumer preference. In striving for multisensory coherence, it becomes obvious that a product’s package should be regarded as an integral part of the overall product design. Unfortunately, many packages seem to be designed primarily to protect their content and, thereby, create an almost invincible barrier to experience the product inside. One possible strategy to counteract this is to introduce holes in the package to let potential consumers experience how the product inside feels, smells and sounds. Another strategy is to convey sensory product information through the sensory characteristics of the packaging. We have already seen how visual components (e.g. color) can influence the perception of fragrances and foods. Analogously, the tactual, olfactory, and auditory aspects of package design are also potentially important, even if currently underutilized by many designers. For example, a brand of toilet tissue that claims to be velvety-smooth recently started treating the plastic packaging in which rolls of their toilet paper were sold, so that the plastic itself actually felt velvety-smooth (see Spence, 2007b). It is also possible to impregnate packaging materials with fragrances that are released upon touching the package. Furthermore, through the selection of packaging materials the sounds perceived when opening the package can be modified to enhance, for example, freshness and crispiness of foods (crispy paper or metal foil, see Brown, 1958) or softness of personal care products (thick plastic with rounded shapes producing only soft sounds).

Multisensory product experience

149

FIGURE 5.4 ‘Mary Biscuit’ biscuit box, designed by Stefano Giovannoni (1995) (courtesy Alessi spa).

A nice example of a product in which all the senses are stimulated to bring about a unitary multisensory experience is Alessi’s ‘Mary Biscuit’ designed by Stefano Giovannoni (see Figure 5.4). This biscuit-box seems to enhance the coziness of social visits. The majority of biscuit boxes are made from metal: They feel cold and may contain sharp edges; the sounds they make when they are opened are also sharp, and when the cover accidentally falls on the floor it can make a loud and unpleasant noise. Hence, through the senses of touch and audition, the majority of biscuit tins communicate hostility and coldness rather than friendship and warmth. The ‘Mary Biscuit’ container, however, is different in that it is made of plastic, and contains only rounded edges, and its shape resembles that of a pillow. The box feels soft and warm to touch, it makes only soft noises when you open it or put it down, and the box itself seems to invite the user to cuddle it. In contrast to the metal box, which usually does not have a distinctive smell of its own, the cover of the ‘Mary Biscuit’ is impregnated with a vanilla-like odor that becomes apparent when the container is opened. Because many cookies contain the flavor of vanilla anyway, the additional smell may enhance the experience of tasting a cookie. In addition, the smell might also evoke nostalgic memories of family visits to one’s grandmother and may, thereby, enhance the feeling of sharing an experience with intimate friends or relatives. Upon closer inspection, the ‘Mary Biscuit’ container can therefore be seen to use the majority of a consumer’s senses in order to communicate similar, but also seemingly redundant, information to produce a pleasing multisensory impression. In the years to come, the increasing awareness of the roles of the different sensory modalities and the interactions between them will shift the focus for many designers away from the physical product to the specific experience that a product evokes. For example, designers can develop a scenario of the sensory events that occur when a person encounters a product (MacDonald, 2002). They may then use this scenario as the starting point for the design of a new product. Such an approach is likely to result in more interesting, involving, and ultimately more engaging products and product experiences, because such products exploit the full potential of people’s sensory connections with the surrounding world (Howes, 2005). However, designers may not always want to design products for which all of the senses communicate the same message. On the contrary, in some cases designers may want to evoke surprise by introducing sensory discrepancies or uncertainties. Examples of such incongruities include a chair made from rope, which makes a person wonder

150

Product Experience

whether it is stable enough to sit on, or a cup that seems to be made from metal but is actually made from plastic, and is thus much lighter than one would expect. The challenge in these cases seems to be to combine familiarity and originality within the same design (see Hekkert, Snelders and van Wieringen, 2003). Even designers who want to surprise people will generally make sure that the majority of the product-related information communicates the same message, while only one particular aspect is responsible for the element of surprise. An analysis of designs presented in the International Design Yearbooks (1999– 2004) suggests that up to 6% of these designs incorporate some form of visual-tactual incongruity (Ludden, Schifferstein and Hekkert, 2007a), implying that the product feels different from what would be expected on the basis of visual information alone. The surprising products found mainly consisted of furniture and other interior products, such as lamps, vases, tiles, and dinner-sets. This might imply that surprise as a design strategy is typically limited to products in well-known categories that consumers can easily identify. On the basis of an analysis of 101 products exhibiting visual-tactual incongruity, Ludden, Schifferstein, and Hekkert (2007a,b) made an important distinction between two different types of surprise. The ‘Visible Novelty’ (VN) type of surprise consists of products that seem unfamiliar to the perceiver. Consequently, only uncertain expectations can be formed on the basis of associations with the product category, overall shape, or material use. Although the consumer’s surprise reaction is likely to be limited, because they are unsure of the product’s exact properties and purpose, the product may be interesting and intriguing because of the ambiguity it creates in their mind. An example is provided by the bench ‘Shrunken furniture’ designed by Bertjan Pot (Figure 5.5 (left)). This bench is made out of polystyrene, which is covered in knitted cloth and then vacuumed and hardened with wax. The combination of materials with the new shape results in a bench that looks like it is made out of a familiar soft material, like foam rubber. In reality, the bench feels hard. In addition,

FIGURE 5.5 Products evoking the Visible Novelty (left) and the Hidden Novelty (right) surprise types.

Multisensory product experience

151

because this material is lighter than the materials that are commonly used in the construction of benches (such as wood), this bench is much lighter than one would expect. On the other hand, products belonging to the ‘Hidden Novelty’ (HN) surprise type seem very familiar to the perceiver, and thus lead to stronger reactions of surprise, because the tactual properties are very different from those that are expected. For example, the vase in Figure 5.5 (right) looks like a familiar, traditional vase made from glass or crystal. In actual fact, it is made from plastic and, consequently, a consumer will experience it as being surprisingly light when he/she picks it up. Ludden et al. (2007b) found that people react differently to products evoking these two types of surprise. Surprise ratings were higher and facial expressions typically associated with surprise occurred more often for HN than for VN products. People tended to use more vocal expressions when interacting with HN products, and more exploratory behavior while interacting with VN products. It is possible that people enjoyed exploring VN products more than HN products, or else that they were more curious about the exact material properties of VN products. Alternatively, however, they may have needed more time in order to interact with the VN products, because they wanted to try and figure out what exactly it was that had triggered their surprise reaction. On the other hand, it seems that the surprise experienced by consumers upon touching the HN product was understood immediately, and that further exploration or cognitive effort was unnecessary. This may partly explain why people typically experience VN products as more interesting than HN products.

8. SENSORY DOMINANCE When a person interacts with a product, the inputs from the various senses should be integrated in order to give rise to a unified multisensory product experience. How does this multisensory integration take place? Laboratory studies of multisensory integration have typically tried to determine the relative contribution of each input variable to (an aspect of) the overall product experience. But what are the factors that affect this integration process? The results of many empirical studies now show that the relative importance of the various senses does not only depend on the particular type of product being investigated, but also on the specific task that the user has to perform with the product (e.g. Schifferstein, 2006), as well as on the personal characteristics of the user (e.g. Peck and Childers, 2003; Wrzesniewski, McCauley and Rozin, 1999). One general rule-of-thumb is that the interpretation of sensory information that is more ambiguous usually receives less weight in the formation of multisensory product judgments (Ernst and Bülthoff, 2004; Hoch and Ha, 1986; van Beers, Sittig and van der Gon, 1999). Therefore, on the basis of an evaluation of the roles of the individual sensory modalities (see Section 2), one might conclude that vision and touch are likely to dominate product perception and experience in many real-life situations (Schifferstein and Cleiren, 2005). Moreover, vision is likely to have an even larger impact on product experiences than touch, because visual information is processed more quickly and is typically available sooner (i.e. we see an object on the shelf before we decide to pick it up; we inspect the food on our plates before we decide what to eat next). Furthermore, while vision can be used to explore large objects with a minimum of delay, this would be impossible using only touch. As a consequence, vision has been found to direct exploratory behavior involving the other senses (e.g. Heller, 1982; Klatzky, Lederman and Matula, 1993). However, the dominant role of vision and, to a lesser extent, touch is likely to be mainly limited to the functional user-product interaction and to the conscious

152

Product Experience

TABLE 5.5 Mean importance ratings of the five modalities on a five-point scale during usage of several products (adapted from Schifferstein, 2006)

Car Vacuum cleaner TV Couch Vase Hammer Paper Wristwatch Shoes Underwear Deodorant Flowers Cookies

Vision

Touch

Audition

Smell

Taste

4.3 3.5 4.4 4.6 4.4 2.7 3.6 4.6 4.7 4.2 2.5 4.8 4.1

3.8 3.3 2.3 4.4 2.9 3.8 3.5 4.1 4.5 4.5 3.8 3.1 3.3

4.3 4.2 4.5 2.6 1.7 2.2 2.1 3.5 3.0 1.9 1.7 1.4 1.9

3.5 2.7 1.9 2.9 2.0 1.3 2.8 2.0 3.1 3.4 4.6 4.4 4.3

1.1 1.1 1.1 1.1 1.1 1.1 1.2 1.3 1.1 1.4 1.2 1.6 4.9

experience of that interaction. The other sensory modalities may nevertheless still play important roles in terms of modulating the emotional experiences that are evoked by products (Schifferstein and Desmet, 2007). In one recent study, Schifferstein (2006) compared the conscious evaluations of the relative importance of the different senses during actual product usage. Eighty people reported on a five-point scale how important they found the information provided by each of the five main senses during the usage of 45 different products (see Table 5.5). The results showed that the usage value of a vase, for instance, was determined primarily by its visual appearance. For a TV, the sound was judged as being just as important as its appearance. For simple tools and utensils, such as a hammer, the tactile characteristics were of primary importance, followed by their visual appearance. For products associated with cleaning and personal care, olfactory cues generally played a key role in combination with their tactual and, to a lesser extent, visual properties. For food products, taste was judged as being the most important sense, generally followed by smell and then vision. Going through the list of products, examples were found for which only a single sensory modality was important during usage (vase), but also examples for which four or more sensory modalities were found important (car, vacuum cleaner, shoes, cookies). In controlled experimental studies, the relative importance of the different senses has primarily been investigated by providing participants with conflicting information on (an aspect of) a single object through two or more sensory modalities at the same time. Although incongruent information is presented in these studies, the research focus has not been on the participant’s surprise reaction to that incongruity, but rather on if (and how) the available discrepant sensory information is integrated (weighted). By comparing the overall responses to the objects, the impact of the various modalities can be quantified. Note that the degree of conflict in these studies has often been kept below the threshold where such discrepancy becomes noticeable (e.g. Ernst and Banks, 2002; Rock and Victor, 1964). These controlled experimental studies have shown that the importance of a sensory modality depends both on the stimuli being evaluated, as well as on the specific task instructions given to participants (Guest and Spence, 2003; Lederman, Thorne and Jones,

Multisensory product experience

153

1986; Welch and Warren, 1980). Several explanations for these findings have been put forward. For example, according to the ‘modality appropriateness’ hypothesis (Freides, 1974; Welch and Warren, 1980), people weigh sensory inputs according to their relative unimodal performance capabilities for a specific task. According to this hypothesis, the more suitable a sensory modality to perceive a product attribute, the larger its impact on the perception of that particular product aspect will be. According to another hypothesis (Posner, Nissen and Klein, 1976), however, the modality that receives the observers’ directed attention will dominate perception. Note that these two accounts yield similar predictions in many cases. Strictly speaking, perception depends mainly on the nature of the information and the physical laws that govern the situation, and not so much on the specific sensory modality the information happens to be presented in. For instance, by means of a camera that transforms visual information into electronic pulses that can be felt on the tongue, blind people can (in some sense) learn to ‘see’, and people who have lost sensitivity in the vestibular system can regain their sense of balance (Bach-y-Rita, 2004; O’Regan and Noë, 2001). In addition, devices can be developed to convey important information through a different sensory modality under extreme conditions where the original sensory system is unable to function properly. For example, van Erp and van Veen (2006) recently reported on the development of a vest containing 56 vibrators that can help an astronaut to obtain a sense of ‘up’ and ‘down’ in a microgravitational environment. Nonetheless, under natural conditions the type of information perceived is always linked to a particular sensory modality. Consequently, the perceived importance of a particular sensory modality is likely to be determined by whether it can transduce important information or not. Accordingly, Schifferstein (2006) concluded that ‘the often referred to dominance of vision is likely to reflect people’s overall tendency to find visual input relatively important when its role is evaluated for the ensemble of activities performed. As a consequence, the role of the senses is likely to depend on the specific products used, the frequency with which they are used, and the importance attached to the activities performed.’ (Schifferstein, 2006, p. 60). Schifferstein suggested that the importance of vision in Western societies may have increased over time due to the range of products that have been created. For example, many products that facilitate human communication (such as television, newspapers, the Internet) require major visual input. When such products become part of people’s daily lives, the importance of vision is, if anything, likely to increase (see also Spence, 2002). Interestingly, the importance of a sensory modality does not only depend on the nature of a particular product, but also on its stage of ownership. People’s subjective reports suggest that the role of vision is primarily dominant during the acquisition process. After purchase, the role of the other modalities often increases significantly, at the expense of vision. However, during ownership, the importance of vision increases once again, because the owner’s appreciation of the product’s visual appearance decreases due to fashion changes and to the accumulation of signs of usage, such as scratches (Fenko, Schifferstein and Hekkert, 2007). Design is an integrative discipline that requires the ability to materialize predefined intentions into new design solutions. Designers can manipulate many different variables, such as product shape, weight, smell, and finish, to fulfill these intentions. In the end it is the interplay between all of these design variables that will determine the experiential value of the design, and in most cases it is impossible to determine the contribution of individual variables to the end result. Nevertheless, knowledge regarding the relative importance of the different senses can be valuable because it helps to prevent a fixation

154

Product Experience

on the visual modality. Understanding the relative importance of the different senses can be useful for balancing the time and resources invested in new product development projects (Lindstrom, 2005). A broad focus on all the senses can stimulate designers to explore the value of hitherto neglected sensory modalities for creating new market opportunities. For example, for a mobile phone the smell and feel of the cover are generally not regarded as important product attributes. Nevertheless, a mobile phone can be regarded as a very personal, almost intimate, object that people carry around all day long and that they keep close to their face during calls. In this case, introducing phone covers made from different materials, varying in surface texture and smell characteristics, might make phones feel and smell more personal, thereby clearly discriminating between ‘own’ and ‘foreign’, and increasing the intimacy of the calling experience. As a consequence, people may become more attached to their phone over time (see Chapter 17).

9. CONCLUSIONS AND DIRECTIONS FOR FUTURE RESEARCH The rapid growth of research in recent years on the topic of multisensory perception has produced many fascinating insights into how our brain integrates the information received through the different senses. In the previous sections we have attempted to provide an overview of the state-of-the-art knowledge regarding the ways in which people use their senses in their interactions with products. Through the accelerated application of this growing source of knowledge, a future generation of consumer products can be developed that offer experiences that involve more sensory modalities, and that make the stimulation of the senses more unified and coherent or, possibly, more interesting and surprising, but in all cases, more pleasing. However, a number of additional topics will need to be investigated more systematically in the years to come. They have not been treated here in full because of the dearth of available literature on these topics at present. One particularly important topic for future research relates to individual differences. Given that there are robust and striking individual differences in sensory perception, manufacturers may increasingly start to develop products that have been sensorially tailored for a particular section of the population. Differences between individuals in sensory capabilities may, for example, be due to sex (Velle, 1987) or age (Corso, 1971; Doty et al., 1984; Robinson, 1988). It is important to note that the elderly do not form a homogeneous population: Their capabilities, health, functional status, and nutritional needs vary widely. What’s more, sensory capabilities do not necessarily decline to the same degree for each individual (e.g. see Stevens, Cain and Burke, 1988) and, therefore, the heterogeneity of sensory capabilities in the population is likely to increase. It is therefore possible that demographic changes will not only lead to products that have been specifically designed to appeal to the advanced-age segment of the population (e.g. see Pirkl, 1994; Schiffman, 2000), but also to products that are designed to fit the individual needs of the end user (e.g. Mugge, Schifferstein and Schoormans, 2006). Apart from these variations in sensory capabilities, individuals may also differ in the ways in which they prefer to respond to sensory stimulation. Schifferstein and Smeets (2006) named the latter construct a person’s perceptual style. Differences in perceptual style may, for example, influence whether consumers tend to rely on visual or tactual input when they buy a product (e.g. Peck and Childers, 2003). In an educational context, it has been argued that differences in perceptual style partly determine whether a teacher should present visual images, verbal messages, or physical objects that can be explored tactually, in order to communicate information efficiently to a particular student

Multisensory product experience

155

(e.g. Dunn and Dunn, 1992). Interestingly, the sensory expertise of the product user may also be identified as a determining factor of perceptual style (e.g. Binns, 1937). Another area that has, as yet, received relatively little attention from researchers relates to possible cultural differences in how people use their senses in their interaction with products, and the effect this has on their evaluation of these products. A culture specifies the rituals that are connected to product usage (Howes, 1991; MacDonald, 2002). Through exposure to nature, surroundings, and products in everyday life, people develop a sensory frame of reference. On the one hand, this frame of reference contains a person’s sensory knowledge that has accumulated over time, which may affect what a person can perceive through a particular sensory modality (e.g. Ayabe-Kanamura et al., 1998). On the other hand, it may determine which sensory impressions go together and which do not (Scanlon, 1985; Walker, 1987). For example, learned odor-color associations are likely to differ cross-culturally: Lemons are yellow in Europe, but green in parts of South America; cinnamon is associated with red in the USA (sweets), but with brown in the UK (spice); meanwhile the odor of cucumber is associated with green in the UK, but is often associated with red in Spain due to its ubiquitous presence in gazpacho soup (Spence, 2007b). These differences clearly need to be taken into account when developing products for different countries or for multi-cultural societies. It is worth highlighting the fact that technology can also change the way in which people interact with everyday products. For example, when Unilever introduced their newly-designed canister for the Axe/Lynx deodorant brand (where the spray mechanism was built into the cap), it was noted that certain consumers (particularly young males) often used their thumb to operate the aerosols, while consumers have traditionally used their index finger instead. One of Unilever’s packaging directors attributed this change to the enhanced usage of the thumbs when text messaging or playing video games (Carter, 2006). Product designers need to take such changes in haptic behaviour into account when engaging in the design process. Another topic that is of great interest to designers and that is likely to increase in popularity is the use of sensory icons. Traditionally, companies have invested a lot of money in trying to define and communicate a desirable identity for their brands. Next to formulating a verbal message, they have typically developed visual stimuli in order to support this process, such as a brand logo, and a selection of colors and fonts to be used in packaging, advertising, and other types of graphic design (see Hine, 1995). More recently, however, sensory impressions created in the other sensory modalities have increasingly been used to create and modify brand identity (see Lindstrom, 2005). For example, in the automotive industry, the contribution of auditory cues to product perception has been known about for many years (e.g. Packard, 1957, p. 111). On occasion the unique, or characteristic, sounds made by a specific vehicle have been linked to a specific manufacturer or brand name. For example, Harley-Davidson went to great lengths to try and trademark the distinctive sound of their motorcycle engines, which they considered to be an essential part of their product experience (Lyon, 2003; Sapherstein, 1998). In fact, car designers have invested a great deal of research into the sound of everything from the engine to the sound made when the door is closed (see Spence and Zampini, 2006). Distinctive product sounds, sometimes known as ‘signature sounds’, can denote ‘character’ and can become strongly associated with a particular product and its functionality. For food products, think of the snap of the ‘Kitkat’, the distinctive crack of the chocolate breaking as one bites into a ‘Magnum’ ice cream, or the sound of the release of carbonation as a bottle of ‘Schweppes’ is opened. It also seems increasingly likely that companies will develop their own unique ‘signature scents’ that will help to create a distinctive, unifying, and memorable image that

156

Product Experience

will help to distinguish their products from those of their competitors (see Miller, 1993; Spangenberg, Crowley and Henderson, 1996). In fact, according to one report in the Wall Street Journal (Ellison and White, 2000), British Airways has already trialled the use of a signature scent called ‘Meadow Grass’ in their executive airport lounges. After all, what could be nicer for the world-weary traveler than to be greeted by a familiar scent when they walk into the lounge, signaling that they are almost home? It is probably only a matter of time before companies start copyrighting their signature scents in order to protect their own unique olfactory images (Classen, Howes and Synnott, 1994). Recently in the Netherlands, Lancôme successfully defended the copyright on the smell of their Trésor fragrance against a copyist (Frijters and van Houte, 2004). Furthermore, several products are already on the market that have registered a specific scent as (part of) their trademark. UK consumers can now buy darts arrows that have been impregnated with the smell of beer, while consumers in the Netherlands can buy tennis balls impregnated with the smell of green grass as part of a registered brand (Goosen, 1998). By analogy to the previous examples, it should also be possible to register a certain, characteristic feel as part of a certain brand. This may then open up possibilities for the creation of multisensory brands, consisting of the definition of multiple sensory impressions underlying brand identities, with possibilities to register characteristic sensations or combinations of sensations as part of the brand (Howes, 2005; Lindstrom, 2005).

ACKNOWLEDGMENTS This research was supported by MAGW VIDI grant 452-02-028 of the Netherlands Organization for Scientific Research (N.W.O.) awarded to H. N. J. Schifferstein. The authors are indebted to all of their co-workers for their contributions to the research presented.

REFERENCES Ayabe-Kanamura, S., Schicker, I., Laska, M., et al. (1998). Differences in perception of everyday odors: a Japanese–German cross-cultural study. Chemical Senses, 23, 31–38. Bach-y-Rita, P. (2004). Tactile sensory substitution studies. Annals of the New York Academy of Sciences, 1013, 83–91. Bahrick, L. E. and Lickliter, R. (2000). Intersensory redundancy guides attentional selectivity and perceptual learning in infancy. Developmental Psychology, 36, 190–201. Baron, R. A. and Kalsher, M. J. (1998). Effects of a pleasant ambient fragrance on simulated driving performance: the sweet smell of ... safety? Environment and Behavior, 30, 535–552. Baron-Cohen, S., Harrison, J., Goldstein, L. H. and Wyke, M. (1993). Coloured speech perception: is synesthesia what happens when modularity breaks down? Perception, 22, 419–426. Bell, S. S., Holbrook, M. B. and Solomon, M. R. (1991). Combining esthetic and social value to explain preferences for product styles with the incorporation of personality and ensemble effects. Journal of Social Behavior and Personality, 6, 243–274. Bertelson, P. (1999). Ventriloquism: a case of cross-modal perceptual grouping. In: G. Aschersleben, T. Bachmann and J. Müsseler (Eds.) Cognitive contributions to the perception of spatial and temporal events, pp. 347–362. Amsterdam: Elsevier. Betts, G. H. (1909). The distribution and functions of mental imagery. New York: Teachers College, Columbia University. Binns, H. (1937). Visual and tactual ‘judgement’ as illustrated in a practical experiment. British Journal of Psychology, 27, 404–410. Biocca, F., Kim, J. and Choi, Y. (2001). Visual touch in virtual environments: an exploratory study of presence, multimodal interfaces, and cross-modal sensory illusions. Presence: Teleoperators and Virtual Environments, 10, 247–265.

Multisensory product experience

157

Blomqvist, E. H., Bramerson, A., Stjarne, P. and Nordin, S. (2004). Consequences of olfactory loss and adopted coping strategies. Rhinology, 42, 189–194. Bolanowski, S. J., Verrillo, R. T. and McGlone, F. (1999). Passive, active and intra-active (self) touch. Somatosensory and Motor Research, 16, 304–311. Boring, E. G. (1942). Sensation and perception in the history of experimental psychology. New York: Appleton-Century-Crofts. Brown, R. L. (1958). Wrapper influence on the perception of freshness in bread. Journal of Applied Psychology, 42, 257–260. Bult, J. H. F., de Wijk, R. A. and Hummel, T. (2007). Investigations on multimodal sensory integration: texture, taste, and ortho- and retronasal olfactory stimuli in concert. Neuroscience Letters, 411, 6–10. Burns, L. D., Brown, D. M., Cameron, B., Chandler, J. and Kaiser, M. J. (1995). Sensory interaction and descriptions of fabric hand. Perceptual and Motor Skills, 81, 120–122. Cardello, A. V. and Sawyer, F. M. (1992). Effects of disconfirmed consumer expectations on food acceptability. Journal of Sensory Studies, 7, 253–277. Carskadon, M. A. and Herz, R. S. (2004). Minimal olfactory perception during sleep: why odor alarms will not work for humans. Sleep, 27, 402–405. Carter, S. (2006, 5–6 December). Brand owner case study: Axe ‘Millenium’ body spray. Paper presented at the 3rd Packaging Impact and Innovation conference, London, UK. Chan, A. H. S. and Chan, K. W. L. (2006). Synchronous and asynchronous presentations of auditory and visual signals: implications for control console design. Applied Ergonomics, 37, 131–140. Classen, C., Howes, D. and Synnott, A. (1994). Aroma: the cultural history of smell. London: Routledge. Congedo, M., Lécuyer, A. and Gentaz, E. (2006). The influence of spatial delocation on perceptual integration of vision and touch. Presence: Teleoperators and Virtual Environments, 15, 353–357. Corso, J. F. (1971). Sensory processes and age effects in normal adults. Journal of Gerontology, 26, 90–105. Cytowic, R. E. (1989). Synesthesia: a union of the senses. New York: Springer Verlag. Demattè, M. L., Sanabria, D. and Spence, C. (2006). Cross-modal associations between odors and colors. Chemical Senses, 31, 531–538. Demattè, M. L., Sanabria, D., Sugarman, R. and Spence, C. (2006). Crossmodal interactions between olfaction and touch. Chemical Senses, 31, 291–300. Desmet, P. M. A. (2002). Designing emotions. Unpublished PhD dissertation, Delft University of Technology, Delft. Dilks, D. D., Dalton, P. and Beauchamp, G. K. (1999). Cross-cultural variation in responses to malodors. Chemical Senses, 24, 599. Dinh, H. Q., Walker, N., Song, C., Kobayashi, A. and Hodges, L. F. (1999, 13–17 March). Evaluating the importance of multi-sensory input on memory and the sense of presence in virtual environments. Paper presented at the IEEE Virtual Reality Conference, Houston, TX. Doty, R. L., Shaman, P., Applebaum, S. L., Giberson, R., Siksorski, L. and Rosenberg, L. (1984). Smell identification ability: changes with age. Science, 226, 1441–1443. Dubois, D. (2000). Categories as acts of meaning: the case of categories in olfaction and audition. Cognitive Science Quarterly, 1, 35–68. Dunn, R. and Dunn, K. (1992). Teaching secondary students through their individual learning styles. Needham Heights, MA: Allyn and Bacon. Edworthy, J. and Hellier, E. (2006). Complex nonverbal auditory signals and speech warnings. In: M. S. Wogalter (Ed.) Handbook of warnings, pp. 199–220. Mahwah, NJ: Erlbaum. Ellison, S. and White, E. (2000, November 27). ‘Sensory’ marketers say the way to reach shoppers is the nose. The Wall Street Journal. Engen, T. (1982). The perception of odors. New York: Academic Press. Ernst, M. O. and Banks, M. S. (2002). Humans integrate visual and haptic information in a statistically optimal fashion. Nature, 415, 429–433. Ernst, M. O. and Bülthoff, H. H. (2004). Merging the senses into a robust percept. Trends in Cognitive Sciences, 8, 162–169. Fenko, A., Schifferstein, H. N. J. and Hekkert, P. (2007). Shifts in sensory dominance between various stages of user–product interactions (submitted). Field, T. (2001). Touch. Cambridge, MA: MIT Press. Frank, R. A., van der Klaauw, N. J. and Schifferstein, H. N. J. (1993). Both perceptual and conceptual factors influence taste–odor and taste–taste interactions. Perception and Psychophysics, 54, 343–354. Freides, D. (1974). Human information processing and sensory modality: cross-modal functions, information complexity, memory, and deficit. Psychological Bulletin, 81, 284–310. Frijters, J. E. R. and van Houte, C. P. M. (2004). De geur van parfum zit niet in de fles (The smell of perfume does not reside in the bottle). Nederlands Juristen Blad, 79, 1987–1989.

158

Product Experience

Garber, L. L., Hyatt, E. M. and Starr, R. G. (2001). Placing food color experimentation into a valid consumer context. Journal of Food Products Marketing, 7(3), 3–24. Goosen, W. F. P. (1998, 5 March 1998). Geuren als merk (Smells as brands). Paper presented at the Geurenmarketing congress, Amsterdam. Greenwald, A. G., McGhee, D. E. and Schwartz, J. L. K. (1998). Measuring individual differences in implicit cognition: the implicit association test. Journal of Personality and Social Psychology, 74, 1464–1480. Grossenbacher, P. G. and Lovelace, C. T. (2001). Mechanisms of synesthesia: cognitive and physiological constraints. Trends in Cognitive Sciences, 5, 36–41. Guest, S. and Spence, C. (2003). Tactile dominance in speeded discrimination of textures. Experimental Brain Research, 150, 201–207. Hekkert, P., Snelders, D. and van Wieringen, P. C. W. (2003). ‘Most advanced, yet acceptable’: typicality and novelty as joint predictors of aesthetic preference in industrial design. British Journal of Psychology, 94, 111–124. Heller, M. A. (1982). Visual and tactual texture perception: intersensory cooperation. Perception and Psychophysics, 31, 339–344. Herz, R. S. (1998). An examination of objective and subjective measures of experience associated to odors, music, and paintings. Empirical Studies of the Arts, 16, 137–152. Herz, R. S. and Schooler, J. W. (2002). A naturalistic study of autobiographical memories evoked by olfactory and visual cues: testing the Proustian hypothesis. American Journal of Psychology, 115, 21–32. Hine, T. (1995). The total package: The secret history and hidden meanings of boxes, bottles, cans, and other persuasive containers. New York: Little Brown. Hinton, P. B. and Henley, T. B. (1993). Cognitive and affective components of stimuli presented in three modes. Bulletin of the Psychonomic Society, 31, 595–598. Ho, C., Reed, N. and Spence, C. (2006). Assessing the effectiveness of ‘intuitive’ vibrotactile warning signals in preventing front-to-rear-end collisions in a driving simulator. Accident Analysis and Prevention, 38, 988–996. Ho, C. and Spence, C. (2005a). Assessing the effectiveness of various auditory cues in capturing a driver’s visual attention. Journal of Experimental Psychology: Applied, 11, 157–174. Ho, C. and Spence, C. (2005b). Olfactory facilitation of dual-task performance. Neuroscience Letters, 389, 35–40. Ho, C., Tan, H. Z. and Spence, C. (2005). Using spatial vibrotactile cues to direct visual attention in driving scenes. Transportantion Research Part F, 8, 397–412. Hoch, S. J. and Ha, Y. W. (1986). Consumer learning: advertising and the ambiguity of product experience. Journal of Consumer Research, 13, 221–233. Hoffman, H. G., Hollander, A., Schroder, K., Rousseau, S. and Furness, T. I. (1998). Physically touching and tasting virtual objects enhances the realism of virtual experiences. Journal of Virtual Reality, 3, 226–234. Horrey, W. J. and Wickens, C. D. (2006). Examining the impact of cell phone conversations on driving using meta-analytic techniques. Human Factors, 48, 196–205. Howes, D. (1991). The varieties of sensory experience: a sourcebook in the anthropology of the senses. Toronto: University of Toronto Press. Howes, D. (2005). Hyperesthesia, or, the sensual logic of late capitalism. In: D. Howes (Ed.) Empire of the senses: the sensual culture reader, pp. 281–303. Oxford: Berg. Kallinen, K. and Ravaja, N. (2007). Comparing speakers versus headphones in listening to news from a computer – individual differences and psychophysiological responses. Computers in Human Behavior, 23, 303–317. Kames, A. J. (1978). A study of the effects of mobile telephone use and control unit design on driving performance. IEEE Transactions on Vehicular Technology, VT-27, 282–287. Kemp, S. E. and Gilbert, A. N. (1997). Odor intensity and color lightness are correlated dimensions. American Journal of Psychology, 110, 35–46. Kitagawa, N. and Spence, C. (2006). Audiotactile multisensory interactions in information processing. Japanese Psychological Research, 48, 158–173. Klatzky, R. L., Lederman, S. J. and Matula, D. E. (1991). Imagined haptic exploration in judgments of object properties. Journal of Experimental Psychology: Learning Memory and Cognition, 17, 314–322. Klatzky, R. L., Lederman, S. J. and Matula, D. E. (1993). Haptic exploration in the presence of vision. Journal of Experimental Psychology: Human Perception and Performance, 19, 726–743. Klatzky, R. L., Lederman, S. J. and Metzger, V. A. (1985). Identifying objects by touch: an ‘expert system’. Perception and Psychophysics, 37, 299–302. Kosslyn, S. M. (1994). Image and brain: the resolution of the imagery debate. Cambridge, MA: MIT Press.

Multisensory product experience

159

Krumhansl, C. L. (2002). Music: a link between cognition and emotion. Current Directions in Psychological Science, 11, 45–50. Laird, D. A. (1932). How the consumer estimates quality by subconscious sensory impressions. Journal of Applied Psychology, 16, 241–246. Lederman, S. J. (1979). Auditory texture perception. Perception, 8, 93–103. Lederman, S. J. and Klatzky, R. L. (2004). Multisensory texture perception. In: G. A. Calvert, C. Spence and B. E. Stein (Eds.) The handbook of multisensory processes, pp. 107–122. Cambridge, MA: MIT Press. Lederman, S. J., Thorne, G. and Jones, B. (1986). Perception of texture by vision and touch: multidimensionality and intersensory integration. Journal of Experimental Psychology: Human Perception and Performance, 12, 169–180. Li, H., Daugherty, T. and Biocca, F. (2002). Impact of 3-D advertising on product knowledge, brand attitude, and purchase intention: the mediating role of presence. Journal of Advertising, 31(3), 43–57. Li, H., Daugherty, T. and Biocca, F. (2003). The role of virtual experience in consumer learning. Journal of Consumer Psychology, 13, 395–407. Lindstrom, M. (2005). Brand sense: build powerful brands through touch, taste, smell, sight, and sound. New York: Free Press. Ludden, G. D. S., Schifferstein, H. N. J. and Hekkert, P. (2007a). Surprise as a design strategy. Design Issues (in press). Ludden, G. D. S., Schifferstein, H. N. J. and Hekkert, P. (2007b). Visual – tactual incongruities in products as sources of surprise. Empirical Studies of the Arts (submitted). Lyon, R. H. (2003). Product sound quality – from perception to design. Sound and Vibration, 37(3), 18–22. MacDonald, A. (2002). The scenario of sensory encounter: cultural factors in sensory–aesthetic experience. In: W. S. Green and P. W. Jordan (Eds.) Pleasure with products: beyond usability, pp. 113–123. London: Taylor and Francis. Maga, J. A. (1974). Influence of color on taste thresholds. Chemical Senses and Flavour, 1, 115–119. Marks, L. E. (1978). The unity of the senses: interrelations among the modalities. New York: Academic Press. Marks, L. E. (2002). The role of attention in chemosensation. Food Quality and Preference, 14, 147–155. Marks, L. E. and Wheeler, M. E. (1998). Attention and the detectability of weak taste stimuli. Chemical Senses, 23, 19–29. Martin, M. G. F. (1995). Bodily awareness: a sense of ownership. In: J. L. Bermudez, A. Marcel and N. Eilan (Eds.) The body and the self, pp. 267–289. Cambridge, MA: MIT Press. Meredith, C., Edworthy, J. and Rose, D. (1999). Observational studies of auditory warnings on the intensive care unit. In: N. A. Stanton and J. Edworthy (Eds.) Human factors in auditory warnings, pp. 305–317. Aldershot: Ashgate. Miller, C. (1993). Scent as a marketing tool: retailers – and even a casino – seek sweet smell of success. Marketing News, 27 (January 18), 271–272. Miron, D., Duncan, G. H. and Bushnell, M. C. (1989). Effects of attention on the intensity and unpleasantness of thermal pain. Pain, 39, 345–352. Momtahan, K., Hétu, R. and Tansley, B. (1993). Audibility and identification of auditory alarms in the operating room and the intensive care unit. Ergonomics, 36, 1159–1176. Mugge, R., Schifferstein, H. N. J. and Schoormans, J. P. L. (2006). The different ways to personalize products: a categorization study. Working paper: Delft University of Technology, the Netherlands. O’Regan, J. K. and Noë, A. (2001). A sensorimotor account of vision and visual consciousness. Behavioral and Brain Sciences, 24, 939–1031. Packard, V. (1957). Hidden persuaders. Harmondsworth, Middlesex: Penguin Books. Pecher, D., Zeelenberg, R. and Barsalou, L. W. (2003). Verifying different-modality properties for concepts produces switching costs. Psychological Science, 14, 119–124. Peck, J. and Childers, T. L. (2003). Individual differences in haptic information processing: the ‘need for touch’ scale. Journal of Consumer Research, 30, 430–442. Pirkl, J. J. (1994). Transgenerational design: products for an aging population. New York: Van Nostrand Reinhold. Pöppel, E. (1988). Mindworks: time and conscious experience. New York: Harcourt Brace Jovanovich. Posner, M. I., Nissen, M. J. and Klein, R. M. (1976). Visual dominance: an information-processing account of its origins and significance. Psychological Review, 83, 157–171. Ramsdall, D. A. (1978). The psychology of the hard-of-hearing and the deafened adult. In: H. Davis and S. R. Silverman (Eds.) Hearing and deafness (4th Ed.), pp. 499–510. New York: Holt, Rinehart and Winston. Redelmeier, D. A. and Tibshirani, R. J. (1997). Association between cellular-telephone calls and motor vehicle collisions. New England Journal of Medicine, 336, 453–458.

160

Product Experience

Reisberg, D. (1992). Auditory imagery. Hillsdale, NJ: Erlbaum. Robinson, D. W. (1988). Threshold of hearing as a function of age and sex for the typical unscreened population. British Journal of Audiology, 22, 5–20. Rock, I. and Victor, J. (1964). Vision and touch: an experimentally created conflict between the two senses. Science, 143, 594–596. Rozin, P. (1982). ‘Taste–smell confusions’ and the duality of the olfactory sense. Perception and Psychophysics, 31, 397–401. Sapherstein, M. B. (1998). The trademark registrability of the Harley-Davidson roar: A multimedia analysis. Retrieved 12/03/2007, from http://www.bc.edu/bc_org/avp/law/st_org/iptf/articles/content/1998101101. html#fnB24 Scanlon, B. A. (1985). Race differences in selection of cheese color. Perceptual and Motor Skills, 61, 314. Scharf, A. and Volkmer, H. P. (2000). The impact of olfactory product expectations on the olfactory product experience. Food Quality and Preference, 11, 497–503. Scherer, K. R. (2003). Vocal communication of emotion: A review of research paradigms. Speech Communication, 40, 227–256. Schifferstein, H. N. J. (2001). Effects of product beliefs on product perception and liking. In: L. Frewer, E. Risvik and H. Schifferstein (Eds.) Food, people and society: A European perspective of consumers’ food choices, pp. 73–96. Berlin: Springer Verlag. Schifferstein, H. N. J. (2006). The relative importance of sensory modalities in product usage: a study of selfreports. Acta Psychologica, 121, 41–64. Schifferstein, H. N. J. (2007). Comparing mental imagery between the sensory modalities (submitted). Schifferstein, H. N. J. and Cleiren, M. P. H. D. (2005). Capturing product experiences: a split-modality approach. Acta Psychologica, 118, 293–318. Schifferstein, H. N. J. and Desmet, P. M. A. (2007). The effect of sensory impairments on product experience and personal well-being. Ergonomics, (in press). Schifferstein, H. N. J. and Smeets, M. A. M. (2006, 27–29 September). Towards the assessment of perceptual style. Paper presented at the 5th Design and Emotion Conference, Göteborg, Sweden. Schifferstein, H. N. J. and Tanudjaja, I. (2004). Visualising fragrances through colours: the mediating role of emotions. Perception, 33, 1249–1266. Schiffman, S. S. (2000). Intensification of sensory properties of foods for the elderly. Journal of Nutrition, 130, 927S–930S. Selcon, S. J., Taylor, R. M. and McKenna, F. P. (1995). Integrating multiple information sources: using redundancy in the design of warnings. Ergonomics, 38, 2362–2370. Smets, G. J. F. and Overbeeke, C. J. (1989). Scent and sound of vision: expressing scent or sound as visual forms. Perceptual and Motor Skills, 69, 227–233. Sorkin, R. D. (1988). Why are people turning off our alarms? Journal of the Acoustical Society of America, 84, 1107–1108. Spangenberg, E. R., Crowley, A. E. and Henderson, P. W. (1996). Improving the store environment: do olfactory cues affect evaluations and behaviors? Journal of Marketing, 60(April), 67–80. Spence, C. (2002). The ICI report on the secret of the senses. London: The Communication Group. Spence, C. (2007a). Audiovisual multisensory integration. Acoustical Science and Technology, 28, 61–70. Spence, C. (2007b, 8–11 February). Creating innovative packaging that appeals to all the senses. Paper presented at the PACE: Packaging and Converting Executive Forum meeting, Paris. Spence, C., Bentley, D. E., Phillips, N., McGlone, F. P. and Jones, A. K. P. (2002). Selective attention to pain: A psychophysical investigation. Experimental Brain Research, 145, 395–402. Spence, C. and Driver, J. (1997a). Cross-modal links in attention between audition, vision, and touch: implications for interface design. International Journal of Cognitive Ergonomics, 1, 351–373. Spence, C. and Driver, J. (1997b). On measuring selective attention to an expected sensory modality. Perception and Psychophysics, 59, 389–403. Spence, C. and Driver, J. (1999). A new approach to the design of multimodal warning signals. In: D. Harris (Ed.) Engineering psychology and cognitive ergonomics Vol. 4: Job design, product design, and human– computer interaction, pp. 455–461. Hampshire: Ashgate Publishing. Spence, C. and Driver, J. (Eds.) (2004). Crossmodal space and crossmodal attention. Oxford: Oxford University Press. Spence, C., McGlone, F. P., Kettenmann, B. and Kobal, G. (2001). Attention to olfaction: a psychophysical investigation. Experimental Brain Research, 138, 432–437. Spence, C., Nicholls, M. E. R. and Driver, J. (2001). The cost of expecting events in the wrong sensory modality. Perception and Psychophysics, 63, 330–336. Spence, C. and Read, L. (2003). Speech shadowing while driving: on the difficulty of splitting attention between eye and ear. Psychological Science, 14, 251–256.

Multisensory product experience

161

Spence, C., Shore, D. I. and Klein, R. M. (2001). Multisensory prior entry. Journal of Experimental Psychology: General, 130, 799–832. Spence, C. and Squire, S. (2003). Multisensory integration: maintaining the perception of synchrony. Current Biology, 12, R519–R521. Spence, C. and Zampini, M. (2006). Auditory contributions to multisensory product perception. Acta Acustica united with Acustica, 92, 1009–1025. Spence, C. and Zampini, M. (2007). Affective design: modulating the pleasantness and forcefulness of aerosol spray: by manipulating aerosol spraying sounds. Co Design, 3 (Supplment 1), 107–121. Stein, B. E. and Meredith, M. A. (1993). The merging of the senses. Cambridge MA: MIT press. Stevens, J. C., Cain, W. S. and Burke, R. J. (1988). Variability of olfactory thresholds. Chemical Senses, 13, 643–653. Stevenson, R. J. and Case, T. I. (2005). Olfactory imagery: a review. Psychonomic Bulletin and Review, 12, 244–264. Strayer, D. L. and Johnston, W. A. (2001). Driven to distraction: dual–task studies of simulated driving and conversing on a cellular telephone. Psychological Science, 12, 462–466. van Beers, R. J., Sittig, A. C. and van der Gon, J. J. D. (1999). Integration of proprioceptive and visual position-information: an experimentally supported model. Journal of Neurophysiology, 81, 1355–1364. van Egmond, R. (2004). Compact disc cover design: transfer of musical style and emotional contents into graphical design. In: D. McDonagh, P. Hekkert, J. van Erp and D. Gyi (Eds.) Design and emotion: The experience of everyday things, pp. 238–242. London: Taylor and Francis. van Erp, J. B. F. and van Veen, H. A. H. C. (2006). Touch down: the effect of artificial touch cues on orientation in microgravity. Neuroscience Letters, 404, 78–82. Velle, W. (1987). Sex differences in sensory functions. Perspectives in Biology and Medicine, 30, 490–522. Veryzer, R. W. and Hutchinson, J. W. (1998). The influence of unity and prototypicality on aesthetic responses to new product designs. Journal of Consumer Research, 24, 374–394. Vlahos, J. (2006). The smell of war. Popular Science, 8, 73–95. von Hornbostel, E. (1931). Über Geruchshelligkeit (About olfactory brightness). Pflügers Archiv für die Gesamte Physiologie des Menschen und der Tiere, 227, 517–538. Walker, R. (1987). The effects of culture, environment, age, and musical training on choices of visual metaphors for sound. Perception and Psychophysics, 42, 491–502. Ward, J., Huckstep, B. and Tsakanikos, E. (2006). Sound–colour synaesthesia: to what extent does it use crossmodal mechanisms common to us all? Cortex, 42, 264–280. Warm, J. S., Dember, W. N. and Parasuraman, R. (1991). Effects of olfactory stimulation on performance and stress in a visual sustained attention task. Journal of the Society of Cosmetic Chemists, 42, 199–210. Washburn, D. A., Jones, L. M., Vijaya Satya, R., Bowers, C. A. and Cortes, A. (2003). Olfactory use in virtual environment training. Modelling and Simulation, 2(3), 19–25. Weel, K. G. C., Boelrijk, A. E. M., Alting, A. C., et al. (2002). Flavor release and perception of flavored whey protein gels: perception is determined by texture rather than by release. Journal of Agricultural and Food Chemistry, 50, 5149–5155. Welch, R. B. and Warren, D. H. (1980). Immediate perceptual response to intersensory discrepancy. Psychological Bulletin, 88, 638–667. Wickens, C. D. (1980). The structure of attentional resources. In: R. Nickerson (Ed.) Attention and Performance VIII, pp. 239–257. Hillsdale, NJ: Erlbaum. Wickens, C. D. (1992). Engineering psychology and human performance (2nd Ed.) New York: HarperCollins. Wrzesniewski, A., McCauley, C. and Rozin, P. (1999). Odor and affect: individual differences in the impact of odor on liking for places, things and people. Chemical Senses, 24, 713–721. Zampini, M., Guest, S. and Spence, C. (2003). The role of auditory cues in modulating the perception of electric tooth brushes. Journal of Dental Research, 82, 929–932. Zampini, M. and Spence, C. (2004). The role of auditory cues in modulating the perceived crispness and staleness of potato chips. Journal of Sensory Studies, 19, 347–363. Zampini, M. and Spence, C. (2005). Modifying the multisensory perception of a carbonated beverage using auditory cues. Food Quality and Preference, 16, 632–641. Zellner, D. A., Bartoli, A. M. and Eckard, R. (1991). Influence of color on odor identification and liking ratings. American Journal of Psychology, 104, 547–561. Zellner, D. A. and Whitten, L. A. (1999). The effect of color intensity and appropriateness on color-induced odor enhancement. American Journal of Psychology, 112, 585–604.

6

HUMAN CAPABILITY AND PRODUCT DESIGN JOHN CLARKSON University of Cambridge, Cambridge, UK

1. INTRODUCTION A US-wide survey of 15,477 working-age adults and computer users, commissioned by Microsoft (2003), asked questions about levels of difficulty users experienced with ordinary daily tasks. The results showed that ‘In the US, 60% (101.4 million) of working age adults who range from 18 to 64 years old are likely or very likely to benefit from accessible technology.’ Another US-wide web-based survey of 1501 internet users, aged 18–75⫹, undertaken by Philips (2004), found that: • Only 13% of the American public believes that in general ‘technology products are easy to use’. • Close to 65% of Americans say ‘they have lost interest in purchasing a technology product because it seemed too complex to setup or operate’. • Only 23% of Americans say ‘they use the full range of features on most new technology products’. • Consumers are beginning to demand that products be accessible for easy operation and set-up … ‘ease-of-use’ is so important to the public (76%) that it is now equal in importance to the dimension of ‘high quality’ (76%). As a direct response to the 2004 survey, Philips have focused on simplifying their products, as evidenced by their new corporate strapline, ‘sense and simplicity’. These surveys show that the proliferation of technology products has not always made life easier for prospective users, rather in some cases it has made it more difficult. Some are excluded from using particular products, some experience undue difficulty and some are merely frustrated (Figure 6.1). Thankfully, some find the same products easy to Product Experience Copyright © 2008 Elsevier Ltd.

165

Population

166

Product Experience

Finds easy

Frustrated

Has difficulty

Most able

Excluded

Least able

FIGURE 6.1 An illustration of capability variation within the user population and its relationship to product exclusion.

use. The mark of a ‘good’ product is then one that minimizes the levels of exclusion, difficulty, and frustration, while maximizing the sense of user satisfaction. Designing a product to minimize exclusion requires knowledge of the demands made by a product on its users’ sensory, cognitive and motor capabilities, and knowledge of the range of these capabilities within the user population. Any users whose capability does not meet that demanded by the product is likely to be excluded from using that product, or at the very least experience difficulty. Hence ‘good’ design requires an understanding of the characteristics and capabilities of the target users of a new product or service, so that it can be designed to operate within their capabilities and meet their needs and desires. The remainder of this chapter describes various forms of information that help to create a better understanding of these characteristics, along with the users’ sensory, cognitive and motor capabilities.

2. USER CHARACTERISTICS Users may be described in many ways. Businesses talk about market segments and preferences, medical professionals talk about symptoms and conditions, government talks about socioeconomic factors, and designers talk about anthropometry and capability. All represent different ways in which to describe the diversity of human form, performance and behaviour. All have some relevance to the design of products and services, however, the focus here will be on anthropometry and, in particular, capability.

2.1. Anthropometry Anthropometric data is used for the study of human body measurement for anthropological classification and comparison. It includes body measurements, such as height, weight and hand size, and functional measurements, principally concerning how far people can reach in different directions. The distribution of these measurements in a population tends to follow a typical ‘bell-shaped’ curve, as illustrated in Figure 6.2. It has been common practice to design for the middle 90% of this variation. However, this approach can exclude the smallest 5% and the largest 5%, who are likely to find the product hard or impossible to use. In practice, the numbers excluded are likely to be even higher, as those who are excluded by height may not be the same as those excluded by arm length, etc. It is also important to take into account the variation in data distribution by gender, age and geographical location. For example, a product which includes 90% of UK men may include only a small proportion of UK women, a small proportion of men over 75 or fewer than 90% of men from another country.

167

Human capability and product design

Average (mean)

Middle 90%

Smallest 5%

Largest 5%

FIGURE 6.2 Typical variation of anthropometric measurement.

Anthropometric data is relatively easy to find, with basic data available for many countries. However, consistency across different data sets, and often within the same data set, is less easy to find. Data for seemingly related measurements is often gathered using different population samples, making more detailed comparative analysis difficult. More recently, some efforts have been made to recognize the variation of measurement by age. For example, the UK Department of Trade and Industry (DTI) has published separate data for children (Norris and Wilson, 1995), adults (Peebles and Norris, 1998) and older adults (Smith et al., 2000). This provides a useful introduction to the anthropometric variation that may be associated with an aging population.

2.2. Capability People have a range of sensory, cognitive and motor capabilities which they use when interacting with products. Losses in any of these capabilities can make it difficult or even impossible to use a product or service. Since the incidence of capability loss increases with age (Figure 6.3), the number of those who have difficulty using products and services will be greater for older users. This is compounded by the fact that the population of the developed world is aging (Figure 6.4), with increases in the number of those in older age bands, and corresponding

50

85⫹

80–84

75–79

70–74

65–69

60–64

55–59

50–54

45–49

40–44

35–39

30–34

25–29

0

20–24

25

16–19

GB population (%)

75

Age band FIGURE 6.3 Prevalence of capability loss in Great Britain as a function of age.

168

Product Experience

% Increase 1990–2050

Italy

UK

Germany

Spain

0

USA

60

France

120

Japan

Increase in number of people over 65 (%)

180

FIGURE 6.4 Population aging.

decreases in younger groups. In many countries 50⫹ is the fastest growing segment of the population. In the UK since 1900, life expectancy has increased by an average of 2.5 years per decade and by 2020 nearly 50% of the UK population will be over 50. Similarly, 20% of the USA population and 25% in Japan will be over 65. Interacting with most products involves three main stages: Perceiving, thinking and acting. For example, opening a carton of juice involves: • Perceiving: Sensing where the opening is. • Thinking: Determining how to open it. • Acting: Carrying out the movements required to actually open the carton. Each of these primarily uses a different kind of capability, for example: • Sensory: Which covers areas like vision and hearing. • Cognitive: Areas considered with thought, intelligence and communication. • Motor: Relating to movement, such as dexterity, reach and stretch, and locomotion. As a result, performing an action as a whole involves a combination of sensory, cognitive, and motor capabilities. Even simple functions like seeing are, in reality, a result of sensory, motor, and cognitive capabilities working together. What our separate senses detect is complex and unorganized, and needs to be filtered and processed by the brain to enable decisions and actions. Higher level functions like attention; recognizing objects visually; understanding speech and planning movement; also involve cognition. This is best described as an interaction loop, such as the one illustrated in Figure 6.5. Sensory input is processed by attention and higher cognition, which generate a motor response, causing a visible effect in the world. A person’s overall capability may be described by the combination of their sensory, cognitive and motor capabilities. Conversely their overall capability loss may be ascribed to one or more individual losses, since it is often the case that people suffer multiple minor capability losses in later life.

2.3. Prevalence According to the UK Office of National Statistics (ONS), the total number of people in Great Britain identified as having some degree of reduced capability is over 8.5 million, approximately 18% of the GB population. This figure is in line with those reported by many other countries across the world (UN, 1990). The prevalence and extent of capability loss is not the same for all capability types. Figure 6.6 shows the amount of loss for some of the capabilities most relevant to the use

169

Human capability and product design

Memory

Cognitive processor

Perceptual processor

Product

Motor processor

Actions

Body / limbs

FIGURE 6.5 The interaction between sensory, cognitive and motor capabilities.

GB population (thousands)

3000 High

Medium

Low

2000

1000

ea om m

un

H

ic at io n In fu tel nc lec tio tu ni al ng

g rin

on si Vi

y rit te ex D

C

Lo

co

m

ot io n R ea c st h re a tc nd h

0

Areas of disability

FIGURE 6.6 Prevalence of capability loss in Great Britain by type for those aged 16 or over.

of products and services for those aged 16 or over. Closer examination of the underlying data reveals that those aged 75 or over are ten times more likely to exhibit some capability loss than those aged 16–49. With reference to Figure 6.6, a person’s sensory capability describes the combination of capabilities in vision (recognition and reading) and hearing (communication and signal discernment). According to the ONS, the total number of people in Great Britain identified as having reduced sensory capability (vision and hearing only) is over 3.9 million (8.3% of the population). A person’s cognitive capability describes the combination of capabilities in intellectual functioning (reading, writing, counting and memory) and communication (interpersonal

170

GB population (%)

6

Product Experience

Male

Female

4

2

0

High ability

Medium ability

Low ability

FIGURE 6.7 Overall capability – variation by ability and gender for those in Great Britain aged 16 or over.

understanding). Again, according to the ONS, the total number of people in Great Britain identified as having reduced cognitive capability is over 2.6 million (5.5% of the population). A person’s motor capability describes the combination of capabilities in locomotion (walking, stair climbing, bending, and balance), reach and stretch (upper-body strength and control) and dexterity (picking up, carrying, holding and twisting using the hands). According to the ONS, the total number of people in Great Britain identified as having reduced motor capability is over 6.7 million (14.3% of the population). Note that the sum of these figures exceeds the total prevalence of reduced capability (3.9 ⫹ 2.6 ⫹ 6.7 ⬎ 8.5) since many people, especially those in older age groups, experience more than one capability loss. Designs that only address one of these losses will still exclude these people from using the product. It is therefore important to consider combinations of capability loss when evaluating and improving designs. It is also important to recognize the hierarchy of sensory, cognitive and motor capabilities: A person’s lack of sensory capability may exclude them from using a product regardless of their cognitive or motor capabilities. From the product perspective, an excessive sensory demand will not be counterbalanced by a low cognitive or motor demand. For example, unreadable labels on well-designed controls for a simple product will result in unnecessary exclusion. Finally, for the purposes of examining products, it can be useful to measure each sensory, cognitive, and motor capability on a four-point scale: 1. Fully able – has adequate use of the capability for everyday activities. 2. Moderately able – experiences some difficulties with the capability in several everyday situations. 3. Partially able – has significant problems with the capability in most everyday situations. 4. Minimally able – cannot use that capability for practical everyday purposes. The original ONS data, illustrated in Figure 6.7 (Grundy et al., 1999), may be integrated to create a cumulative curve, which in turn may be mapped against the four-point scale as shown in Figure 6.8. The latter graph shows how many people in Great Britain (GB) have less than the specified level of ability, indicating how many people would have particular difficulty using a product with that level of demand. It is difficult to identify comparative data for other countries, and what exists is rather superficial in terms of detail (UN, 1990). However, while it is accepted that ‘local’ variation of capability loss might be expected, the discussions that follow in this chapter remain pertinent and indicative of the challenges that designers would face in designing for their local, or indeed an international, user community.

171

Human capability and product design

GB population (%)

20

15

10

5

0 Full ability

Moderate ability

Partial ability

Minimal ability

FIGURE 6.8 Overall capability – cumulative frequency graph showing the proportion of Great Britain population aged 16 or over that does not meet the specified ability level.

3. PRODUCT DESIGN The previous section presented an outline description of users’ capabilities. The following sections will provide detailed descriptions of each capability and their impact on product design. However, we first need to clarify the link between product accessibility and user capability. All products make demands of their users, in the context of a particular task being undertaken in a particular environment. When such demands exceed the capabilities of a particular user, exclusion or difficulty may arise. A product that makes minimal demands is likely to exclude only those users with minimal ability, while one that makes high demands might also exclude those with full ability. The environment will also have an impact on the level of exclusion. For example, bright light cast on a text display can increase exclusion, not only by making the characters more difficult to see, but also by compromising the vision capability of the user. Products that are designed with these challenges in mind are more likely to moderate their demands on the user, increasing accessibility and ultimately the utility of the product. Figure 6.1 illustrated the proposition that for every user that might be excluded from using a product, there are likely to be more that find difficulty or frustration in using the same product. Equally, it is proposed that effort expended in reducing exclusion is likely to have a beneficial effect on levels of difficulty and frustration. In practice, there is a continuum between these modes of interaction since, for example, severe difficulty may quickly lead to exclusion for a tired user, or the development of a coping strategy may enable a previously excluded user to operate a product. Consequently, the exact shape of the curve in Figure 6.1, and the boundaries between the different modes of interaction, will vary from product to product. A thorough understanding of user capability enables informed design decisions to be made regarding user interaction with the product. For example, an appreciation of how dexterity capability might be reduced due to increased age or presence of disease, enables a designer to design interactions involving the hands for a wider range of capabilities, thus reducing exclusion. Hence, the following sections describe each of the sensory, cognitive and motor capabilities in turn, and their likely variation within the user population. In addition, design guidance is provided to aid inclusion, along with descriptions as to how the four-point scale presented in the previous section may be applied to each capability.

172

Product Experience

4. VISION Our sense of vision allows us to perceive the world in images, motion and color. We use information from the visual sense in order to move around and interact with objects and environments. The effective design of any product or environment should take into account the range of human visual abilities. The human eye works by allowing light to enter through the pupil (Figure 6.9). The light rays then pass through the lens where they are refracted. These rays are then focused on the retina, the surface at the back of the eye that is sensitive to light. The retina consists of two types of receptor cells known as rods and cones. Rods allow us to differentiate between shades of black and white, while cones allow us to see color in adequate levels of illumination. Most cones are concentrated in a region of the retina called the fovea. This region allows us to see the greatest detail. The presence of stimulation of the retinal cells is transmitted to the brain via the optic nerve at the back of the eye. The brain utilizes and interprets the signals from both eyes to construct the visual image that we see. Because images from both eyes are used, we are able to see in three dimensions and perceive depth (see Chapter 1). The visual system serves various functions, four of which are discussed in the context of designing products: • • • •

Visual acuity. Contrast sensitivity. Color perception. Usable visual field.

Because of aging and various eye conditions, the structure and function of the eye can change. These changes can cause various levels of reduction in visual acuity and contrast sensitivity.

4.1. Visual acuity Visual acuity is the ability of the eye to see fine detail. It is dependent on two factors: The viewing distance from the product; and the smallest feature or space that the eye can detect

Optic nerve

Pupil Lens

Retina

FIGURE 6.9 The shape and structure of the human eye.

173

Human capability and product design

(for example, text size or stroke thickness). Reduced visual acuity, due to a reduced ability of the lens in the eye to focus light on the retina, results in images that appear blurry. Designing for reduced visual acuity involves considering the type size, weight, and font style of text (Arditi, 2006a). The ability to perceive details also depends on the ambient illumination and the contrast between the foreground and background. Visual acuity is important for tasks such as: Seeing and reading text in print material and on products (for example, product manuals and product controls); seeing and identifying graphical symbols in print material and on products; reading signage on the roads and in public spaces; and recognizing faces at various distances. Particular problems can arise when there is a requirement to change the viewing distance as part of an activity, since the ability of the eye to accommodate such change (Figure 6.10) reduces with age (Vassilief and Dain, 1986).

4.2. Contrast sensitivity Contrast is the difference in brightness between the foreground and the background. It is related to the size and illumination of the object to be detected. For example, the ability to distinguish a number key from the body of a mobile phone depends on the contrast between the two. If a person has low contrast sensitivity, it means that low contrast text (for example, gray on white) becomes very difficult or impossible to distinguish. The greater the contrast between foreground and background text, the more likely that it will be easily detected by people with low contrast sensitivity. The same holds true for controls on products. Buttons, knobs, indicators, and levers all need to be of sufficient contrast from the rest of the product body to be easily visible. Maximum contrast occurs with white on black or vice versa. For colors, the difference in lightness between the foreground and background colors determines the contrast. Contrast sensitivity is important for activities such as reading and recognizing low contrast text, moving around in the environment, and detecting the outlines of buildings, roads, and pavements.

4.3. Color perception Color can be described in terms of its hue, saturation and brightness. When choosing colors for product features, two important issues for consideration are color contrast (Arditi, 2006b) and color blindness.

Ocular accommodation (dioptres)

14 12

Mean

10 Upper limit

8 6 4 Lower limit

2 0 0

10

20

30

40

50

60

70

Age (years)

FIGURE 6.10 Changes in accommodation ability of the eye with age (Vassilief and Dain, 1986).

174

Product Experience

Certain combinations of text and background color result in poor contrast. High luminance foreground on a low luminance background (or vice versa) gives maximum contrast. Color blindness is the inability to distinguish between various colors: It does not mean that people with color blindness cannot perceive color (this is extremely rare). Color blindness refers to a wide spectrum of effects, but the most common form is red– green color blindness, where a person may be unable to effectively distinguish, under all lighting conditions, between colors from red to green in the color spectrum.

4.4. Usable visual field With increasing age and various eye conditions, the usable field of view can change. This can occur in two ways: The usable field of view can be reduced from the centre of the visual field and move out (central field loss), or it can start from the outside to the inside (peripheral field loss). The central visual field is used for focusing and perceiving detail. When the central field is obscured, tasks that require perception of detail (such as reading) become very difficult. People generally adapt to compensate for this loss and attempt to use the peripheral visual field. However, the peripheral field is less suited for these tasks. A reduction in the usable peripheral visual field results in tunnel vision and can affect mobility. The parking ticket machine in Figure 6.11A shows controls that are placed far away from the dispensing slot. For a user with peripheral field loss, the slot might be more difficult to locate. Conversely, the machine shown in Figure 6.11B, like most ATMs, is a good example where the dispensing slot is close to the display and controls.

4.5. Context of use The environment plays an important role in determining the ability of a person to have a successful visual interaction. It can affect interaction with the product by introducing effects such as glare, and can compromise the ability of the person using the product with distracting ambient lighting. Glare results if the intensity of light entering the eye off a surface is greater than the ambient intensity to which the eye is accommodated. Highly reflective surfaces can cause glare problems for people with reduced visual ability.

Controls

Dispensing slot

A Poor design?

B Better design?

FIGURE 6.11 The positioning of key controls can have a significant impact on the usability and accessibility of a complex interface.

Human capability and product design

175

The location of the product in the environment is also an important consideration for any design, since visual acuity depends on the distance from the product. For example, if signage is installed too far away or very high up (despite large text size and good contrast), problems may occur. Information on displays and screens can also suffer degradation when they are installed in locations that demand viewing at an angle.

4.6. Design guidance Better design can be achieved as a result of a better understanding of the above capabilities and associated capability losses (Evamy and Roberts, 2003; RNIB, 2006). The following guidance with regard to design of the product or service should assist in making designs more inclusive: • Attempt to make text as large as possible within the constraints of the design, and maximize the contrast between foreground text and the background. • Where possible use sans-serif fonts (such as Arial) at larger text sizes with plain instead of patterned backgrounds to increase clarity. • Avoid the use of decorative and cursive font styles (for example, fonts that mimic handwriting) in favor of clearer, more legible, sans-serif typefaces. • Attempt to make graphical symbols as large and clear as possible within the constraints of the design. • Attempt maximum contrast between product parts (such as buttons, keys and other controls) against the product body within the constraints of the color palette chosen for a design project. • Keep the different forms of color blindness in mind when choosing the color palette for a design project. If red and green are to be used together, try to provide an alternative clue (such as a text description) as to what the lights mean. • Attempt to avoid shiny and highly reflective surfaces that increase the likelihood of glare problems, using materials with matte finishes where possible. The following guidance with regard to design of the environment should also assist in making designs more inclusive: • Reduce glare by positioning light sources away from the user’s line of sight and by using shielding or diffusers on light sources. • Consider providing adjustable light sources (such as lamps) to allow different users to set the lighting environment to their needs. • Reduce glare and angle of view problems by providing displays and screens that can easily be repositioned.

4.7. Prevalence data Finally, for the purposes of examining products, it can be useful to measure vision capability (with any desired vision aids) on a four-point scale: 1. Full ability – can recognize a friend across the street with no difficulty; – can read ordinary news print with no difficulty. 2. Moderate ability – can recognize a friend across the room; – can read large print at an average reading distance.

176

Product Experience

GB 16⫹ population (%)

15

10

5

0 Full ability

Moderate ability

Partial ability

Minimal ability

FIGURE 6.12 Vision capability – cumulative frequency graph showing the proportion of GB population aged 16 or over that does not meet the specified ability level.

3. Partial ability – can recognize a friend at arms length away; – can read a newspaper headline. 4. Minimal ability – cannot tell where the windows are in daylight. Figure 6.12 presents cumulative values for vision capability, showing how many people in Great Britain have less than the specified level of ability, indicating how many people would have difficulty using a product with that level of demand. The vertical scale used matches that used of subsequent graphs to aid comparison of levels of loss for different capabilities.

5. HEARING The sense of hearing is the human response to sound vibration. We can identify simple sounds such as beeps and tones, and complex sounds such as speech and music. Hearing also allows for speech perception and understanding, which forms the basis of our ability to communicate with others. The human ear is divided into three parts: The outer ear, the middle ear and the inner ear (Figure 6.13). The outer ear collects sound energy and channels it to the middle ear via the ear drum. The sound energy is converted into mechanical vibrations of small bones in the middle ear. These vibrations are transmitted to the inner ear (cochlea) which is filled with fluid. A membrane in the cochlea picks up the fluid vibration and converts the vibrations into neural impulses. These impulses are then transmitted to the brain via the auditory nerve (see Chapter 3). When blockages exist anywhere in the passage from the outer ear to the middle ear, it results in conductive hearing loss. This results in the loss of ability to hear faint sounds. When the cochlea is affected by aging or disease, it results in sensory neural hearing loss. This affects the quality of the sound detected and results in loss of ability to understand speech and discriminate various sounds (Moore, 2003). Three major hearing functions are considered in design applications: 1. Sound detection. 2. Speech discrimination. 3. Sound localization.

177

Human capability and product design

Outer ear

Middle ear

Inner ear

Cochlea Cochlear nerve

Ear canal Eardrum

Auditory bones

FIGURE 6.13 The shape and structure of the human ear.

Changes in hearing function are known to occur with age. With conductive hearing loss, the ability to detect faint sounds is reduced, along with the ability to hear sounds of higher pitch. A person’s hearing threshold therefore increases. Speech discrimination becomes more difficult, especially in the presence of noise. Also, the ability to localize sounds decreases with age, particularly for sounds with relatively low volume and short duration (for example, watch beeps). These changes in hearing functions need to be considered in the design of products with auditory output.

5.1. Sound detection

Hearing loss (decibels)

Sound detection is the ability to detect beeps, tones and other sound output from various products. Sound vibrations can range from very low intensity to high intensity – this is perceived as loudness or volume. The sound vibration of simple tones can range from low frequency to high frequency – this is perceived as pitch. Complex sounds such as speech and music contain a range of frequencies at various levels of intensity. In order to detect a sound, it must be of a sufficient loudness and within a frequency range (Figure 6.14) that is audible to the listener, taking account of the variation that arises due to aging (Beales, 1965). 0 ⫺10 ⫺20 ⫺30 ⫺40 ⫺50 ⫺60 ⫺70 ⫺80 ⫺90 ⫺100

10–19 years 20–29 years 30–39 years 40–49 years 50–59 years 60–69 years 70–79 years

500

1000

2000

4000

8000

Frequency (Hertz)

FIGURE 6.14 Changes in frequency response of the ear with age (Beales, 1965).

178

Product Experience

Detecting sounds in the presence of noise is more difficult than detecting sounds in isolation. For example, hearing a phone conversation in a crowded restaurant or hearing a phone ring while the television is on require the ability to distinguish the sounds from background noise. Most real world tasks take place with some level of ambient noise.

5.2. Speech discrimination Speech discrimination is the ability to understand speech in quiet and noisy environments. Human ability to discriminate speech is an important consideration when designing products that facilitate verbal communication, and products that use speech output. Examples include communication products (such as telephones and mobile phones), ATMs, car navigation systems, and even accessible technology that aids people with low vision via alternative speech output. Speech sound can be detected if it is of a sufficient loudness, but discrimination of that speech can be difficult for some people, especially in the presence of noise (Moore, 2006). Male or female voices can be used for speech. Generally male voices are of lower pitch and are more likely to be within the range of hearing. However, the defining characteristics of speech, the consonants, may still be difficult to detect for those exhibiting a loss of high-frequency hearing capability. Various accents and intonation can also affect how well speech is understood. Speech output can be pre-recorded and replayed or speech can be synthesized. Synthesized speech is more difficult for older people to understand, especially if it is recorded, played back or synthesized at high speed (a large number of words per minute).

5.3. Sound localization Sound localization is the ability to tell the direction from which a sound is originating. When sound localization ability is low, it affects a person’s safe interaction in an environment. Lack of sound localization ability can influence safe movement in public spaces. If a person cannot tell the direction of an approaching vehicle, with or without accompanying horn or siren, it may lead to fatal consequences. Sound localization is important when interacting with products that warn the user and indicate their location by using sound output. People with low sound localization ability cannot tell that it is their own watch or reminder alarming and may miss, for example, a reminder to take medication. Hence, such products need to indicate in other ways to the user that it is a given time. People with loss of this ability also have trouble with their mobile phone, as they cannot tell that it is their own phone that is ringing. For other products in a home or office environment, it might be difficult to identify which appliance is giving sound output when a few appliances are sounding at the same time.

5.4. Context of use The most important factor that affects hearing is the presence of noise. Noise is essentially the ambient sound environment that interferes with the perception of the sound of interest. Noise introduces hearing demands, in that the user has to discriminate the sound of interest from a mixture of other sounds. Noise can range from low intensity to high intensity, and from general white noise to background speech and music (such as in a pub, restaurant or crowded area). Spaces that introduce large amounts of reflection and reverberation of sound can cause problems with hearing. The sound becomes less intelligible and it can be difficult to discriminate. This occurs in public spaces where announcements are important, such as train and underground stations, sports arenas and music halls. The increased reverberation

Human capability and product design

179

already affects people with average hearing abilities, and affects people with lower hearing abilities to a much larger extent.

5.5. Design guidance Better design can be achieved as a result of a better understanding of the above capabilities and associated capability losses (ASHA, 2006; RNID, 2006). The following guidance should assist in making designs more inclusive: • Make volume levels adjustable if possible and try to ensure that frequencies of sound are in the range 800 to 1000 Hz. • Avoid synthesized speech in favor of natural speech (recorded) if possible, and use lower pitched voices in preference to higher pitched voices. • Attempt to provide alternative feedback (such as visual or tactile) for people with very low hearing ability and facilitate connections with auditory aids. • Design environments and spaces to minimize background noises, sound reflection, and reverberation as much as possible to ensure clarity of sound transmission. • Attempt to ensure that when sounds of high pitch are used, they are of a long duration to maximize detection.

5.6. Prevalence data Finally, for the purposes of examining products, it can be useful to measure hearing capability (with any desired hearing aids) on a four-point scale: 1. Full ability – can hear a conversation in a moderate amount of background noise without great difficulty; – can follow a TV program at a volume that others find acceptable. 2. Moderate ability – can understand someone talking with a loud voice in a quiet room with no difficulty; – can hear someone talking over the telephone; – can hear a doorbell or alarm clock. 3. Partial ability – can follow a TV program if the volume is turned up. 4. Minimal ability – cannot hear any sounds at all. Figure 6.15 presents cumulative values for hearing capability, showing how many people in Great Britain have less than the specified level of ability, indicating how many people would have difficulty using a product with that level of demand.

6. INTELLECTUAL FUNCTIONING Most of the processes that underlie intellectual function occur in the brain, for example, memory and attention (Baddeley, 2004; Wickens and Hollands, 2000). The brain organizes incoming sensory information, processes it in the light of conscious awareness and attention, and initiates responses in the form of actions. Biological studies have shown that different regions of the brain are involved in different intellectual functions, such as: Perception, memory, and movement initiation as well as coordination and speech (Figure 6.16). However the precise nature of how the

180

Product Experience

GB population (%)

15

10

5

0 Full ability

Moderate ability

Partial ability

Minimal ability

FIGURE 6.15 Hearing capability – cumulative frequency graph showing the proportion of GB population aged 16 or over that does not meet the specified ability level. Basic movements Motor control areas

Skilled movements Emotions

Sensory areas

Memory, awareness

Visual object recognition

Making speech and language

Basic vision Balance and muscle coordination

Smell Hearing

Understanding speech and language

FIGURE 6.16 Intellectual functioning capability – summary of brain functions.

brain works is not observable or measurable, so cognitive psychologists have to rely on hypothetical models to help their understanding. Well-established memories and skills are usually unaffected with aging, whereas learning new things tends to take longer. The time required to make decisions and respond to sensory information also increases, as does the error rate for tasks requiring memory, numeracy and spatial orientation. The ability to perform simultaneous cognitive tasks also decreases. Degenerative brain disorders are more prevalent with age. These include Parkinson’s disease, Alzheimer’s, and vascular dementia, which can affect memory, attention, movement, perception, reasoning and social interaction. Although there are a wide range of functions carried out by the brain, the following are identified as the most significant functions for product design: • • • •

Working memory. Attention and performance. Visual-spatial thinking. Learning, recall and long-term memory.

Working memory describes the conscious ability to temporarily store, process and rearrange information (Figure 6.17). Attention and performance refers to the number

181

Human capability and product design

Long term memory Environment

Learning and recall Working memory Attention and performance Visual-spatial thinking

Perceptual processor

Ref le resp x level ons e

Response selection

Product in the world

Execution

FIGURE 6.17 Intellectual functioning capability – a hypothetical model.

of different things that can be kept in mind at once, and the speed at which information can be processed, decisions made or problems solved. Visual-spatial thinking refers to the ability to process, manipulate and rotate objects in mental space. Data can also be stored in the form of words, and sequences of linked past experiences can be remembered together. Information used repeatedly in working memory can be learnt, and stored in longterm memory indefinitely. Information can also be recalled into working memory when required. For example, a prior experience with similar products can help to learn the features of a new one. At a higher level, the sensory, cognitive and motor functions are integrated together within the brain, and understanding the interaction between all these functions is crucial for product design.

6.1. Working memory Working memory refers to the ability to manipulate and rearrange data within conscious thought, and these data can be in many different forms, such as numbers, images or words. In its simplest form, short-term memory refers to a fixed capacity number of bins that can simultaneously hold information. For example, when a person reads a random string of single digit numbers, each character takes up one bin, and the person can only correctly recall the number of characters equal to the number of bins they have. However, this process is further complicated by chunking, which allows information to be stored as chunks within each bin. A common example of chunking is letters joining together to form words, and words joining to form phrases.

182

Product Experience

In everyday tasks, working memory (as the fixed capacity number of bins) operates together with ‘attention and performance’, and ‘visual-spatial thinking’. Working memory can be thought of as a scratch pad or rough working area for items being attended to, where such data may have come from sensory perception, or by recall from long-term memory. The capacity of working memory in adults has been found to be around 5 to 9 chunks or items (Miller, 1956).

6.2. Attention and performance Attention can be consciously directed at specific tasks, or it can be ‘grabbed’ by an event such as a flashing light or sound. A person could be in deep concentration for a specific task, but their attention would still be distracted by a loud noise, a bright flashing light, or the sound of their name. This can be advantageous, to direct a person’s attention towards an impending hazard, or can be a nuisance if a flashing light disrupts attention away from the task at hand. Attention can be overloaded when too many things have to be kept in mind at once, and in this case the user may forget items or tasks. For example, if a person attempted to simultaneously cook with the grill, oven and four hobs while watching the news, it is likely that one of the foods will burn. Working memory performance is also affected by the time taken to process incoming sensory information. For example, when driving a car, sensory information is continually arriving in the form of road hazards, signs, and information from the vehicle. If the user cannot process and respond to the incoming information fast enough, then newly arriving information has to be ignored, or existing information forgotten to free up working memory space. Working memory performance (Figure 6.18) is also known to vary with age (Baddeley, 2004).

6.3. Visual-spatial thinking The most relevant aspects of visual-spatial thinking for product design are the mapping of controls to devices, and visual grouping. The mapping of controls can be illustrated with reference to domestic cookers. Minimum visual-spatial capability is required when the controls are laid out in an identical

130 120

Mean IQ

110

Verbal IQ 100 90

Performance IQ

80 70 0 20

30

40

50

60

70

Age (years)

FIGURE 6.18 Changes in intellectual function with age (Baddeley, 2004).

Human capability and product design

183

manner to the heating elements that they control, or when there is a visual or physical link between the control and device. Both of these allow the user to pick any control, and immediately work out which heating element it will operate, or pick any element, and immediately work out which control operates it. Yet the common practice for nearly all cookers is to relate the heating elements to their controls with a small diagram by each control. Typically the user knows which element they want to operate, but then has to perform a trial and error process to check each control and find out what it does: This requires higher visual-spatial ability, takes longer, and can often be mistaken, even by able bodied users. Understanding these explanatory diagrams is even harder if they need to be rotated to match the objects they represent. The pre-conscious process of visual grouping leads us to see objects as part of a group or separate set because of similarities in their shape, color or spatial positioning (see Chapter 1). This can help users to categorize buttons that have similar functions, thereby making the device easier to understand.

6.4. Learning, recall, and long-term memory Long-term memories require learning and repetition to become fixed, and once stored they can last a long time and be retrieved by triggers or cues. For example, seeing a specific door handle triggers a memory that the door should be pulled to open rather than pushed, without needing to read the sign on the door. Prior use of a product enables us to learn aspects of its features and use that are held in long-term memory, and make it easier to use in the future. To maximize learning the product must provide good feedback so that the users can immediately see the result of their actions and adjust them if necessary. The brain can learn to associate symbols, shapes and colors with particular features or functions, provided sufficient support is available by explanatory text and visual layout. For example, learning the symbols for the buttons used on the toolbar of a word editor is dramatically enhanced if explanatory text appears when the mouse is left over the symbol. Long-term memory may be transferred to help learn and understand other similar products. Our ability to learn decreases with age, and older generations that did not grow up with technology products lack the prior experience to understand functions that the designer may have considered obvious, which is why user testing is crucial to ensure usability.

6.5. Context of use Using products that place a high demand on working memory will be extremely difficult in a busy or noisy environment, and especially when attention can be distracted by other environmental cues such as flashing lights or beeps. Reducing the demand on working memory will make the product more usable in a wider variety of environments and situations. Memories are not perfect and can be forgotten or corrupted. The ability to correctly retain information can be affected by: • How the memory was learnt. A memory is not likely to be well stored if the information was not perceived to be of high importance at the time of learning, or if the learning process was distracted by other factors. • The nature of intervening experiences. Long-term memories decay slowly with time, but are refreshed each time they return to conscious thought.

184

Product Experience

• The conditions experienced when retrieving memories. Memories that had previously been unobtainable can suddenly be recalled when an appropriate cue is provided. Our ability to remember can be improved by clear, distinctive memory cues, such as icons or buttons that are easily distinguished by their shape, size, color, texture, and spatial layout.

6.6. Design guidance Better design can be achieved as a result of a better understanding of the above capabilities and associated capability losses (Huppert, 2002). The following guidance should assist in making designs more inclusive with regard to working memory: • Avoid overloading memory by reducing the number of simultaneous chunks that are presented at any time, and by not exceeding seven as the best practice; examples of this include the number of: – Buttons within any button grouping; – Sub-menu options within any menu; – Choices for any decision. • Try to minimize the levels of hierarchy used in any menu system and ensure that the current location within the overall hierarchy is always obvious; a distinctive spatial position for each menu option can be useful to aid learning and recall. • Avoid the need to scroll a screen to obtain more menu items and provide a ‘back-up’ button to aid menu navigation, ensuring it is as obvious as the ‘select’ button. • Consider the use of tabbed interfaces to provide a distinction between different levels of hierarchy, and a map of the current location within the hierarchy; drop down menus do not provide such distinction. • Use shapes, colors and alignment to group alike buttons and displays into chunks, thereby reducing the time and working memory required to locate a desired feature. The following guidance should assist in making designs more inclusive with regard to visual-spatial thinking: • Align controls in a spatial orientation that matches the objects they affect, minimizing the spatial transformations (for example, rotation) required for interpretation. • Ensure that the control needed is immediately obvious from each device, and that the device that will be affected is immediately obvious from each control. The following guidance should assist in making designs more inclusive with regard to time pressure, attention and learning: • Avoid unnecessary demands for time pressure in product interaction – for example, the expectation that the second of two digits should be entered within a particular time frame. • Ensure that attention is only required to be directed in one place at any one time – multiple simultaneous flashing lights or alarms will simply confuse the user. • Support learning by immediate feedback on any action and ensure all actions are easily and immediately reversible. • Provide error messages to guide the user to fix the problem.

185

Human capability and product design

GB population (%)

15

10

5

0 Full ability

Moderate ability

Partial ability

Minimal ability

FIGURE 6.19 Intellectual functioning capability – cumulative frequency graph showing the proportion of GB population aged 16 or over that does not meet the specified ability level.

6.7. Prevalence data Finally, for the purposes of examining products, it can be useful to measure intellectual capability on a four-point scale: 1. Full ability – can usually do something without forgetting what the task was in the middle of it; – can usually hold a conversation without losing track of what is being said; – can think clearly with no muddling thoughts; – can usually tell the time of day with no confusion; – can watch a 30-minute TV program and tell someone what it was about; – can read a short newspaper article; – can write a short letter to someone with no help; – can count well enough to handle money; – can remember a message and pass it on; – can usually remember to turn things off, such as cookers or taps; – can usually remember the names of friends/family that are seen regularly. 2. Moderate ability – can do seven of these tasks (cannot do four of them). 3. Partial ability – can do four of these tasks (cannot do seven of them). 4. Minimal ability – cannot do any of these tasks. Figure 6.19 presents cumulative values for intellectual functioning capability, showing how many people in Great Britain have less than the specified level of ability, indicating how many people would have difficulty using a product with that level of demand.

7. COMMUNICATION A communication simply requires someone that sends some information, and someone that receives and interprets it. Communication can occur through speech and text, but may also include non-verbal communication such as visual and iconic messages, together with sounds and gestures. In the context of product design, communication refers to the process of interaction between a person and a product. This involves the person’s perception of the product,

186

Product Experience

ENVIRONMENT Think Perceive

Product

Act

FIGURE 6.20 A model of product interaction.

and also their ability to perform actions such as pushing buttons. A person’s ability to communicate depends on their educational level and social skills. Communication can be affected by impairment of the physical capabilities necessary for interacting with a product, such as vision and movement (Wickens and Hollands, 2000). Environmental factors that affect these capabilities will also affect their use for communication purposes. Speech disabilities arise from both speech articulation and voice quality impairments, causing slowing, slurring and stuttering. Language disorders include reduced vocabulary, inappropriate use of words and faulty grammar. Cognitive impairment resulting from brain injuries, such as stroke or developmental disorders, can also affect linguistic capability. An interaction with a product (Figure 6.20) typically involves several cycles of: • Perceiving. • Thinking. • Acting. Both perceiving and acting are high level functions that may involve the combination of several lower level functions such as vision and dexterity. Although these functions are considered separately for the purpose of design guidance and prevalence data, in reality they are all combined during cycles of product interaction. The processes involved during thinking were covered in Section 6, while those relating to perceiving and acting are detailed here.

7.1. Perceiving Perceiving is the ability to comprehend information, which can be in forms such as speech, text, sounds, shapes or images. In addition to these specific outputs from a product, consumers are strongly influenced by the device’s general character. The form, color, and style of the product all influence the user’s assessment of its aesthetic, symbolic and practical value. This affects not only their willingness to interact with the device, but also their ability to do so successfully. In the context of product communication, perceiving refers to the ability of a person to receive and interpret information from a product. This information can be presented in many different forms, for example: • Speech, that can be used as an auditory output to convey detailed information. • Sounds, that may be used to enhance understanding when using a product, such as providing feedback for whether an operation was successful or not.

Human capability and product design

187

• Text, that can be used to describe what buttons do, provide detailed instructions, or be presented on a screen as output. • Images, that can be more powerful and quicker to understand than words, although some images can be too small, complex or abstract, in which case the meaning is lost. • The physical arrangement of a product, that can be used to convey information, such as a slider being adjacent to the device it controls, and the position of the slider indicating its current state. • The shape and form of a product, that should assist the user to understand which areas they can interact with, and in what way. Different people prefer to receive information in different forms, so understanding is most likely to be successful when information is presented in multiple different ways.

7.2. Acting The ability to transfer information to a product can occur in many different ways, such as the direct manipulation of physical controls, or more indirectly by speaking or entering text. In the context of product communication, acting refers to the ability to transfer information to a product, through the correct manipulation of its interface. Acting can be categorized according to whether the action is physical or symbolic. Physical actions refer to the manipulation of a control to achieve a prescribed physical effect. Such controls include: Discrete controls, for example, power and light switches; and analog controls, for example, dimmer switches and brake pedals. Symbolic actions refer to those that confer no physical effect, but can control a product after being interpreted. Such actions include: Selecting menu options; using touch screens or clicking on icons; and using a keyboard to enter text or speaking to a product. Symbolic actions can provide much more diversity than physical ones, yet must be carefully thought out by designers to ensure that they remain simple to use and provide an equivalent level of feedback.

7.3. Context of use A person’s ability to communicate can be influenced by their background, nationality or place of origin. Text and speech may require translating into different languages, whereas pictures have the potential to be understood independently of a person’s native language. A communication with a product uses lower level functions such as vision, hearing and dexterity, so environmental factors that hamper these functions will also hamper product communication.

7.4. Design guidance Better design can be achieved as a result of a better understanding of the above capabilities and associated capability losses. The following guidance should assist in making designs more inclusive: • Ensure the areas that the user can interact with, and the correct way to interact with them, are obvious from the overall form of the device. • Ensure that an uninitiated user can form a correct mental model of how the controls will affect the product and provide positive feedback so that the user can ascertain when their actions have been successful.

188

Product Experience

GB population (%)

15

10

5

0 Full ability

Moderate ability

Partial ability

Minimal ability

FIGURE 6.21 Communication capability – cumulative frequency graph showing the proportion of GB population aged 16 or over that does not meet the specified ability level.

• Ensure that the current state or mode of the device is obvious and avoid unnecessarily high demands on user capabilities during product interaction. • Provide helpful assistance in the event that the user has performed an incorrect action, detailing why their action was unsuccessful and what options are available. • Minimize the adverse consequences when errors or mistakes do occur and ensure all actions are reversible. • Provide the potential for information to be transferred by different modes, such as textual, verbal, pictorial, tactile, lights and sounds.

7.5. Prevalence data Finally, for the purposes of examining products, it can be useful to measure communication capability on a four-point scale: 1. Full ability – can communicate with strangers with no difficulty. 2. Moderate ability – can communicate with strangers with some difficulty; – can communicate with well known people easily. 3. Partial ability – can communicate with strangers with extreme difficulty; – can communicate with well known people with some difficulty. 4. Minimal ability – cannot communicate with strangers or well known people. Figure 6.21 presents cumulative values for hearing capability, showing how many people in Great Britain have less than the specified level of ability, indicating how many people would have difficulty using a product with that level of demand.

8. LOCOMOTION Locomotion is the ability to sit down and stand up, get up and down from the floor, or move and walk around in the environment. Activities such as walking, getting in and out of vehicles, on and off furniture, and maintaining balance are affected by loss of

Human capability and product design

189

locomotion ability. In order to move around, we require adequate muscle, motor control, and balance. Our legs and feet support the rest of our body and provide the force to propel us forward. Age related musculo-skeletal conditions cause loss of locomotion ability. In the case of arthritis joint mobility is limited, leading to reduced range of leg and lower body movement. The ability to maintain balance is also reduced, resulting in an increased risk of falling. Because muscle strength in the lower body such as the legs decreases with an increase in age, the arms are more frequently used to support the body when, for instance, sitting down and standing up. The reduction in visual ability with age also causes problems with moving around an environment. Coupled with reduced muscular strength, it can be difficult for older and disabled people to move around with speed and agility. The following locomotion functions are described in the context of product and service design: • Walking and balance. • Getting up, down, in and out.

8.1. Walking and balance In order to minimize the locomotion capability required to move around, good design should attempt to reduce the distance for continuous walking without rest, provide support to help people maintain balance, and ensure access is possible for those who cannot ascend or descend steps.

8.2. Getting up, down, in and out Actions such as kneeling or sitting down, standing up, or getting into and out of confined spaces or vehicles can be very difficult for people with low locomotion ability. Reducing the muscle strength and flexibility required to move the body into these required positions provides a more satisfying product for all, and helps to include those with reduced locomotion capability. This can be achieved through careful use of dimensions and shapes, and making allowance for the hands to help move the body around.

8.3. Context of use Provision of a suitable environment can dramatically affect those with reduced locomotion capability, who often use various aids such as walking sticks, crutches, frames and trolleys, wheelchairs and scooters, in order to increase their mobility. Product and environmental design should factor the use of these aids into design solutions. Eliminating steps and space constrictions to help wheelchairs and other mobility aids results in a design that also benefits those with push-chairs, bicycles, suitcases on wheels, etc. Balance aids are essential in transportation situations such as buses and trains, but can also reduce design exclusion and increase user satisfaction to aid any situation involving standing for long periods, such as when queuing. Environments should provide adequate capability for people to rest while getting from one place to another. Regular seating intervals can also be useful for those carrying heavy rucksacks, or who want to stop for a rest or to tie-up their shoelaces.

190

Product Experience

8.4. Design guidance Better design can be achieved as a result of a better understanding of the above capabilities and associated capability losses. The following guidance should assist in making designs more inclusive: • Attempt to provide adequate space for access and egress when designing doorways, entrances, and exits. • Consider the use of locomotion aids such as walkers, wheelchairs, and scooters in setting the dimensions of doorways, entrances and pathways. • Provide adequate seating at regular intervals in public spaces such as parks, airports, and shopping centres. • Furniture, shower, and toilet design should assist actions such as sitting down, standing up, getting in and out, by providing grab bars, handles or other means of support. • Design items such as seats, showers, and toilets to assist actions such as sitting down and standing up, or getting in and out, by providing grab bars, handles or other means of support. • Attempt to integrate grab bars and handles into the overall aesthetic appeal of the design and avoid designs that look ‘medical’ or ‘assistive’. • Reduce the need to bend the back or reach below waist level for any product interaction.

8.5. Prevalence data Finally, for the purposes of examining products, it can be useful to measure locomotion capability on a four-point scale: 1. Full ability – can walk 250 metres with no stopping or severe discomfort; – can ascend/descend a flight of stairs with no handrails and with no resting; – can use a dustpan and brush to sweep the floor and straighten up again. 2. Moderate ability – can walk 175 metres with no stopping or severe discomfort; – can ascend/descend 12 steps with no rests while using a handrail; – can balance for long periods without holding on to something; – can reach down to pick something up from the floor. GB 16⫹ population (%)

15

10

5

0 Full ability

Moderate ability

Partial ability

Minimal ability

FIGURE 6.22 Locomotion capability – cumulative frequency graph showing the proportion of GB population aged 16 or over that does not meet the specified ability level.

191

Human capability and product design

3. Partial ability – can walk 50 metres with no stopping or severe discomfort; – can ascend/descend 12 steps by using a rail and resting; – can balance for short periods without holding on to something; – can bend knees and straighten up again. 4. Minimal ability – cannot walk or use steps; – cannot balance even for short periods or perform any bending tasks. Figure 6.22 presents cumulative values for locomotion capability, showing how many people in Great Britain have less than the specified level of ability, indicating how many people would have difficulty using a product with that level of demand.

9. REACH AND STRETCH Different products require the ability to reach one or both arms out from the body. This reaching can either occur in front of and above the body, for example to put on a hat, or out to the sides, to reach objects while sitting at a desk (Figure 6.23). Reaching is more difficult to perform when the arm has to be moved further away from the rest position, and is more difficult when two arms have to reach simultaneously. The relative ability to position the hands compared to the head and body depends on the range of motion of the elbow and shoulder joints. The absolute position that the hands can reach to is also influenced by anthropometric data such as body height and arm length, together with other factors such as whether the person is in a wheelchair, although this absolute position is not considered here. Age related conditions such as arthritis can cause reductions in joint mobility and stiffness, leading to limited reaching ability. The distance a person might be able to reach out could be significantly less than the length of the arm. Temporary injuries such as a broken or bruised arm or collarbone will also affect the amount that a person can reach. The following reach and stretch functions are described in the context of product and service design: • Reaching out in front; • Reaching out to the sides.

9.1. Reaching out in front Most products require the ability to put one hand in front of the body to use them. Where possible, requiring both hands to be placed in front of the body simultaneously should be avoided, and using the product should be possible either by reaching the left, or right arm. Requiring a person to reach above their head can be more difficult than just reaching out in front.

A

B

C

FIGURE 6.23 A Some people can reach over their heads; B others only out in front; and C some cannot reach out at all.

192

Product Experience

9.2. Reaching out to the sides Reaching out to the sides is used to access items while sitting at a desk, or to put on a jacket. The further the arm has to reach out to the side or back, the more difficult it can be to reach items.

9.3. Context of use Environmental factors can additionally reduce a person’s reach and stretch capability. For example, clothing such as heavy jackets and multiple layers can reduce a person’s ability and comfort to reach out. Carrying things while using a product can reduce the mobility of one or both hands. Equally, it may not be desirable for the user to have to put down the baby, handbag or carrier bags to use the product.

9.4. Design guidance Better design can be achieved as a result of a better understanding of the above capabilities and associated capability losses. The following guidance should assist in making designs more inclusive: • Allow for single-handed operation where possible, by eliminating the need to reach both hands out simultaneously, and facilitating the option to reach either the left or right arm out to operate a product. • Ensure that products or services that require access by the public are able to cope with the range of heights that people can reach to, including those in wheelchairs. • Minimize the need to exert forces with the arms outstretched or, in particular, when reaching over the head. • Consult available data sources on reach ranges when setting the dimensions of products and environments.

9.5. Prevalence data Finally, for the purposes of examining products, it can be useful to measure reach and stretch capability on a four-point scale: 1. Full ability – can raise both arms up to the head, behind the back or out in front with no difficulty. 2. Moderate ability – can only raise one arm up to head, behind the back or out in front with no difficulty (has difficulty with other arm). 3. Partial ability – can only raise one arm up to head, behind the back or out in front with no difficulty (cannot use other arm). 4. Minimal ability – cannot raise either arm out in front of body or up to head; – cannot put either arm behind back. Figure 6.24 presents cumulative values for reach and stretch capability, showing how many people in Great Britain have less than the specified level of ability, indicating how many people would have difficulty using a product with that level of demand.

193

Human capability and product design

GB population (%)

15

10

5

0 Full ability

Moderate ability

Partial ability

Minimal ability

FIGURE 6.24 Reach and stretch capability – cumulative frequency graph showing the proportion of GB population aged 16 or over that does not meet the specified ability level.

10. DEXTERITY Most products require physical manipulation of controls and manual handling. The hands are used to grasp, move and exert forces to use and operate various products. Objects can be grasped, pushed and pulled. The human hand is composed of four fingers and a thumb (Figure 6.25). Muscles in the hand and arm control the movement of the fingers and the wrist. The presence of the opposable thumb is key to the many manual tasks that we can perform. We can exert clamping forces between fingers, and also clamp and hold larger objects in the palm of the hand. Individual fingers can also be used to exert pulling and pushing forces. Sometimes we use both hands at the same time to manipulate objects. This coordinated movement requires not only strength and dexterity in the fingers, but also visual and motor control ability. There are many causes of pain than can limit dexterity. Arthritis is a common cause of chronic pain, which is especially prevalent in older people. It can cause pain, stiffness, and swelling in the joints. This may affect not only the joints, but also other parts of the body, including muscles and organs. People vary in their ability to tolerate pain caused by those actions required for product use. This means that although the use of a product may be possible for those of a certain capability level, it will be fatiguing and painful. Continued use may lead to flare-ups of chronic pain, making product use impossible. Four major dexterity functions are considered in design applications: 1. 2. 3. 4.

Force exertion without grip. Precision gripping. Power gripping. Two-handed operations.

Various musculo-skeletal conditions such as various forms of arthritis cause joint stiffness, pain, and reduced joint mobility. This results in a reduction of movement and the ability to exert forces on objects. This reduction (Figure 6.26) generally occurs with increasing age (DTI, 2002; Burke et al., 1953).

10.1. Force exertion without grip Forces can be exerted on products with the fingers or the palm of the hand without requiring a grasping movement. Pushing buttons and depressing toaster sliders are examples of

194

Product Experience

Thumb Index Radius Middle Ulna

Ring Little

Wrist bones

Finger bones

FIGURE 6.25 The shape and structure of the human hand.

120 Grip strength

110

Pounds

100 90 80 70

Grip strength endurance

60 50

75–79

70–74

65–69

60–64

55–59

50–54

45–49

40–44

35–39

30–34

25–29

20–24

16–19

12–15

40

Age (years)

FIGURE 6.26 Changes in grip strength with age (Burke et al., 1953).

this type of action. A person must therefore have the ability to exert forces on the product controls and chassis in three planes: 1. Up-down hand motion and forces – adequate pushing force is required to depress levers and sliders of, for example, a toaster. 2. Right-left hand motion and forces – adequate pushing force is required to push the product from side to side on a surface, or push in a button at the side of the product. 3. Hand motion and forces away and toward the body – adequate pushing force is required to push the product away from or towards the body. In general, it is easier for forces to be exerted downward and toward the body, rather than upward and away from the body.

10.2. Precision gripping Precision gripping (or pinch gripping) is the ability to use the fingers and thumb to grasp product controls and exert the required forces to operate them. These grips are low force and high precision movements because they largely depend on the function of individual fingers. The resulting actions can be linear or rotational (twisting) movements. Everyday activities that use this fine manual dexterity include writing (pen gripping) and using chopsticks.

Human capability and product design

195

The pinch grip is used on many controls where the thumb and index finger provide a pinching force when in opposition. Examples include opening a coke can, grasping a small rotary dial, and picking up a small object.

10.3. Power gripping A power grip is a grip formed with the fingers and the palm of the hand in order to move or manipulate objects. This type of grip is used for grasping handles and holding smaller products such as mobile phones and remote controls. Power grips are high force and low precision movements because they use the bulk of the hand for force generation rather than the strength of individual fingers. The resulting actions lead to large arm movements for lifting and picking up larger and heavier objects. Everyday activities such as lifting a mug, a blender or a juice carton require power grips.

10.4. Two handed tasks Many tasks with everyday products require the use of both hands at the same time. Each hand will adopt either a power or precision grasp depending on the task. Tasks such as lifting a heavy casserole dish might require each hand having similar grasp. Tasks such as opening jars might require one hand to have a power grasp while the other hand adopts a precision grasp. The dominant hand (either right or left) is usually the preferred hand for accomplishing the more dextrous action (such as twisting the lid when opening a jar). Some people have only one functional hand. Certain tasks can be more difficult if this hand is not the dominant hand. Two handed tasks or tasks that require complex motions are also more difficult for older and disabled people to perform.

10.5. Context of use Ambient temperatures can affect hand function. Manual dexterity is reduced in colder environments, making it more difficult to perform manual tasks with products. The flexibility and sensitivity of the fingers are decreased in these circumstances. Wearing protective clothing such as gloves can also make it harder to operate controls on products. Activities such as making a call on a mobile phone in winter can be more demanding because of reduced dexterity. Surfaces that are smooth can be difficult to grasp and hold effectively. If such surfaces are placed in environments that are dusty, or where liquid can get onto grasping surfaces, the product becomes more difficult to hold. Products used in kitchen environments can get wet and covered with various cooking ingredients. Hand tools and other products used in workshop environments can become covered with dust particles or grease, or sweat in high-temperature conditions.

10.6. Design guidance Better design can be achieved as a result of a better understanding of the above capabilities and associated capability losses. The following guidance should assist in making designs more inclusive: • Consider the compatibility of grip and intended action on the product, to avoid situations where a product requires a certain type of grip or motion that is not compatible with the overall task.

196

Product Experience

• Attempt to lower all force requirements to operate the product controls (grasping, pushing, pulling, twisting and lifting forces), making allowance for older people and people with disabilities who generally have reduced strength compared to younger and fully able people. • Avoid, where possible, controls that require simultaneous manipulations such as pushing and twisting at the same time, such as those often used with dials and bottle caps. • Utilize pushing in preference to rotating, since for the latter a pincer grip is required in addition to the application of rotational force. • Cover surfaces to be gripped with materials that result in adequate friction between the surface of the product handle and the hand, since slippery or smooth surfaces are more difficult to grasp, whereas rubbery and slightly deformable surfaces are easier and more comfortable to hold.

10.7. Prevalence data Finally, for the purposes of examining products, it can be useful to measure hand function capability on a four-point scale: 1. Full ability – can pick up a safety pin or 2.5 kg bag of potatoes with either hand; – can tie a bow in laces with no difficulty; – can squeeze water from a sponge with either hand. 2. Moderate ability – can pick up a 2.5 kg bag of potatoes with one hand (but not the other); – can use a pen with no difficulty; – can unscrew the lid of a jar of instant coffee with no difficulty; – can wring out light washing with no difficulty. 3. Partial ability – can pick up and hold a safety pin or half a litre of milk in one hand (but not the other); – can turn a tap with one hand (but not the other); – can squeeze water from a sponge with one hand (but not the other). 4. Minimal ability – cannot hold and pick up relatively light objects such as cups and mugs; – cannot manipulate or control small objects; – cannot squeeze objects.

GB population (%)

15

10

5

0

Full ability

Moderate ability

Partial ability

Minimal ability

FIGURE 6.27 Dexterity capability – cumulative frequency graph showing the proportion of GB population aged 16 or over that does not meet the specified ability level.

Human capability and product design

197

Figure 6.27 presents cumulative values for dexterity capability, showing how many people in Great Britain have less than the specified level of ability, indicating how many people would have difficulty using a product with that level of demand.

11. SUMMARY Designing a product to minimize exclusion requires knowledge of the demands made by the product on its users, knowledge of the range of capabilities of the target users, and their prevalence, and knowledge of the context of use. This chapter has provided an introduction to such knowledge. The user capability cumulative frequency graphs indicate that some types of capability loss are much more prevalent than others. In addition, the shape of the graphs provides some indication of the design challenge required to maximize inclusion. For example, significant efforts are required to minimize exclusion due to dexterity capability losses (Figure 6.27) since the levels of exclusion do not decrease appreciably until the product demand is near minimal, thus accommodating those with partial ability. Such challenges can have a significant impact on user exclusion and the priorities for design, where ‘good’ design ensures that a product can operate within the capabilities of as many users as possible and meet their needs and desires. There are likely to be some aspects of a design or concept that can be improved at little cost, for example, the font type and size used on a safety label, while others may demand more radical change, such as the redesign of a menu-driven interface to avoid the use of multi-level menus. However, ‘good’ design is often compromised by conflicting requirements, for example, a small, light mobile phone cannot have both a large screen and large buttons with size 20 font labels. As a result, designers will make decisions that result in users being excluded; the challenge is to make sure that they knowingly do so. Knowledge of human physiology, its associated function and resulting capability variation can help designers to determine the costs and corresponding benefits for making products more inclusive. Consideration can also be given to the expected contexts of use of the product in order to increase its accessibility within an appropriate range of environments. Capability prevalence data encourages the setting of appropriate and achievable accessibility targets for each product, rather than generic targets that might encourage systematic exclusion. Inclusive design is an aspiration that should drive designers to better understand the diversity of the users of their products and services, enabling them to delight more customers.

ACKNOWLEDGMENTS The contents of this chapter have drawn on the collective efforts of the Inclusive Design team at the Cambridge Engineering Design Centre, including: Nicholas Caldwell, Carlos Cardoso, Susannah Clarke, Nathan Crilly, Joy Goodman, Mari Huhtala, Patrick Langdon, Tim Lewis, Umesh Persad, Nick Reddall, Sam Waller and Suzanne Williams.

REFERENCES Arditi, A. (2006a). Making text legible [Online article]. URL http://www.lighthouse.org/print_leg.htm Arditi, A. (2006b). Effective color contrast [Online article]. URL http://www.lighthouse.org/color_contrast.htm ASHA. (2006). Type, degree and configuration of hearing loss [Online resource]. American Speech-Language Hearing Association. URL http://www.asha.org/public/hearing/disorders/types.htm

198

Product Experience

Baddeley, A. D. (2004). Your memory: A user’s guide (4th Ed.) London: Carlton Books. Beales, P. H. (1965). Noise, hearing and deafness. London: Michael Joseph. Burke, W. E., Tuttle, W. W., Thompson, C. W., Janney, C. D. and Weber, R. J. (1953). The relation of grip strength and grip strength endurance to age. Journal of Applied Physiology, 15, 628–630. DTI. (2002). Specific anthropometric and strength data for people with dexterity disability [Online report]. London: Consumer and Competition Policy Directorate, Department of Trade and Industry. URL http:// www.berr.gov.uk/files/file21811.pdf Evamy, M. and Roberts, L. (2003). In sight: A guide to design with low vision in mind. Mies, Switzerland: Rotovision. Grundy, E., Ahlburg, D., Ali, M., Breeze, E. and Sloggett, A. (1999). Disability in Great Britain. London: Department of Social Security, Corporate Document Services. Huppert, F. A. (2002). Designing for older users. In: P. J. Clarkson, R. Coleman, S. Keates and C. Lebbon (Eds.) Inclusive design: Design for the whole population, pp. 30–49. London: Springer Verlag. Microsoft. (2003). The market for accessible technology: The wide range of abilities and its impact on computer use [Online report]. Accessible technology market research commissioned by Microsoft, conducted by Forrester Research, Inc. URL http://www.microsoft.com/enable/research/phase1.aspx and http://www. microsoft.com/enable/research/phase2.aspx Miller, G. A. (1956). The magical number seven, plus or minus two: some limits on our capacity for processing information. The Psychological Review, 63, 81–97. Moore, B. C. J. (2003). An introduction to the psychology of hearing (5th Ed.) San Diego, CA: Academic Press. Moore, B. C. J. (2006). Factors affecting speech intelligibility for people with cochlear hearing loss. In: S. Greenberg, and W. A. Ainsworth (Eds.) Listening to speech: An auditory perspective, pp. 273–287. Mahwah, NJ: Erlbaum. Norris, B. and Wilson, J. R. (1995). Childata: The handbook of child measurements and capabilities – data for design safety. London: Department of Trade and Industry. Peebles, L. and Norris, B. (1998). Adultdata: The handbook of adult anthropomorphic and strength measurements – data for design safety. London: Department of Trade and Industry. Philips. (2004). The Philips index: Calibrating the convergence of healthcare, lifestyle and technology. URL http://www.design-council.org.uk/Documents/About design/Design techniques/Inclusive design/Philips Index(us version).pdf RNIB. (2006). Good design guidelines [On-line resource]. URL http://www.rnib.org.uk/xpedio/groups/public/ documents/code/public_rnib003460.hcsp RNID. (2006). RNID information resources [Online resource]. URL http://www.rnid.org. uk/information_resources/productsandequipment/productnews/2006/?ciid⫽286419 Smith, S., Norris, B. and Peebles, L. (2000). Older adultdata: The handbook of measurements and capabilities of the older adult – data for design safety. London: Department of Trade and Industry. UN. (1990). Disability statistics compendium: Series Y. No. 4. New York: United Nations. Vassilief, A. and Dain, S. (1986). Bifocal wearing and VDU operation: A review and graphical analysis. Applied Ergonomics, 17(2), 82–86. Wickens, C. D. and Hollands, J. G. (2000). Engineering psychology and human performance (3rd Ed.) Upper Saddle River, NJ: Prentice-Hall.

7

CONNECTING DESIGN WITH COGNITION AT WORK DAVID WOODS The Ohio State University, OH, USA

AXEL ROESLER University of Washington, Seattle, WA, USA

1. INTRODUCTION In design, we either hobble or support people’s natural ability to express forms of expertise. There is no neutral. (Woods, 2002)

What is the relationship between design and cognition? This question is especially important because new technological capabilities have been appearing at an accelerating rate. First, technologies for more autonomous machines create the image of future machines that do things as substitutes for our own activities, or that even become alternatives to us (Kiesler and Hinds, 2004). Secondly, the technology for connecting people at a distance provides new opportunities to coordinate activity across large distances and to integrate activities that go on at different places or at differing time scales (Hinds and Kiesler, 2002). Thirdly, the technology of collecting, sorting, and focusing data has grown so rapidly that people have available to them hyper-massive fields of data for analyzing situations and generating new or revised plans of action (Woods et al., 2002; Thomas and Cook, 2005). Fourthly, there are new ways to visualize data sets, to fuse feeds from different sensor types, to look at aspects of the world as if we were standing in places that are difficult, expensive, dangerous or impossible to stand in. Sensors connect us and project us into remote situations where we can assume and change different points of observation. All of these vectors of technological change have profound implications for what people do, why they want to do these things, and how they will do these activities in the future. How can designers use the new powers to accelerate human skill acquisition and amplify human performance? Product Experience Copyright © 2008 Elsevier Ltd.

199

200

Product Experience

What is the relationship between cognition and design? One longstanding perspective on this relationship is that people have severe limits on their memory, attention and problem-solving capabilities. People are prone to illusions and biases of many kinds (Gilovich et al., 2002; Kahneman and Tversky, 1996; but see Gigerenzer, 2000; Zsambok and Klein, 1997). In this view, design then would use new technological capabilities to develop prostheses that overcome the inherent weaknesses of people. For example, some people want to design machines that watch over people in action to see how badly their limitations may be impacting their performance (e.g. Schmorrow, 2005). Depending on the machine’s assessment of how people’s mental or physical state is changing, the machine will decide when and how to change the interface, and change user tasks so that the demands remain within people’s limited capabilities. But equally longstanding is a completely different perspective about cognition and the impact of design. People are active, adaptive, goal seeking agents who re-design objects through their activities so that devices work for them (Winograd and Flores, 1986; Flach and Hoffman, 2003; Woods and Hollnagel, 2006). This view starts with people as meaning seeking, explanation generating, attention focusing, learning agents (Bruner, 1990; Weick, 1995). Actually, people, when acting expertly, are a model of competent action, and often the only model of competence available to study when one wants to understand how these processes of cognition work (see Feltovich et al., 1997; Ericsson et al., 2006; Roesler and Woods, in Chapter 8 of this volume). People will learn to express various forms of expertise with experience, and they will shape the artifacts they encounter and interact with as resources to cope with the demands of the situations they face as they seek to meet their goals. Designed artifacts and systems then are resources that stimulate people to adapt to achieve their purposes – triggering adaptive behavior that expands what people can achieve (Woods and Dekker, 2000; Woods et al., 2002). These expansive adaptations are not simply good or bad. Rather they transform systems leading to new roles, new ways of doing things, new reasons to do things. Technology change and new designs are one set of drivers in these processes of organizational transformation and human adaptation. The changes that are triggered result in new levels of performance on some dimensions, new squeezes on performance in other places, new side effects when things that were separate become connected, and new forms of complexity. The basic dynamic is captured in the Law of Stretched Systems (Woods, 2002; Woods and Hollnagel, 2006). The Law of Stretched Systems: every system is continuously stretched to operate at its capacity. People will exploit ‘improvements,’ for example, in the form of new technology, to better achieve goals by pushing the system out to operate near the edge of its new capacity boundaries. The process of adapting to exploit the improvement results in a new intensity, complexity, and tempo of activity.

The following empirical summary captures this dynamic, adaptive process and represents the findings from several researchers (Carroll et al., 1991; Winograd and Flores, 1986). Much of the equipment deployed … was designed to ease the burden on the operator, reduce fatigue, and simplify the tasks involved in operations. Instead, these advances were used to demand more from the operator. Almost without exception, technology did not meet the goal of unencumbering the personnel operating the equipment … Systems often required exceptional human expertise, commitment, and endurance … There is a natural synergy between tactics, technology, and human factors … Effective leaders will exploit every new advance to the limit. As a result, virtually every advance in ergonomics was exploited to ask personnel to do more, do it faster and do it in more complex ways … One very real lesson is that new tactics and technology simply result in altering the pattern of human stress to achieve a new intensity and tempo of operations. (Cordesman and Wagner, 1996, p. 25) [Edited to rephrase domain referents generically].

Connecting design with cognition at work

201

In adaptive cycles, designs act as stimulants in two major ways (Woods and Hollnagel, 2006). First, designs can trigger expansive adaptations by users and stakeholders that exploit capabilities as they seek to achieve their ends. When these expansive adaptations occur, we discover that people have exploited designs in ways typically unforeseen by the designers (Woods and Dekker, 2000). But, second, design change also can introduce impediments that create complexities to be adapted around and overcome – the workarounds often captured in compilations of designs with poor usability (Norman, 1988; Koopman and Hoffman, 2003). Though specific designs often create a mix of both affordances to be exploited and complexities to be worked around, this mix can vary greatly for different people in different roles across an organization. The epigraph to this section captures the basic concept: Fielded designs have real effects on people in various roles, squeezing or enhancing their ability to express expertise. Designs can be examined in terms of how they release or undermine the cognitive skills of people as they carry out roles, handle changing situations, coordinate with others’ activities, and adapt to meet goals. Good designs then release these skills, and poor designs hinder or undermine people’s ability to acquire or demonstrate cognitive skills (Norman, 1993; Bruner, 2003). As Christopher Alexander (1977) noted, design has the special responsibility to connect technological capability to the adaptive power of people as goal-directed agents. Design should understand how skills, expertise, and adaptive capacity develop, how the work of designing can release these basic human processes, and how designs inadvertently can thwart these processes, leaving people to act as though they were cognitively impaired (Woods et al., 1994, Chapter 5). This chapter provides an introduction to some of the basic concepts about design and cognition. Mastery of these concepts will help design processes to enable or release human expertise, or, as Don Norman (1993) put it, how to innovate ‘ things that make us smart’. A note about terminology – artifacts – People who write about cognition and design often use a neutral word for a designed interface, visualization, product, or object. They refer to these as artifacts since the object is just that, an object, until put in use by some person in some role. In use the physical thing or artifact is adapted to provide support. Through use the artifact becomes a cognitive tool, in that it supports or enables people to exhibit skilled and coordinated activity. But many artifacts introduce complexities which add burdens to the people trying to carry out some activity in the world.

2. DESIGN AND COGNITION AT WORK: IMPAIRED OR UNIMPAIRED MICRO-COGNITION To understand how properties of artifacts shape or influence cognition and activity, one has to appreciate some basics about cognition at work. One dominant perspective about cognition focuses on mental processes, skills, and reasoning strategies as things that happen only ‘in the head’. This is a microcognitive perspective (Klein et al., 2003). Micro-cognition refers to basic mental processes, such as working memory, that are believed to act as building blocks for more complex information processing. When cognition occurs only in someone’s head, the processes are invisible. As a result, research methods focus on overcoming this difficulty using various techniques to make internal mental processes explicit and observable to outsiders. For design, micro-cognition is concerned with the impact of a proposed or new artifact on the mental costs of carrying out the details of specific tasks and sub-tasks

202

Product Experience

for their role. For example, designs can act as external memory aids helping people call to mind relevant knowledge, or designs can act as memory barriers forcing people to remember various codes and abbreviations in order to carry out their basic activities through that device. A quite common and relatively simple case occurs with alarm codes: Consider this example from NASA’s mission control dialog over voice loops as the lunar lander descended to the moon during Apollo 11 (Murray and Cox, 1989). ‘1202.’ Astronaut announcing that an alarm buzzer and light had gone off and the code 1202 was indicated on the computer display. ‘What’s a 1202?’ ‘1202, what’s that?’ ‘12 … 1202 alarm.’

The alarms did not direct attention to the trouble, or jump start diagnosis of the source of the anomaly, or direct mission control’s response to the anomaly. Instead, the alarm codes resulted in a memory bottleneck. People, without help from the external displays or representations, had to remember what each code stood for (Woods and Hollnagel, 2006). This case is not just about alarms (or other examples of specific inscrutable codings), nor is it just a historical curiosity describing past designs. Many books continue to collect examples where the design of a device, interaction, or display turned out to increase memory demands on those using the artifact to accomplish something (e.g. Cooper, 1999). The impact of a design is summarized as a specific version of a very general story line – how a new artifact inadvertently created a bottleneck in micro-cognition. Often the demonstration of new memory bottlenecks produced by a design includes pictures of paper notes that users added to electronic systems as memory workarounds (e.g. users adding interpretive keys to overcome a bottleneck introduced by poor design). From a micro-cognitive perspective, poor designs create or exacerbate bottlenecks in elemental cognitive processes. Inadvertently increasing memory demands is just one class of negative consequences of poor design on micro-cognition. Potential bottlenecks in micro-cognition include: • Knowledge bottlenecks, i.e. the design makes it harder for people to call to mind knowledge relevant to the task at hand (Feltovich et al., 1997). • Memory bottlenecks (Byrne and Bovair, 1997). • Attention bottlenecks, i.e. the design makes it harder for people to switch attention between different threads of activity or lines of reasoning, such as interruptions (Czerwinski et al., 1991). • Workload bottlenecks, i.e. the design makes it harder for people to shift or prioritize workload to avoid risk of failure at workload peaks. • Oversimplified mental models, i.e. the design makes it harder for people to develop appropriate and accurate models of how a device or process works (Kieras and Bovair, 1984; Cook et al., 1991). • Intent bottlenecks, i.e. the design makes it harder for people to form coherent intentions to act given uncertainty and multiple goals which can conflict (see Kieras et al., 2001 on the modeling the translation of intention into action and see Cook, 2006, or Woods et al., 1994, Chapter 4 for illustrative cases of managing goal conflicts in health care). A good example is the bottlenecks that arise when poor design of transitions from one display to another as people perform interrelated tasks through a computer interface

Connecting design with cognition at work

203

creates the keyhole effect (Woods and Watts, 1997). The keyhole effect occurs when people can only see a small portion of the relevant information space at one time. The keyhole effect creates a lack of peripheral awareness that can leave people lost in the display set and not know where to look next to find relevant information (an attentional bottleneck). The navigation workload goes up dramatically when there is a keyhole effect which leads to memory and knowledge bottlenecks, as they must call to mind more information and remember more about where important data might be found among the many displays hidden behind the keyhole of the computer screen (Watts-Perotti and Woods, 1999; Doherty and Upton, 2005). Another example is how the principles for designing direct manipulation interfaces have been validated by studies that show such designs eliminate micro-cognitive bottlenecks for memory and workload by connecting intentions to actions when people perform tasks via classic computer interfaces (Kieras et al., 2001). Thinking about the relationship between cognition and design in terms of microcognition helps, but only within some very definite limits. Assessing the micro-cognitive consequences of designs is good for certain aspects of usability evaluation. If one has access to a prototyped or designed device that includes a partial specification of the tasks and sequences, of the procedures in different cases, and of the associated interactions and screen layouts, then one can use models of micro-cognition to detect memory and other bottlenecks. For example, micro-cognitive models have been used to predict long learning times and some types of basic task sequencing errors (Byrne and Bovair, 1997; Gray, 2007). In these cases, the specification of tasks is translated into a computerized cognitive model of micro-cognition (e.g. how working memory load varies as tasks are performed, and how chunking occurs as a mechanism of learning from consistent experience over repeated tasks). When the computer simulation is able to carry out the task, one can use parameters of the underlying cognitive model to detect bottlenecks on working memory or long learning times (Bovair et al., 1990 provides one illustration of these tests). But note there are several ‘catches’ in the relationship of micro-cognition to design. First, for design to use information about how an artifact impacts micro-cognition requires one first has a design (or at least a partial design), since the tests require specification of the details of interaction. In other words, micro-cognition can be used to evaluate or test the impact of designs, but cannot play a direct role in the processes of innovation necessary to generate useful or promising possibilities (Woods, 1998; Lee et al., 2006). Secondly, micro-cognitive tests have difficulty detecting cumulative complexity that builds up as more and more features are added to a product over time in a competitive business environment (Lee et al., 2006). Each feature, looked at alone, appears to pass usability tests and to provide something for a use case or to compete with another product. Inadvertently, however, there is a cumulative complexity effect that comes to dominate any benefits added by each individual feature or option. Pragmatically, businesses have begun to notice the costs of cumulative complexity and the resulting risk of poor product performance in the competitive business landscape (Rust et al., 2006). Several companies and commentators have begun to call for reductions in the number of features on products in a search for a kind of simplicity (Pogue, 2005; Phillips Co., 2005; Wallace, 2006). Thirdly, by using micro-cognitive tests one can discover how a design hobbles cognition and retards learning, but one cannot discover new affordances that may lead to useful adaptations by practitioners that expand their ability to reach goals. These tests can identify how a design creates bottlenecks by forcing cognition to the micro-level so that the specific features that create complexity can be modified or repaired. But the tests do not contribute to the innovation processes that would lead to new concepts for products. When users face complexities such as bottlenecks in micro-cognition, they work around them (e.g. Woods et al., 1994; Cook and Woods, 1996; Cook et al., 2000). It is interesting

204

Product Experience

to note that this is one of the ways that the complexities are noted, not by measuring memory or workload bottlenecks directly, but by observing the workarounds people devise to avoid being hobbled too much by these sorts of complexity. The ways that users tailor their strategies, and the ways that they re-shape artifacts to workaround complexities and fill gaps act as pointers to other adaptive processes. Designs can help foster expansive adaptations. But to do so we need to look beyond bottlenecks in micro-cognition, and examine the functions of macro-cognition that link what people do, as responsible agents carrying out roles, to larger processes of coordinated activity and to larger functions such as anomaly recognition, sense making, and replanning.

3. DESIGN AND COGNITION AT WORK: EXPANDING THE IMPACT OF MACRO-COGNITION A broader perspective on cognition looks at how mental processes support coordinated activities in the world. Most vividly, it has been described as cognition in the wild (Hutchins, 1995) or cognition in natural settings (Klein, 1998), though other labels are joint cognitive systems (Hollnagel and Woods, 2005); situated cognition (Suchman, 1987), or distributed cognition (Hutchins, 1995). This approach considers how macro-cognitive functions such as joint attention, anomaly recognition, replanning, sense making, and others are situated in the context of human goals, desires, activity, and work (Klein et al., 2003; Klein et al., 2006a,b). Macro-cognition moves away from an ‘in the head’ view of the internal processing of an individual because such a perspective loses sight of the active role of people in constructing meaning and exploring their environment. Micro-cognitive approaches to interaction design tend to get lost in a single person in front of a computer screen, missing how people engage in broader activities for larger purposes that move well beyond what appears on computer screens. Putting cognition in the world, or in the situation, emphasizes how people actively explore and find meaning in the world (Gibson, 1979). The ability to look, listen, smell, taste, or feel requires an animal capable of orienting its body so that its eyes, ears, nose, mouth, or hands can be directed toward objects and relevant stimulation from objects. Lack of orientation to the ground or to the medium surrounding one, or to the earth below and the sky above, means inability to direct perceptual exploration in an adequate way. (Reed, 1988; p. 227 on Gibson and perceptual exploration.)

People as agents exist in an environment that is more than what can be seen directly. The actor’s environment includes social structures, shared concepts, conventions, and human-fashioned artifacts. The active exploration process operates just as strongly when the people’s experiences, observations, and actions are mediated by properties of digital media. Heuristically, macro-cognitive functions can be grouped into a few categories of anomaly recognition, sense making, joint activity, and replanning. Anomaly recognition – Noticing when events do not fit (or are beginning to deviate from) the current assessment or expectations (Woods and Hollnagel, 2006, Chapter 8). An anomaly is an event that produces a ‘break’ or interruption in the smooth flow of activity (Winograd and Flores, 1986). Attention, information search, explanation building (sense making), and problem solving then follow as one tries to understand the anomaly, what it means, and how to modify future actions. A simple example is software problems

Connecting design with cognition at work

205

that occur when one is preparing material to meet a critical deadline – the break in flow occurs when the word processing software freezes. The crash produces a break in the activities enabled or empowered by the software; people’s attention and resources have to flow to the software as an object in itself and away from the activity and its purpose. Macro-cognitive studies examine what constitutes an anomaly, how people may overlook or discount anomalous indications or be very sensitive to early signs of trouble (Klein et al., 2005b), and how recognizing an anomaly influences information search, explanation building and explanation revision (Woods, 1995a). Sense making – How people find meaning in the data they see around them (Weick, 1995; Pirolli, 2007). People are explanation machines looking for meaning in what they experience (Bruner, 1990). Klein et al. (2006a,b) have proposed that sense making is built on how people frame and re-frame the events around them, where ‘frame’ refers to the perspective or context used to interpret the data or event in itself. The framing effect in psychology is the much replicated finding that changing the context can change how people see and interpret the very same data or event. In Klein’s model, sense making refers to how people generate and modify the frame or context given the flow of experience (as opposed to the usual study that examines how the interpretation of the flow of experience changes if the larger frame called to mind changes). For Klein et al. (2006b) sense making consists of how people elaborate a starting frame as new information comes in or new events occur, how they begin to question the current frame, and, eventually, how they look for a new frame to understand the events in question – reframing. Sense making, then, is concerned with how people maintain a ‘big picture’ and avoid getting lost in the details surrounding a situation (or to use an aviation analogy, how to stay ‘heads-up’ when there are many individual sub-tasks to be carried out which tend to take you ‘heads-down’ in each), how people can demonstrate what Clausewitz in 1832 on the art of war called coup d’oeil, that is, how people can see at a glance the essential characteristic or leverage point in a complex situation (insight), how people are able to move from mere information gathering to develop syntheses and alternative explanations, how people are able to revise assessments and avoid being stuck in one view, how people are able to develop broadening checks to help avoid premature closure on a weak assessment or explanation. Interestingly, many of the aspects of sense making work best when they do not rely just on an expert individual, but when they are part of collaborative interactions with others. For example, it can be very difficult for a single person to revise an assessment even as new evidence begins to accumulate that the assessment is incorrect (Feltovich et al., 2004). It is surprisingly easy to discount or rationalize away the new evidence and remain stuck in a previous assessment. Revision generally occurs when a fresh point of view enters the situation (Woods and Hollnagel, 2006). Errors in plans are best detected when cross-checks are built into a system by providing review over multiple and diverse perspectives (Patterson et al., 2004). Joint activity – How to coordinate and synchronize activities that are distributed over multiple parties and artifacts (Hutchins, 1995; Sebanz et al., 2006). Moments of micro-cognition are part of, and contribute to, larger flows of interaction and coordination. For example, one often thinks of attention as a characteristic of an individual at some point in time – focused on one out of many activities, cues, or lines of thought. However, people demonstrate the ability to assess where another person’s attention is focused and to follow another’s focus, even at very young ages (Moore and Dunham, 1995). In addition, people signal to others where their attention is focused in order to influence where the other’s attention will focus. This ability for joint attention illustrates

206

Product Experience

how interpersonal cognitive capabilities play critical roles in processes of interaction and coordination across people. Joint attention turns out to be critical when different agents need to modify what they are doing when new events occur (Woods and Hollnagel, 2006). Another part of joint attention is judging the interruptibility of others. Without this capability, miscoordination occurs where interruptions have high costs (Czerwinski et al., 1991; Ho et al., 2004). How technology is used influences joint attention as one kind of macro-cognitive function, for example the ability to listen in on other conversations by being part of a voice loop has been shown to be a critical contributor to very high levels of coordination in space shuttle mission control, even though from a micro-cognitive perspective the voice loop seems very noisy, confusing, and distracting (Patterson et al., 1999). Klein et al. (2005a) describe various other functions that are necessary for joint action over multiple parties and roles. For example, maintaining and repairing common ground – a shared frame of reference about the activities and intentions going on – is a critical aspect of joint activity (Clark and Brennan, 1991; Monk, 2003). Replanning – How to modify a plan in progress when disruptions occur or opportunities arise (Shattuck and Woods, 2000). Plans are always in motion at some time scale, and there are degrees of commitment to planned actions ahead of a continuous rolling horizon. To modify plans in progress, participants need to be able to recognize unexpected events that disrupt the unfolding plans or unexpected events that provide new opportunities to achieve goals behind the plan. It can be quite difficult to recognize when new events affect a plan in progress or to see all of the implications of a new event for the plan in progress. Replanning is an adaptive exercise adjusting or re-formulating a plan when new events introduce impasses or undermine the assumptions behind the plan. It may be even more difficult to recognize when new events provide new opportunities to achieve the goals behind the plan. In replanning situations, people can often under-adapt by trying to continue the plan despite the disruptions (Woods and Shattuck, 2000). Or they can over-adapt – that is, they recognize events challenge the literal plan sequence, but their actions fail to respect many important constraints that had been considered in developing the original plan. New actions are taken to get around the specific disrupting event work around the disruption, but these actions often have undesirable side effects because they fail to conserve other important aspects of the situation, original plan, and goals. This failure mode is referred to as ‘missing the side effects of changes to planned activities following changes in the external situation’. Replanning becomes more difficult as the plan involves more groups or organizations over wider ranges of space and time (Smith et al., 2007). Changing one aspect of a plan in progress can produce a cascade of modifications or even disruptions for other activities in motion by other groups. Assessing the impact and coordinating the adjustments is increasingly difficult as the time and spatial scale of the planned activities increases. Effective replanning depends on being able to see and track the cascades, reverberations, and side effects of new events and possible plan modifications.

4. CONTRASTING MICRO- AND MACRO-COGNITIVE VIEWPOINTS Note the contrast of the list of macro-cognitive functions and the list of micro-cognitive bottlenecks. The macro-cognitive list refers to difficult capabilities that can be enabled or expanded through design of interactions, displays, and devices. The micro-cognitive list refers to troubles that arise when design hobbles people.

Connecting design with cognition at work

207

One can see the contrast in how each view approaches basic topics in workload and attention. Micro-cognition focuses on the limits of an individual to carry out mental operations and keep track of multiple lines of reasoning. Studies examine how people can time share or divide attention over different kinds of processing that could be required to execute various tasks, for example, listening to one conversation while visually controlling a dynamic process (such as occurs in driving a car while talking on a cell phone). The studies can help identify bottlenecks in workload and attention which could lead to poor performance (the cell phone conversation takes mental resources away from driving-related tasks, leading to poor vehicle control and slow responses to changes in the driving situation). On the other hand, macro-cognition is interested in how people adapt to manage their tasks over time to avoid bottlenecks or how they prepare coping strategies to use should they be forced into a bottleneck situation. The starting point is in the world of activity, and the macro-cognitive observer notes that activity ebbs and flows, with periods of lower activity and more self-paced tasks interspersed with busy, high-tempo, externally paced operations where the quality of task performance is more critical. As people learn where and when bottlenecks that threaten performance may occur, they can invoke some basic adaptive strategies (Woods and Hollnagel, 2006). They can develop a strategic response by either shifting some of the activity to other times to lower workload or by recruiting more resources, for example by bringing other people into the situation (for example, critical care physicians carry out these kinds of responses to reduce the chances of being trapped in a workload bottleneck should a patient’s status deteriorate surprisingly; e.g. Cook and Woods, 1996). These adaptive responses depend on the ability to anticipate potential upcoming bottleneck points and the changing pressures on task performance as situations ebb and flow or vary, being more routine or more exceptional. With anticipation one can recruit more resources such as extra staff, special equipment, additional expertise, or additional time, but since this strategy consumes organizational resources there can easily be pressures from above that restrict this strategic response. A part of expertise, then, consists of being able to anticipate potential bottlenecks and high tempo periods and to invest in certain tasks, or conserve certain resources, which reduce the risk of being caught in a bottleneck (and the accompanying risk of prioritizing the wrong parts or doing the whole task poorly) or being unable to cope with an anticipated bottleneck when it does occur. This becomes an illustration of the sense making macro-cognitive function at work and the sub-function of problem detection (Klein et al., 2005b, 2006b). People may not always be good at sense making in situations with the potential for significant workload bottlenecks, but we can investigate the strategies experts use and which strategies work well in some situations to envision new promising possibilities for design. From this short exploration of a single issue we can see the general contrast between the two perspectives on cognition and design. The micro-cognitive perspective provides concepts and tools to detect when design limits performance, and can provide an impetus or justification for (but not innovate in) new design work – it initiates repair. In the case of cell phone distractions, if others have developed new designs, micro-cognitive tests can be carried out to see if attentional bottlenecks are reduced. The macro-cognitive perspective generates directions to explore in the conceptual and ideation phases of design – it participates in innovation. And importantly, macrocognitive results provide criteria that can be used to see if early design work will prove to be a promising direction that merits further investment of energy, talent, time and other design resources (rather than waiting for a prototype to be well specified enough to begin testing). Finally, macro-cognition provides a guide to create concrete scenarios that

208

Product Experience

instantiate challenges for anomaly recognition, sense making, joint activity, and replanning as relevant to the scope of the design project. These scenarios can be used in envisioning and participatory design techniques and as test cases for evaluation. From a macro-cognitive viewpoint, cell phones are just one of many additional tasks that can take driver’s attention away from vehicle control and anticipating upcoming events. It is one example of a general problem created by new technology in the car – how to maintain a ‘heads-up’ or big picture view of the driving situation, despite the need or desire to perform many other ‘heads-down’ or secondary tasks. Heads-up is a label that comes from aviation. It refers literally to looking out the window and figuratively to the sense making processes that allow one to maintain a big picture view of the priorities for the context. Heads-down literally refers to looking at one of the many interfaces or devices used for secondary tasks that require drivers or pilots to look down inside the cockpit or cab; the label has the connotation of getting lost in details. New tools are being devised especially for aviation that help pilots shift attention to the big picture by using tactile signals to indicate the need to shift out of secondary tasks and re-examine the strategic situation (achieving joint attention while judging interruptability; see Ho et al., 2004).

5. MACRO-COGNITION AND EXPANSIVE ADAPTATIONS Often new technological developments are important because they have the potential to change how macro-cognitive functions are carried out. Effective leaders and innovators will discover ways to exploit the new technological power to better achieve goals which will either require or result in changes in macro-cognitive functions. One ongoing trend is new sensor, robotic, and networking technology that makes it possible for macrocognitive functions to be extended over much larger fields of available information and over a larger set of relevant stakeholders. This power – broader connection across groups that are physically separated – is exploited to achieve more, more quickly (e.g. global product engineering networks that speed product development). This new organization enabled by the technology for connectivity at distance expands the scope of sense making and replanning activities over new groups and new relationships. But the tools that support sense making and replanning also change when the scope changes. How will engineers in Bangalore, Seattle, and Moscow collaborate to maintain seeing the big picture as the work encounters obstacles? How will the groups track the reverberations of design changes to avoid missing side effects of change decisions? How will one group realize the new opportunity represented by an advance developed by another group in another part of the world? The process of transformation triggered by new technology changes the demands placed on macro-cognition, changes the affordances for support of macro-cognition, and requires new innovative design efforts to build new support tools. For example, simply copying the devices and displays that support collocated teams and re-using them when teams are distributed has led to lower performance on macrocognitive functions (Hinds and Kiesler, 2002). Macro-cognitive functions are resources people also use to cope with change. To the degree a team or organization is better able to do anomaly recognition, sense making, joint activity, and replanning, they will also be better able to adapt to change by avoiding unnecessary complexity. When people can perform macro-cognition effectively, they are better able to exploit change to achieve more while managing the side effects and tradeoffs produced by those changes.

Connecting design with cognition at work

209

When macro-cognitive functions are effective, what dimensions of adaptive capacity expand? The answer is three-fold; macro-cognitive capability expands the following: 1. The ability to be resilient when situations challenge the boundary conditions of normal practices or when surprising situations occur; macro-cognitive functions support how systems can be resilient when surprises occur and how to manage the unexpected (anomaly recognition, reframing). 2. The scope of potential control, by better coordinating activity over multiple people and groups in different roles; macro-cognitive functions support coordinated and not fragmented activity as multiple parties become more closely tied together (joint activity, replanning). 3. The scope and distances over which people can search for or create meaning and then project their intent from a distance; macro-cognition supports how people can project themselves into, and make sense of, remote settings relative to different purposes (sense making and replanning). Each of these three kinds of adaptive capability is the necessary complement to a vector of technology change. Inevitably, new technological powers are adapted to re-shape what we do, how we do it, and why we do it. Each form of technological power constitutes an infrastructure which demands a new superstructure (of resilience, coordination, and sense making, respectively) that works in parallel. The first dimension of adaptation – resilience – is a critical complementary partner to technological developments that tightly couple together different parts of a larger process or system for hyper-efficiency. A variety of technological powers are tapped in order to squeeze systems of human activity to be more and more optimal. In parallel, these systems become more brittle and can fail dramatically when surprising situations arise – situations which challenge the boundary conditions of textbook operation (Hollnagel et al., 2006). As systems of activity become larger but more tightly coupled, surprising conditions arise in part because a single anomaly will cascade and spread quickly through the system (e.g. modern supply chains in business). Rather than be trapped in workload bottlenecks, experts add resilience to systems by anticipating to avoid workload bottlenecks or by anticipating and preparing coping strategies so that the appropriate priorities are maintained, even if outside events should force workload bottlenecks onto the people responsible. The second dimension of adaptation – coordination – is a critical complementary partner to the technological development of ubiquitous connectivity. The technology of connection by itself does not create coordination, rather it represents underlying powerful infrastructure that is adapted in ways that re-shape how and when we interact with others in what we seek to accomplish. The infrastructure for connectivity, then, challenges our ability to design superstructures that coordinate non-co-located activities and asynchronous activities over greater ranges (Hinds and Kiesler, 2002). Good designs will trigger adaptations that support coordination; poor designs will lead to fragmentation and breakdowns at the boundaries of the different groups or roles (e.g. coordination surprises; Woods and Hollnagel, 2006). The third dimension of adaptation – meaning finding – is a critical complementary partner to the technological developments that combine virtual worlds, new sensor technologies and the ability to shift virtual point of observation with new ‘3-D’ technologies. These powers allow people to virtually stand in places that would be too difficult, expensive, dangerous or impossible to reach physically. How do we make sense of these remote situations? For example, the current video feeds from robots in search and rescue tasks

210

Product Experience

undermine all of the perceptual skills people naturally bring to bear when they are present at a scene (Murphy and Burke, 2005; Woods et al., 2004). The infrastructure of mobile remote sensing (and how to combine these different types of data about the world) demands a new superstructure for sense making as people can ‘forage’ more widely for relevant data and project themselves into distant situations (Pirolli and Card, 1999; Woods et al., 2002). Smith et al. (2007) provide a good example of the connections between infrastructure change and superstructure response that depends on designing affordances to support macro-cognitive functions. Under increasing economic pressure, air traffic control needed to change in order to provide greater flexibility in tailoring and modifying flight plans to match varying congestion points and the changing costs of routing aircraft to destinations. Technological changes made adopting new digital communication infrastructure readily obtainable, and this new capability appeared to allow for much greater flexibility in flight planning and replanning. However, the infrastructure change, while necessary, was insufficient alone without a parallel and complementary change in the superstructure for coordination across multiple groups, especially given changes in weather and failures that could reduce airport capacity. Changing the patterns of coordination over the multiple parts of the national air traffic system was needed so that the system could adapt gracefully and intelligently when disrupting events occurred. Designing the collaborative information and display systems that support distributed work across airline dispatch and strategic and regional air traffic control centers, as situations change, required new design to support macro-cognitive functions (Klein et al., 2005a): • Joint activity required tools for building common ground to help recognize anomalies. • Replanning required synchronized joint activity to consider all of the ramifications of a disrupting event or plan modification. • Shared sense making needed new forms of exchange of information and perspective to recognize when strategic adjustments were necessary.

6. INVENTING THE FUTURE OF COGNITION AT WORK Design changes cognition. By introducing artifacts that mediate information pick up, interaction, and goal-directed action, design influences both micro- and macro-cognition. This is a fundamental finding in cognition – often called the representation effect (Norman, 1993; Zhang and Norman, 1994; Woods, 1995b). Design can impair micro-cognition creating bottlenecks that make it easy to see people as highly limited processors. Adding complexities makes interaction clumsy, makes attention flow to the device itself, rather than the larger goal-directed activity. These impairments add significant costs that limit people and lead to workaround and gap-filling forms of adaptation. Finding bottlenecks and workarounds initiates repair of past design. Alternatively, design produces affordances – support for the difficult macro-cognitive functions of anomaly recognition, sense making, joint activity, and replanning. These advances release the growth of expertise as people learn and share ways to manage the complexities and tradeoffs that challenge all aspects of human activity in our uncertain, changing worlds. These twin effects of designs – clumsiness relative to micro-cognition and affordances relative to macro-cognition – will trigger people, acting in various roles to achieve multiple goals, to adapt. These adaptations may be workarounds to cope with complexities

Connecting design with cognition at work

211

and bottlenecks – gap-filling adaptations. The adaptations may be transformations that exploit new capabilities and that result in new demands and pressures on systems and on human roles in those systems – expansive adaptations. Ironically, these latter act as transformative adaptations (Winograd and Flores, 1986; Woods and Dekker, 2000) which feed back on cognition to create new demands and constraints on carrying out macrocognitive functions, and new costs for micro-cognitive bottlenecks. As has been noted before, in adaptive cycles, today’s design solutions will produce surprises that become tomorrow’s design challenges.

REFERENCES Alexander, C. (1977). A pattern language: buildings, towns, and cities. New York: Oxford University Press. Bovair, S., Kieras, D. E. and Polson, P. G. (1990). The acquisition and performance of text-editing skill: a cognitive complexity analysis. Human-Computer Interaction, 5(1), 1–48. Bruner, J. (1990). Acts of meaning. Cambridge, MA: Harvard University Press. Bruner, J. (2003). The narrative construction of reality. In: M. Mateas and P. Sengers (Eds.) Narrative intelligence. Philadelphia, PA: J. Benjamins. Byrne, M. D. and Bovair, S. (1997). A working memory model of a common procedural error. Cognitive Science, 21, 31–61. Carroll, J. M., Kellogg, W. A. and Rosson, M. B. (1991). The task-artifact cycle. In: J. M. Carroll (Ed.) Designing interaction: Psychology at the human-computer interface. Cambridge: Cambridge University Press. Carroll, J. M. (Ed.) (2003). HCI models, theories, and frameworks: Toward a multidisciplinary science. Boston: Morgan Kaufmann. Clark, H. and S. Brennan, S. (1991). Grounding in communication. In: L. Resnick, J. M. Levine and S. D. Teasley (Eds.) Perspectives on socially shared cognition, pp. 127–149.Washington, DC: American Psychological Association. Clausewitz, C. (1968, original 1832). On war, p. 141. London: Penguin. Cook, R. I., Potter, S., Woods, D. D. and McDonald, J. S. (1991). Evaluating the human engineering of microprocessor controlled operating room devices. Journal of Clinical Monitoring, 7, 217–226. Cook, R. I. and Woods, D. D. (1996). Adapting to new technology in the operating room. Human Factors, 38(4), 593–613. Cook, R. I., Render M. L. and Woods, D. D. (2000). Gaps in the continuity of care and progress on patient safety. British Medical Journal, 320, 791–794. Cook, R. I. (2006). Being bumpable. In: D. D.Woods, and E. Hollnagel. Joint cognitive systems: Patterns in cognitive systems engineering. Boca Raton, FL: Taylor and Francis. Cooper, A. (1999). The inmates are running the asylum: Why high tech products drive us crazy and how to restore the sanity. Indiana: Sams. Cordesman, A. H. and Wagner, A. R. (1996). The lessons of modern war, Vol. 4: The Gulf War. Boulder, CO: Westview Press. Czerwinski, M., Chrisman, S. and Schumacher, B. (1991). The effects of warnings and display similarities on interruptions in multitasking environments. SIGCHI Bulletin, 23(4), 38–39. Doherty, G. and Upton, C. (2005). Designing displays for mobile decision support. L. MacKinnon, O. Bertelsen and N. Bryan-Kinns (Eds.) Proceedings of 19th BCS Conference on Human Computer Interaction (HCI 2005), Vol. 2, pp. 83–88. Ericsson, K. A., Charness, N., Hoffman, R. R. and Feltovitch, P. J. (Eds.) (2006). The Cambridge handbook on expertise and expert performance. Cambridge: Cambridge University Press. Feltovich, P. J., Ford, K. M. and Hoffmann, R. R. (Eds.) (1997). Expertise in context: Human and machine. Cambridge, MA: MIT Press. Feltovich, P. J., Hoffman, R. R., Woods. D. D. and Roesler, A. (2004). Looking at cognitive engineers doing cognitive engineering: Implications of the reductive tendency for the design of complex sociotechnical systems. IEEE Intelligent Systems, May/June, 90–94. Flach, J. and Hoffman, R. R. (2003). The limitations of limitations. IEEE Intelligent Systems,18(1), 94–97. Gibson, J. J. (1979). The ecological approach to visual perception. Boston: Houghton Mifflin. Gigerenzer, G. (2000). Adaptive thinking. New York: Oxford University Press. Gilovich, T., Griffin, D. and Kahneman, D. (Eds.) (2002). Heuristics and biases: The psychology of intuitive judgment. New York: Cambridge University Press.

212

Product Experience

Gray, W. D. (Ed.) (2007). Integrated models of cognitive systems. New York: Oxford University Press. Hinds, P. and Kiesler, S. (Eds.) (2002). Distributed work. Cambridge, MA: MIT Press. Ho, C.-Y., Nikolic. M., Waters, M. and Sarter, N. B. (2004). Not now: Supporting interruption management by indicating the modality and urgency of pending tasks. Human Factors, 46(3), 399–409. Hoffman, R. R., Roesler, A. and Moon, B. M. (2004). What is design in the context of human-centered computing? IEEE Intelligent Systems, 19(4), 89–95. Hollnagel, E. and Woods, D. D. (2005). Joint cognitive systems: foundations of cognitive systems engineering. Boca Raton, FL: Taylor and Francis. Hollnagel, E., Woods, D. D. and Leveson, N. (Eds.) (2006). Resilience engineering: Concepts and precepts. Aldershot: Ashgate. Hutchins, E. (1995). Cognition in the wild. Cambridge, MA: MIT Press. Kahneman, D. and Tversky, A. (1996). On the reality of cognitive illusions: A reply to Gigerenzer’s critique. Psychological Review, 103, 582–591. Kiesler, S. and Hinds, P. (Eds.) (2004). Human-robot interaction: A special double issue of human–computer interaction. Hillsdale, NJ: Lawrence Erlbaum. Kieras, D. E. and Bovair, S. (1984). The role of a mental model in learning to operate a device. Cognitive Science, 8, 255–273. Kieras, D. E., Meyer, D. and Ballas, J. (2001). Towards demystification of direct manipulation: Cognitive modeling charts the gulf of execution. In: M. Beaudouin-Lafon, and R. J. K. Jacob (Eds.) Proceedings of the ACM CHI 2001 Human Factors in Computing Systems Conference, pp. 128–135. Seattle, Washington, USA. Klein, G. A. (1998). Sources of power: How people make decisions. Cambridge, MA: MIT Press. Klein, G. A., Ross, K. G., Moon, B. M., et al. (2003). Macrocognition. IEEE Intelligent Systems, 18(3), 81–85. Klein, G., Feltovich, P., Bradshaw, J. M. and Woods, D. D. (2005a). Common Ground and Coordination in Joint Activity. In: W. Rouse and K. Boff (Ed.) Organizational simulation, pp. 139–178. New York: Wiley. Klein, G., Pliske, R., Crandall, B. and Woods, D. (2005b). Problem detection. Cognition, Technology, and Work, 7(1), 14–28. Klein, G. A., Moon, B. and Hoffman, R. R. (2006a). Making sense of sensemaking 1: Alternative perspectives. IEEE Intelligent Systems, 21(4), 22–25. Klein, G. A., Moon, B. and Hoffman, R. R. (2006b). Making sense of sensemaking 2: A macrocognitive model. IEEE Intelligent Systems, 21(6), 22–26. Koopman, P. and Hoffman, R. R. (2003). Work-arounds, make-work, and Kludges. IEEE Intelligent Systems, 18(6), 70–75. Lee, D.-S., Woods, D. D. and Kidwell, D. (2006). Escaping the design traps of creeping featurism: Introducing a fitness management strategy. Usability Professionals’ Association Annual Conference, Broomfield, Colorado, June 12–16, 2006. Moore, C. and Dunham, P. (Eds.) (1995). Joint attention: Its origins and role in development. Hillsdale, NJ: Lawrence Erlbaum Associates. Monk, A. (2003). Common ground in electronically mediated communication. In: J. M. Carroll (Ed.) HCI models, theories, and frameworks: Toward a multidisciplinary science, pp. 265–290. Boston: Morgan Kaufmann. Murphy, R. R. and Burke, J. L. (2005). Up from the rubble: Lessons learned about HRI from search and rescue. In: Proceedings of the 49th Annual Meeting of the Human Factors and Ergonomics Society, Orlando, FL, Sep. 26–30, 2005. Human Factors and Ergonomics Society. Murray, C. and Cox, C. B. (1989). Apollo, The race to the moon. New York: Simon and Schuster. Neisser, U. (1976). Cognition and reality. San Francisco: W. H. Freeman. Norman, D. (1988). The psychology of everyday things. New York: Doubleday. Norman, D. A. (1993). Things that make us smart. Boston: Addison-Wesley. Patterson, E. S., Watts-Perotti, J. and Woods, D. D. (1999). Voice loops as coordination aids in space shuttle mission control. Computer Supported Cooperative Work, 8(4), 353–371. Patterson, E. S., Cook, R. I., Woods, D. D. and Render, M. L. (2004). Examining the complexity behind a medication error: Generic patterns in communication. IEEE SMC Part A,34(6), 749–756. Philips. (2005). Why simplicity? Available from: http://www.philips.com/about/brand/whysimplicity/ Pirolli, P. and Card, S. K. (1999). Information Foraging. Psychological Review 106(4), 643–675. Pirolli, P. (2007). Information foraging: Adaptive interaction with information. New York: Oxford University Press. Pogue, D. (2005). The cellphone that does everything imaginable, at least sort of. NY Times, May 12, 2005. Reed, E. S. (1988). James J. Gibson and the psychology of perception. New Haven, CT: Yale University Press.

Connecting design with cognition at work

213

Rust, R. T., Thompson, D. V. and Hamilton, R. W. (2006). Defeating feature fatigue. Harvard Business Review, February 2006. Schmorrow, D. D. (Ed.) (2005). Foundations of augmented cognition. Hillsdale, NJ: Lawrence Erlbaum. Sebanz, N., Bekkering, H. and Knoblich, G. (2006). Joint action: Bodies and minds moving together. Trends in Cognitive Sciences, 10, 70–76. Shattuck, L. G. and Woods, D. D. (2000). Communication of intent in military command and control systems. In: C. McCann and R. Pigeau (Eds.) The human in command: exploring the modern military experience, pp. 279–292. New York: Kluwer Academic/Plenum Publishers, Smith, P. J., Spencer, A. L. and Billings, C. E. (2007). Strategies for designing distributed systems: Case studies in the design of an air traffic management system. Cognition, Technology and Work, 9, 39–49. Suchman, L. (1987). Plans and situated action. New York: Cambridge University Press. Thomas, J. J. and Cook, K. A. (2005). Illuminating the path: The research and development agenda for visual analytics. New York: IEEE Computer Society. Wallace, R. (2006). Be smart, be simple. DMI Review Article, 17(2), Spring 2006. Watts-Perotti, J. and Woods, D. D. (1999). How experienced users avoid getting lost in large display networks. International Journal of Human-Computer Interaction, 11(4), 269–299. Weick, K. E. (1995). Sensemaking in organizations. Thousand Oaks, CA: Sage Publications. Winograd, T. and Flores, F. (1986). Understanding computers and cognition. Norwood, NJ: Ablex. Woods, D. D., Johannesen, L., Cook, R. I. and Sarter, N. (1994). Behind human error: Cognitive systems, computers and hindsight. Human Systems Integration Information and Analysis Center, WPAFB, Dayton OH. (available at http://www.hsiiac.org/hsi/products.do?action⫽detailandcode⫽HS-1994-2). Woods, D. D. (1995a). The alarm problem and directed attention in dynamic fault management. Ergonomics, 38(11), 2371–2393. Woods, D. D. (1995b). Towards a theoretical base for representation design in the computer medium: Ecological perception and aiding human cognition. In: J. Flach, P. Hancock, J. Caird, and K. Vicente (Eds.) An ecological approach to human machine systems: A global perspective. Mahwah, NJ: Lawrence Erlbaum. Woods, D. D. and Watts, J. C. (1997). How not to have to navigate through too many displays. In: M. G. Helander, T. K. Landauer and P. Prabhu (Eds.) Handbook of human-computer interaction (2nd Ed.) Amsterdam, The Netherlands: Elsevier Science. Woods, D. D. (1998). Designs are hypotheses about how artifacts shape cognition and collaboration. Ergonomics, 41, 168–173. Woods, D. D. and Dekker, S. W. A. (2000). Anticipating the effects of technological change: A new era of dynamics for human factors. Theoretical Issues in Ergonomic Science, 1(3), 272–282. Woods, D. D. and Shattuck, L. G. (2000). Distant supervision – local action given the potential for surprise. Cognition, Technology and Work, 2, 242–245. Woods, D. D. (2002). Steering the reverberations of technology change on fields of practice: Laws that govern cognitive work. Proceedings of the 24th Annual Meeting of the Cognitive Science Society. [Plenary Address], Atlanta, GA. Woods, D. D., Patterson, E. S. and Roth, E. M. (2002). Can we ever escape from data overload? A cognitive systems diagnosis. Cognition, Technology, and Work, 4(1), 22–36. Woods, D. D. and Hollnagel, E. (2006). Joint cognitive systems: Patterns in cognitive systems engineering. Boca Raton, FL: Taylor and Francis. Zhang, J. and Norman, D. A. (1994). Representations in distributed cognitive tasks. Cognitive Science, 18, 87–122. Zsambok, C. E. and Klein, G. (Eds.) (1997). Naturalistic decision making. Mahwah, NJ: Lawrence Erlbaum.

8

DESIGNING FOR EXPERTISE AXEL ROESLER University of Washington, Seattle, WA, USA

DAVID WOODS The Ohio State University, Columbus, OH, USA

1. INTRODUCTION Designers are often surprised when innovative new systems are introduced to the field. Suddenly, the new products or systems encounter people, and with them practices and operations that have been established before the design of a new product was envisioned. As the new designs are fielded, users are confronted with the new products and wonder how they afford new possibilities and how they can be integrated into current activities. How artifacts are put to use depends in large part on the perspectives, initiatives, and inquiries of these users. The people who will engage in the application of the new products are in many instances more than users; they are practitioners of trained skills. They are professionals who contribute their knowledge to the application of the new products and systems as they utilize these in order to accomplish their goals. Practitioners utilize products and systems with purposes in mind. Do the practitioners consider a product useful? Can they use it to advance their goals? Does it make sense to them in terms of how it influences or changes their activity? As designers, we have learned to study design challenges from the perspectives of the people who will put the new designs to use: How do practitioners know what to do with the new design? What are the intentions and judgments that guide operations? How do practitioners anticipate what will likely happen next as they engage in a particular operation with a new product? How can we – as designers – design artifacts and systems that support practitioners, so that they excel in what they are already good at? How can we integrate technologies by design to support the expertise of others or to help create new forms of expert activity? Understanding how practitioners make decisions and formulate plans for action is a prerequisite for designing systems that facilitate domain expertise in context. We will illustrate in this chapter that expertise is comprised of many factors, and relies on the training and experience of practitioners in their domain. Expertise represents a convergence of Product Experience Copyright © 2008 Elsevier Ltd.

215

216

Product Experience

knowledge, skills, and experience that results in competence – the ability to make appropriate decisions and assessments with regard to the situations at hand. Experts operate in a variety of domains. We find them in serious domains where they act in high-stakes functions as surgeons, pilots, judges, commanders, and high-level decision makers. Many of us are experts in everyday environments, where we converge special understanding and particular skills as technicians, designers, cooks, musicians, wine connoisseurs, and athletes (Hoffman, 1992). Can just anyone who is particularly knowledgeable in a specific domain of human activity be considered an expert by others who depend on the consequences of these decisions or actions? On a day-to-day basis, we use the label ‘expert’ both as an indicator for a high level of proficiency, training, and knowledge that is generally associated with extended practical experience, and as a social construct, where a group of stakeholders identify those who are more expert among them. As more radical innovations in the form of technology-intense systems, softwaredriven products, and digital services continue to transform our daily lives, being expert has acquired a new meaning. Those among us who manage to cope with novelty can learn and adapt to be able to catch up with the continuous series of iterations, updates and improvements in a continuously changing environment. In contrast we often find lab researchers and developers speaking of novice and expert users; in other words, acting as if one could divide the audience of new technological artifacts into those who are in the know, and those who do not know. This has always been a distortion, as expertise is tied to a context and depends on more than just cumulative experience. What is it that experts know and how is it relevant to design? Expertise is rooted in a particularly deep-level understanding of a domain of practice. This understanding utilizes explanations and reasoning strategies that are extracted from a conceptual model of the domain, constructed and continuously consulted, tested, and updated by the domain expert in the course of practicing in the domain by engaging in activities in order to achieve goals. The conceptual model takes the form of an abstract representation (Rappaport, 1997; Heiser and Tversky, 2005). Briefly, a conceptual model is composed of functional and physical relationships in the domain that were identified by the expert and that allow him or her to simulate what is likely to happen next (Klein and Crandall, 1995). The conceptual model assists the expert in finding explanations for observations. The expert has constructed this conceptual model in the course of exploring the domain along various storylines and getting feedback about the different events that follow in various situations. Conceptual models as mental representations serve as structures for reasoning strategies that utilize actual observations in the course of sense-making (Klein et al., 2006). Design influences the acquisition of these mental conceptual models by providing external representations that influence how people see the world and obtain feedback about the world. Design influences the use of these mental conceptual models by changing the mental workload (memory and attentional loads) associated with making sense of situations and planning how to act. These external representations mediate the user’s cognitive work to develop and use conceptual models to make sense of the world accessed through the external devices and displays. Research on expertise is concerned with issues that arise in the course of work with representations in real settings – where people trained to different levels of proficiency work at tasks to different levels of skill. We will, in the following, focus on factors that determine the knowledge and performance of experts – in roles as operators of artifacts and systems, and as stakeholders that face the complexities posed by novel technology. Expert operators and decision-makers form intentions – what needs to be accomplished,

Designing for expertise

217

how to transform these intentions into plans and actions, how to recognize disruptions to these plans, how to cope with complexities and adapt their activities to achieve goals. Note that we replace the common terminology of ‘user’ with the label practitioner. We use the latter term to emphasize that people are not passive recipients of products designed for them or passive rule followers. Au contraire – people actively make things into tools and adapt plans to achieve goals as conditions and environments change. Practitioners will modify unsatisfactory design, devise workarounds, or simply abandon artifacts that do not enable their purposes. Our objective as designers is to support the work of practitioners in context by utilizing innovative technologies that are useful, usable, and easily understood. Only if designers understand the role and nature of expertise can this integration be possible and successful (Ericsson et al., 2006).

2. PERSPECTIVES ON EXPERTISE The designer’s focus is on how technological novelty can affect people. By envisioning how new technologies could take shape in the form of artifacts, systems, and environments, designers mediate technological possibilities with what future practitioners identify as useful, useable, and understandable. As human-centered design, this perspective on design practice dates back to the formation of work studies in human factors engineering and industrial psychology after World War II (Dreyfus, 1955). The initial application of human-centered design techniques in aviation, in industrial manufacturing, and on the battlefield, illustrates that the focus of the practitionercentered view in design has originated in domains where things can go wrong, with serious consequences when designers have solved the wrong problems even when using the right techniques (Woods et al., 1994, Chapter 5). To avoid these design failures, designers have learned to study the prospective users of the proposed design: Who are these practitioners, what are their expectations, and what do they know? As we move into more advanced work domains that encompass considerably more complex technological environments, we deal with highly trained and knowledgeable practitioners who perform as experts in their domain. In order to design from their perspective, we need to understand what identifies their expertise. Plus, technology advances are making more everyday activities considerably complex environments that place increasing demands on people’s expertise – often expertise in working with poor interface designs. Common to all design process models is that they demand that designers understand the domain of human activity before they can be able to integrate useful design innovation into existing contexts (Bayazit, 2004). The question is: If the domain relies on the performance of expert practitioners, do the designers have to become experts in that domain before they can understand the needs of practitioners and thus create appropriate design support? On one hand it simply requires too much time to become a domain expert. But besides the time constraint, it is quite ambitious for a designer to presume that he or she could become an expert surgeon, a pilot, or an intelligence analyst at all. Such achievement could take long training periods, extended apprenticeships, and extensive experience with difficult situations. And expertise at design may have little in common with the kinds of knowledge or skills that experts possess in other domains of human activity. What does it take to become an expert? Is there a fast lane for designers to understand the expertise of practitioners so that they can apply human-centered design techniques that do not fall prey to oversimplifications about what experts in different domains do?

218

Product Experience

Being an expert, on the other hand, can hinder design work where it comes to rethinking existing products. The expert might not be able to see why a routine activity from his or her point of view may pose a challenge to practitioners who perform at different levels of proficiency. Experts may be biased by how things are done today when evaluating diverse alternative strategies and design concepts that would change how things would be done tomorrow. Doing design work in expert domains, designers face the Ethnographer’s Challenge: In order to make interesting observations, they have to be, in part, insiders in the setting they are observing, while remaining, in part, outside the domain in order to have insights about how practice works, how practice fails, and how it could work better given future change. Design observations in the field of practice, where designers watch experts doing cognitive work, relies on being prepared to be surprised in order to distinguish unexpected behaviors that reveal how expertise works and how these experts work (Woods and Hollnagel, 2006). We have encountered examples where user analyses in interaction design contexts mystify rather than clarify domain expertise. It is very difficult to characterize expertise of people in the cost- and time-pressured setting of usability testing environments. It is easy to oversimplify the role of expertise, especially when there are limits on access or high costs to contact with actual practitioners. For example, the need for rapid design input might lead to situations where one might ask software specialists to evaluate the usability of a piece of software designed by other software experts. Dangers like this often lead to the formation of interdisciplinary design teams. But simply juxtaposing specialists from different backgrounds does not make an expert design team. When working in such teams on products for experts, designers in interaction design, industrial design, and visual communication design often feel unprepared to assess complexity in technical domains. They may even feel intimidated by the excessive detail in specifications and the complex nature of underlying processes. Visual designers often state that good solutions need to be simple solutions. Systems engineers, on the other hand, know that only complexity can cope with complexity; that often, there cannot be a simple solution to a complex problem – the problem is complex because an appropriate response can take many forms. Confronting the open possibilities and the critical uncertainties, designers draw, build, and explore where in contrast engineers tend to simulate, analyze, and plot results. Both model, but in completely different ways. Design builds tangible models people can play off as they envision future possibilities; engineers model how processes work fundamentally in order to run simulations that would predict exactly how things would work under postulated different, that is, future, conditions. For the former, design is discovering new possibilities (and this requires generating a wide range of novel approaches and combinations); for the latter, design is applying a set of constraints on what is a solution, to narrow in on a solution provided directly once one has the results from the right series of analyses or simulations. Working only beside each other these two perspectives are both needed and are both incomplete. Simplicity cannot be designed without confronting complexity (e.g. there are limits to using techniques like out-of-the-box testing where the designer assumes the role of the outsider, the non-specialist, and the novice that walks up to an artifact and is assumed to operate it without prior instruction). Analysis alone cannot guide how people adapt new capabilities into real tools, and in this process transform roles, processes, decisions, difficulties and even what is expertise. One challenge that transcends traditional design and engineering perspectives is that new technology changes what it means to be a practitioner. The future practitioner will not be the same in terms of knowledge, skill, expertise, and collaborative interplay as technology changes (Mark et al., 2005). To understand and guide the process of how

Designing for expertise

219

new technology and new designs change what people need to learn and do to be experts, various other experts need to collaborate and transcend their individual areas of knowledge. This collaboration over different kinds of experts is targeted at enhancing the expertise in action in particular domains of human activity. But the targets of the design have a role and a stake in terms of how new products affect their abilities, their roles, and their goals. Participatory design is more than generating collaboration across different specialists in design. The practitioner also participates in design. Practitioners are the experts at applying information to make decisions for actions that will affect states and behaviors in their environment – they are the experts in the use of artifacts. Designers, on the other hand, are experts in the design decisions that steer the development of products to support the decision making of practitioners (Woods, 1995). Ultimately, in participatory design processes practitioners, technologists, and innovators co-mingle, each with critical but incomplete roles as designer (Roesler, Woods and Feil, 2005). The questions that surround the nature and acquisition of the expertise are central to understanding design, both the mixture of experts who contribute to product designs and the expertise in context as people use new devices to do things in the world. The following sections are explorations into the nature of expertise: Observations of expertise in action, how we discover or elicit the knowledge organizations behind expert performance, and how the design of new products affects the ability to acquire and demonstrate expertise.

2.1. A history of the study of expertise Practitioners base their reasoning strategies on creating and interpreting artifacts as a form of external knowledge representation. Designers support practitioners in choosing, structuring, and using these representations rather than relying totally on basic individual mental capabilities without any external tools. Depending on their expertise in a task, area, or field, practitioners will engage in activities differently from what the designers expect them to do, and may or may not arrive at anticipated outcomes used to justify the investment in the development project. This makes expertise an important issue to consider in design, where patterns of interaction between practitioners, artifacts, and the world vary with different degrees and kinds of expertise. The study of expertise has a long history that reaches back to performance evaluations conducted by experimental psychologists in World War I-era studies of skilled machinists (Hoffman, 1992). The methods applied in these observations date back into the 1890s and focus basically on the timing of observable actions in the course of given tasks. Binet’s work at the beginning of the twentieth century set virtually all of the standards that would be applied by post-1950s studies of expertise. Typically the researcher would identify a task – for example, the selection, lifting, and alignment of a part in a machining task – and identify a typical begin and end state of the task. The researcher then would time a worker engaging in the process necessary to complete the task from its beginning to end. The hypothesis was typically that the more experienced worker could accomplish the task faster or engage in more tasks in parallel, given time pressure. In favoring the duration and number of tasks as central measures, these studies fall short in capturing, discussing, and exploring the nature of the individual differences among subjects, the situation that has preceded the test observation, and the context (such as motivations and social structure). These broader factors were initially considered distortions, and by transporting the work task into a controlled laboratory, the researcher could exclude (control) the

220

Product Experience

unwanted contextual circumstances in order to obtain objective results. It soon became apparent that time measures of performance provided a quite limited view into the nature of expertise. Measuring performance times in isolated sub-tasks soon evolved into observing related activities, and in the course of this shift in focus, it emerged that many aspects of expert decision-making reside in the context, social structure, and demands posed by different situations. In addition, it was evident that individual differences and training played a central role in the trade of experts. Cognitive psychologists began to study cognition and reasoning in field settings. One of the first subjects studied was the expertise of chess masters (De Groot, 1946). The reason why chess was a good subject for the study of expertise was that metrics for the determination of chess proficiency were already in place (Ericsson and Smith, 1991). Furthermore, the chess board and configuration of chess pieces serve as both physical representation of states on the board and abstract representation in the chess player’s conceptual model of possible – and best – next moves that would affect the constellations of chess pieces on the board. Simon and Chase’s classic study on expert chess players (1973) found that chess masters can quickly detect meaningful patterns or configurations of chess pieces and can remember a very large number of configurations if they form meaningful patterns relative to different situations that confront the experienced chess player. To demonstrate some of De Groot’s observations about the capabilities of experts, Simon and Chase tested players’ ability to recognize and memorize chess positions. They presented players either with game-meaningful configurations of pieces or with random configurations of pieces. They found that expert players were capable of memorizing, and hence planning ahead, a greater number of proper configurations than chess novices. When presented with random chess piece configurations, however, the chess experts did not perform significantly better than the novices. According to Simon and Chase, chess masters differ from players at lower degrees of proficiency in that they can recall a much higher number of game-meaningful configurations. This allows them to plan further ahead, given that a significant constellation forms the starting point. Compared to other chess players, the experts can assess the consequences of many different options for moves from a current constellation in parallel. Simon and Chase have shown that chess masters apply pattern recognition strategies in their assessment of what to do next. They can do this not because of superior memory for elements (each piece), but because they have a different knowledge organization based on relationships that connect pieces into configurations, and connect configurations into tactical and strategic sequences of action and counter-action. The paradigm shift from studying behavior to studying cognition in American psychology was stimulated and grew along with the introduction and growth of computerized systems into more and more realms of human activity. The development of computer applications quickly grew through attempts to reproduce human decision-making or to create computers that could make decisions on their own (called expert systems). Optimistically, these researchers thought they would be able to replace human cognition with computerized expert systems. While this agenda has repeatedly failed (and been repeated and failed several more times), the goal of autonomous computers required intense study of human expertise. Deployment of these expert computer systems quickly led to situations where the computerized expert and the human expert were both present in the actual decision situation. The developers expected that the computer expert system would improve the decisions of the human practitioner – hence they would be decision support systems. However, providing decision support and building team work between human and machine agents turned out to be a much more sophisticated topic, and the early computer expert systems turned out to be very poor team players. It was found that sophisticated decision support

Designing for expertise

221

systems often had ended up being underutilized or unused by experts during their firsthand reasoning (Woods, and Roth Bennett, 1990; Shanteau, 1992; Hollnagel and Woods, 2005; Woods and Hollnagel, 2006). The reasons for this are many – computer expert systems are inherently brittle, non-cooperative, opaque, disembodied, and designed based on erroneous models of expert knowledge (Hoffman, Feltovitch, and Ford, 1997). The original attempt to understand expertise in production measures of isolated subtasks had failed; the much later attempt to put expertise into a computer system and use that system to replace people as an isolated component also failed. A third line of work focused on studying the reasoning strategies behind expert decisions, usually with a goal of developing new training systems to speed up the acquisition of expertise as people moved from novice to journeyman to expert. Simon and Chase (1973) have shown that valuable insight on expert decision making can be collected by techniques such as think-aloud verbal protocols. Hoffman (1987) reminds us that using this and other knowledge elicitation techniques to study expert reasoning is time consuming and leaves out important factors. Variations on think-aloud protocols have been developed, such as critical incident analysis and cognitive task analysis. Today there is a wide and rich range of techniques that can be used to gain insight about the basis of expertise in particular domains of human activity (Christoffersen et al., 2007). By using a broad array of techniques researchers were able to see how social constructs and organizational factors affected expertise (Wright and Bolger, 1992; Hoffman, et al., 1997; Klein, 1997). One of the new lines of inquiry on expertise is called naturalistic decision making (NDM). NDM is the way people use their experience to make decisions in field settings, or, in a more comprehensive description, the study of NDM asks ‘how experienced people, working as individuals or groups in dynamic, uncertain, and often fast-paced environments, identify and assess their situation, make decisions and take actions whose consequences are meaningful to them and to the larger organization in which they operate’ (Zsambok and Klein, 1997; Klein, 1998). Naturalistic decision making research underlines the role of context, social organization, and domain-specific knowledge in the decision making of experts. NDM directs decision making in high-stake domains such as fire fighting, military command and control, mission control, aviation, and planning. One of the key results is that expertise uses external artifacts to support the processes that contribute to expert performance – expertise is not all in the head, rather it is distributed over a person and the artifacts they use, and over the other agents they interact with as they carry out activities, avoid failure, cope with complexity, and adapt to disruptions and change. For an illustration on how found characteristics in the environment become cognitive artifacts by being associated with cognitive tasks, see Edwin Hutchins’ observations of Micronesian and Western navigation (Hutchins, 1995a). Another key finding is the role of context (Woods et al., 2002). Hoffman, Feltovitch, and Ford (1997) describe context as composed of: • The analyst’s purposes and goals. • The analyst’s views, assumptions, and theories. • The analyst’s methods. What data people regard as meaningful depends on these kind of contextual factors. Experts are very sensitive to context shifts and how these shifts change what knowledge and what mental conceptual models are relevant to the task at hand (Feltovich et al., 1997). Context cannot be neglected in the study of decision making, expertise, and innovation. This has led to the macro-cognitive view on cognitive work (Klein et al., 2003), and with it an extended focus in the study of expertise – one that incorporates context,

222

Product Experience

the nature of the tasks, and the social and organizational factors that identify experts. Macro-cognition addresses the study of decision making and expertise in context, and looks at activities that are unfolding in organizations, situated in the environment of real tasks (Klein et al., 2003; and see Woods and Roesler, in Chapter 7 of this volume). Expertise is situated in real work settings where practitioners have experience with variations on routine performance and can access a rich set of cues that help them anticipate what could happen next. Expert reasoning is more than a specialized algorithm which could be captured in a computer system; expertise adapts reasoning strategies, plans, and actions in the face of changing conditions in the environment in order to achieve goals (Woods and Hollnagel, 2006). Overall, expertise does not reside simply in a single individual, but is situated in the relationships between practitioner, task, environment, and the organizational characteristics that direct and constrain practice in the domain.

2.2. What is expertise? A first assessment of expertise considers a person’s levels of proficiency at a task. Research then asks the question how do people move from one level of proficiency to another. A basic answer is that more knowledge about the task and associated context supports greater expertise. But this view of expertise as more extensive and better organized knowledge does not take into account the role of hands-on practice. Donald Schön’s (1983) work on the intersections of knowledge and practice extends the expertise-as-knowledge position. He focuses on the role of reflection-in-action that occurs when knowledge is put into practice. In his view, expertise is acquired by combining exposure to task-relevant reasoning with task-specific knowledge. Proficiency in domains such as law, medicine, and design is the integration and mutual refinement of knowledge, skill, and experience that can only occur through doing. The traditional assumption in psychology is that expertise is a phenomenon that can be accounted for in terms of the quantity, quality, and organization of the domainspecific knowledge of experts. In this view, ‘expert knowledge is based on deep reflection, allowing experts to do well because they know more, know better, and know in a more useable way’ (Hoffman et al., 1997). Hoffman et al. continue to question this position by reminding us that knowledge and meaning can be seen as extra-personal, and is located in the community rather than just inside a person. This leads us back to context, to addressing the differences between different domains, where knowledge is constructed and maintained in different communities. Expertise, from this angle, can be read as a repository of strategies for applying knowledge to context by taking into account the nature of the task and the social aspects surrounding the task. This view leads us to three characteristics of expertise: 1. First, expertise is domain-specific. 2. Secondly, experts adapt to changes in their environment. 3. Thirdly, experts rarely act as isolated individuals. Summarizing various studies of expertise in the agricultural, medical, weather forecasting, and law sectors, Shanteau (1992) identifies five psychological strategies that help experts in making decisions: 1. Expert judgment shows a willingness to make continuous adjustments in initial decisions. 2. Experts get help from others in order to make better decisions when coping with uncertainty. They can identify the experts in sub-domains.

223

Designing for expertise

3. Experts make use of both formal and informal external decision aids. 4. Even though experts may make small errors in judgment, they avoid making large mistakes. ‘The focus is not on being exactly right, but on avoiding making bad decisions’. To do this, experts first conduct a broad estimate of a problem situation and then conduct a more careful analysis of components in the situation that were identified as more critical in the first pass. 5. Experts decompose complex situations into manageable chunks, and can then re-construct the larger situation from its individual aspects. These five strategies indicate that experts have access to a quite refined conceptual model of their domain of expertise to identify functional relationships, dependencies, and likely outcomes. This model of the domain includes scenarios of what not to do (Minsky, 1997). The expert’s conceptual model also includes an understanding of relationships of their domain with sub-domains that are part of the challenges ahead. Experts know who the experts are in these sub-domains, and when to consult them. Expertise forms a decision-making capacity in the neighborhood of other forces. Hoffman, Feltovitch and Ford illustrate that expertise entails context as a set of other factors that surround and direct expertise with their TEMPEST model (Figure 8.1). The tempest model has evolved from previous models as new research results have accumulated. The original base is James Jenkins’ (1979) tetrahedron model that captured results from learning research. Honeck and Temple (1992) adapted a version to incorporate new results on the cognitive underpinnings of expertise. Building on these models, the TEMPEST model maps expertise as the tetrahedral set of relationships between the expert’s strategies, goals, background experience, and materials used as decision support.

the ‘wind’

materials

the ‘kite’

driving forces

data, tools used in the familiar tasks

the expert

societal needs and expectations, personal motives

cognition, perception

strategies used in familiar tasks

goals of the familiar tasks

background experience knowledge and skills

the ‘tail’

stabilizing forces selection criteria, training methods, professional standards

the ‘line’

controlling forces lawfulness in the world, practical constraints, accountability mechanisms and performance expectations

FIGURE 8.1 TEMPEST model of expertise in context (modified from Hoffman, Feltovitch and Ford, 1997).

224

Product Experience

As a diagram, the two-dimensional tetrahedron resembles a kite, and represents the expert. In taking the kite analogy further, Hoffman et al. illustrate the context of expertise in showing that it takes more to fly a kite than the kite alone. • The wind carries the kite – in the TEMPEST model, the wind resembles the driving forces on expertise, rooted in societal needs and expectations and personal motives. • The kite is steered by the line – controlling forces on expertise that are institutionalized by lawfulness in the world, practical constraints, accountability mechanisms and performance expectations. • The kite is stabilized in the wind by its tail – the stabilizing forces on expertise are selection criteria, training methods, and professional standards for experts. • Tail and line make sure the kite stays stable in the wind and can be controlled. • Together with the wind as the driving force, they form the context of expertise, and if we look at the labels of the contextual forces, we encounter many of the sociological parameters that identify expertise. Note how it takes the interactions between all factors in the diagram to constitute, identify, and steer expertise, and how the four subsystems (kite, wind, line, and tail) can partially compensate for deficiencies in other aspects. If the kite, for example becomes unstable due to a tail that is too short, more reactive steering with the line is required to keep the kite in the air. The TEMPEST model illustrates expertise as a macro-cognitive system of interrelated factors of organizational structure, environmental conditions, decision making, knowledge, and planning ahead. Characteristic of this macro-cognitive representation of expertise is that experts balance multiple factors in parallel, and that all factors represent diversity and contrasting forces that mutually reinforce another. Growing expertise to act effectively requires orchestrating the interplay over this set of factors and driving forces.

2.3. Who are the experts? Expertise is both a social construct and the context-specific application and adaptation of knowledge. Studies on the quality of expert decisions have shown that experts do not generally make better decisions than do non-experts (Shanteau, 1992). This contrasts with the general perception that experts can be identified by their outstanding performance, and hints at a social dimension behind the identification of experts. In virtually every organization, certain people are known as the best ones to consult for specific types of problems. Though experts do not always make the best decisions in a given situation, they have been shown to avoid making decisions that they expect to have particularly negative consequences (Shanteau, 1992). Expert decision-making seems to be driven by appropriateness, not necessarily by correctness. Non-experts in similar situations are prone to making wrong decisions because, often, they cannot assess the repercussions of alternative decisions. Experts will likely avoid making wrong decisions, but will opt for a solution with the least trade-off if they do not see the possibility for an optimal solution. Since experts have a conceptual model of their domain, they know what not to do (how to avoid errors of over-control of complex dynamic processes). Societal institutions play a role in the accreditation of experts. While some experts have been identified by peers for their remarkable performance in a number of situations, other specialists are accredited through licensing systems such as university degrees and academic titles. Certificates document that an expert has access to an agreed-upon body

Designing for expertise

225

of knowledge and that he or she has demonstrated performance in a number of practice situations, as well as passed examinations of expected performance. Knowing who the experts are in a domain can be a challenge. While domain practitioners seem to have no problem in identifying the experts among themselves, outsiders have difficulties identifying who the experts are in a particular domain (Sternberg, 1997). Utilizing insights from the study of expertise, let us refer to a list of nine characteristics that identify an expert (Shanteau, 1992). Experts have access to a vast body of up-to-date content knowledge; as a basis for judgment, this deep level understanding of their domain is characterized by: • Highly developed perceptual/attention abilities: Experts can extract information in their domain that non-experts miss (Fiore and Hoffman, 2007). • Case-based reasoning in context: Experts have a sense of what is relevant when they are making decisions. Where novices often get sidetracked by irrelevant data, experts base their decisions on less information, but ensure that their judgment rests on relevant information. • The ability to simplify complex problems: Experts have developed strategies to decompose complicated situations in their domain into smaller chunks. While novices attempt decomposition, they often fall into making oversimplifications. This is not true for experts, because they know that their domain is composed of several sub-domains. The ability to decompose complicated context is guided by a conceptual model of the domain that the expert has constructed (Feltovich et al., 1997). • The ability to communicate expertise: Experts have no problem communicating information in the areas where they are experts. They assume that their assessments should be preferred over those of others. This outspoken expertise goes hand-in-hand with the social construction of expertise; that confidence in claiming oneself as an expert can accredit a person as expert. Let us keep in mind that this statement is put to the test until our expert unites all nine characteristics in this list. • The ability to handle diversity more successfully than non-experts: Experts continue to make effective decisions even when things are not going well. This characteristic is again supported by the expert’s comprehensive conceptual model of his or her domain. This model does not only include best case scenarios, but also includes forks in the road where things can go wrong, together with scenarios of how not-so-optimal sequences will play out and how to recover from them. • The ability to identify and adapt to exceptions: Experts are not likely to walk down ‘garden paths’. They can identify when an expected situation deviates from the expectation. • Knowing how and when to adapt their decision strategies to changing task conditions: Experts detect when changes in the context of the task demand adapted reasoning strategies. This is possible because experts have access to many alternative response strategies and understand that different situations cannot all be approached with the same strategy. • Strong self-confidence: Experts trust their decisions. • A strong sense of responsibility: Experts can assess how situations will likely play out. They understand their role as expert decision-makers and are aware of the consequences of a wrong decision. In any case, experts know that they must make decisions because they can control the situation at hand. Naturally, these characteristics of expertise make experts difficult people to work with. They are confident in their expertise, and they can assess the expertise of others in their domain. This makes expertise an interesting topic from the training and learning

226

Product Experience

perspective, where domain experts help novices in eliciting domain knowledge to eventually become experts themselves.

2.4. How expertise is acquired? Dreyfus and Dreyfus (1986) have identified five stages in the acquisition of expertise, where a novice starts at a beginner level, moves continually through levels of competence and proficiency, and finally arrives at the expert level. The five stages illustrate increasing levels of proficiency in performance, as well as the role of context-free rules, the realization of patterns in context, and the hierarchical organization of task-relevant clues for decision making or intuitive response. Stage 1: Novice – The beginner engages in an instruction sequence where he or she learns context-free features of the domain that can be recognized without experience in the domain. Beginners in chess, for example, learn a numeric value for each type of piece regardless of its playing value. They also learn a rule that is associated with the piece, as well as the positioning constraints on the piece. Beginners are slow performers during this stage because they need to remember and recall the rules that guide their actions. Stage 2: Advanced beginner – At this stage, the beginner is pointed to specific features that provide situational aspects of the activity. He or she begins to see patterns in the context of the task that provide additional information in support of the rules. Stage 3: Competent – At the competent level, the student continues to identify more task-relevant features in the context of the activity. This eventually results in a large number of patterns that he or she recognizes, culminating in the sheer overload of potentially task-relevant data. To cope with complexity, the student begins to structure patterns and strategies by installing hierarchical reasoning sequences. The competent performer can address a broad spectrum of situations. Performance is slow and depends on the student’s seeking new rules and reasoning strategies in order to devise a plan or take a perspective. Stage 4: Proficient – In the proficient stage, intuitive engagement in the activities continually replaces reasoned responses. Instead of being rule-based, the reasoning strategies of the performer are driven by adapting decision support structures to the current context of a situation. The performer simply knows what to do instead of deciding in the course of a calculative response. Stage 5: Expert – While the proficient performer sees what needs to be done and decides to do it, the expert performer not only knows what needs to be done, but also knows how to achieve the desired state. This allows the immediate response to a clearly identified situation. This immediate and appropriate response is characteristic of expertise. Dreyfus and Dreyfus’ five stages in acquiring expertise focus on the student’s journey. If the student were required to rely solely on his or her individual experience, the extent of lessons learned would depend on the variety of failures that confronted the student, so that limitations would have to be overcome by practice. Ericsson and Charness (1997) point out the importance of deliberate practice in the acquisition of expertise. In deliberative practice, students work with a teacher or coach whose job is to identify individual weaknesses and to design training programs that repeatedly confront students with the weak aspects of their performance. The continuous confrontation engages the student in developing response strategies for overcoming individual weakness. In their study of the training and development of expertise in outstanding musical performers and athletes, Ericsson and Charness identify four developmental stages, from the novice stage to that of eminent expert. Particularly interesting in their observations

Designing for expertise

227

is that they look at an extended temporal frame of expert development, encompassing the three to five decades spanning acquisition of knowledge, skill, and endurance, in the development from the beginner stage towards the highest level of expertise. The extremely long training period required to achieve the mastery of musical performance or world class competition level in a sport is necessary to provide for the mutual adaptation between the performer’s or athlete’s physical capabilities, and the advancement of expertise and endurance during practice. In the course of their classic study on chess, Simon and Chase (1973) have identified that it typically takes ten years of intensive deliberative practice to arrive at an expert level of performance in chess. Studies in other domains (Ericsson, Krampe and TeschRömer, 1993) have confirmed this estimate. Ericsson and Charness’ four phases in the course of becoming an eminent expert pass through the following proficiency levels: 1. In Phase one, musical performers and athletes, at a young age, engage in playful activities in the domain. This phase ends with the identification of potential in the performer, indicating that he or she can move the acquired skills onto the next level. 2. Phase two is an extended preparation. It ends with the commitment of engaging in practice full-time. Usually, this phase entails the consultation of specialized trainers or coaches, and often even requires moving into another geographic region where the required training with an advanced teacher or coach is available. 3. In Phase three, the performer makes a commitment to full-time engagement in the training and practice of the skill. This phase ends either with the performer being able to make a living as a professional, or with the complete termination of the full-time activity (Bloom, 1985). At this stage, the performer is practicing or training with an expert teacher or coach. During the first three stages, students integrate the knowledge and skill that master teachers and coaches can convey (Ericsson and Charness, 1997). 4. To achieve the highest level of expertise – the fourth phase – performers must attain a level of performance where they make an original contribution to the domain of their expertise. In this phase, they reach a level of proficiency that goes beyond that of their teachers. At the eminent level, where the highest level of expertise is innovation, empirical studies of these original contributions have been sparse. This is attributed to the fact that the moment at which the contribution will occur is hard to predict. Ideas, conditions, and decisions that have led to major breakthroughs in thinking and practice can be traced by retrospective process tracing methods (Gruber, 1981; Wallace and Gruber, 1989).

3. INNOVATION AND THE EMINENT LEVEL OF EXPERTISE Ericsson and Charness’ (1997) observation that innovation is an indicator of an eminent state of expertise draws parallels to the difficulties that grip the concept of innovation in design circles. As practitioners, designers tend to know what innovation is, yet a comprehensive description of innovation in design is still largely missing. Like important discoveries by scientists and creative leaps by artists and performers, innovation in design can often only be studied in retrospect, since it is difficult to experimentally set up the conditions that would allow an innovative breakthrough to be observed as it unfolds. Such staging techniques would be more than desirable to designers. Perhaps design methods could be considered as a direction in the quest to demystify innovation; a methodic design approach enables designers to observe, provide, and steer environments, and design conditions so that they provide fertile grounds for innovations to ‘happen’.

228

Product Experience

The design process is the structuring of resource pools that enables designers to envision promising design responses for future iterations of a product or system, based on observations of the performance of current design solutions for that product or system in the field. These observations eventually result in a domain understanding for the designer. In its sequence of formation, this development of insight shows many parallels to the sequence of acquiring expert understanding. Although the sequence of learning from observations plays out in a much shorter timeframe and is conducted from an emic perspective of the designer (from a view inside the culture of design), the insight won by designers makes it possible for them to create novel interventions relative to the target domain of the expert practitioner. As a consequence of the shorter time frame spent in the expert domain, the insight won by the designers is much less comprehensive than the deep level domain understanding acquired by an expert over a span of many years. One possible explanation of these limitations is that the designers are limited to observations and often cannot practice the observed operations of use, since they are not sufficiently prepared. Knowing this restriction, the challenge for designers is in knowing how observations in the field can be linked to the design of concepts that respond to identified problems or needs in the domain of practice. In the design process, the acquisition of expertise can be viewed as an iterative cycle of feedback, concept generation, and selection. In the cycle, observations are intertwined with modeling of a conceptual representation of the field of practice under observation; these conceptual representations are providing the grounds for creating products that can be fielded, and then in turn can be observed. Figures 8.2A and B illustrate a cyclical framework that can be used to chart this design process, the de:cycle model (Roesler and Woods, 2005). Design activity, to be able to respond to the challenges provided by the call for novelty balanced with pressures on realizablity, coordinates expertise from three perspectives – observe, explore, create. These provide linkage points between three broad roles that draw on pools of experience – pools of research patterns, technological possibilities, and experience in the field. Each of these roles Provides their knowledge, processes, and artifacts with respect to the future under design: Practitioner – how they adapt to complexity; innovator – how they envision what would be useful; and technologist – how they bring the anticipated change into the world of practice. The three roles intersect, and design activity within them is conducted in parallel – which provides for numerous interactions across the center of the de:cycle. In a typical design project, design activity begins with observations in the field of practice where practitioners identify limitations of previously fielded objects. In the de:cycle graphic, this point of departure is located at the 4 o’clock position: Design intent is represented in clockwise movements; design effects are projected in counterclockwise rotation. Moving counterclockwise for exposition purposes, in the observation arc (northeast section), design researchers plan studies, collect data records from protocols, carry out ognitive task analyses, and undertake other process tracing techniques. Functional syntheses based on the data and observations lead to abstract models that capture patterns, provide explanations, and lead to proposed alternative directions. For example, researchers may design observations to reveal how practitioners have adapted products to work for them in ways different from the designer’s model (design intent is represented in clockwise movements). Different explanations and characterizations of the basis for these adaptations then become seeds for exploring possible new design directions. In the exploration arc (northwest section), there is movement from the models, patterns, and data of research to design seeds that capture promising concepts hypothesized

229

Designing for expertise

innovator: pools of research patterns

e

ze

e l)

at e r and © 2004 by Axel Roesle

technologist: pools of technological (A) possibilities

i ll r ,a ds oo id W Dav

im p lem

en

t)

bj ec

cr e

gh ts res erv ed.

o b se r v e

de

e lo r

od

xp

a li

(m

n (h y p o t h e s sig is )

co n ce p t u

o t(

practitioner: pools of experience in the field

abstract model explanations abstracted patterns leverage points

developing concepts

developing scenarios

affordances representation techniques

c on c e p t u

stories of use

protocols data records

listening to stories suggestions for improvement

ea

te

,

final design model

im p le m

t en

ec

cr

t)

unsatisfactory product

bj

is ) (h y p o t h e s s ig n

de

model of the ideal design

(

o b s e r ve

building prototypes

e

e l)

animock

lo r

od

hypotheses of what is useful

p ex

m

design seeds

observations of adaptations

aliz e

(o

established field product in field revision 2 in field revision 1

design documentation

detailing for production

(B)

adjusted final release

reproduction model tooling test realese plans and documentation

deployment into the field

FIGURE 8.2 (A) The de:cycle – The three roles in design and their respective interests and expertise. Counterclockwise moves represent design synthesis, as objects are created. Clockwise moves represent the analysis of design that identifies the origins, basis, and expectations associated with design decisions. (B) Handover marks and design process products make design activity tangible and allow designers to plan for deadlines and coordinate with other stakeholders.

to lead to useful change in the target field of practice. We refer to this arc in the de:cycle as the ‘Northwest Passage’ as the processes of innovation and ideation are not supported systematically, but rather have tended to be supported or marked by design artifacts personal to specific designers or teams (at least as compared to the more formal artifacts produced by design in the realization process, e.g. those that mark software development). In the Northwest Passage, ideation mediates between research insights about the situation under design (rotating clockwise) and technological capacity for change (rotating counter-clockwise) as related to judgments about desired improvements from the point of view of practitioners (cutting across the center of the cycle from left to right).

230

Product Experience

The techniques of those proficient in innovation generate insights to generate promising concepts that can be turned into prototype demonstrations that support commitment of resources for realizing these concepts in fieldable forms. In the implementation arc (southern section), technologists choose from a pool of available technologies those that are appropriate for the realization of selected design concepts. The appropriateness of applied technologies is evaluated against factors of functionality, reliability, feasibility, safety, and economic feasibility – and how necessary modifications will reflect on the original design program. Figure 8.2B shows how handovers across the roles serve as demarcations and how materials produced in the process of design, such as scenarios, sketches, mock-ups, and prototypes, make these processes tangible, helping designers reflect on their progress and share ideas with other stakeholders in the design process. The processes of design, focusing on the need to create innovations, can be seen as a type of expertise that demonstrates the same regularities as have been summarized in the preceding sections of the chapter. As designers become acquainted with a situation under design they incrementally attain a deeper level understanding of the design domain. A deepening understanding facilitates the quest for innovation as in the definition of the eminent level of expertise. Figure 8.3 shows the cycle of expertise acquisition and domain change, overlaid on the de:cycle framework. In design, one moves from novice, relative to the target field of practice, to greater expertise, relative to how that domain could function. The design process meets the criteria for eminent expertise with the generation of innovative concepts, though these will turn out to have surprising effects when fielded which will release a new round of adaptation through use. In the observation arc, designers who had entered the design situation as novices have moved through stages of advanced beginner, competence, and proficiency. Starting at the stage of proficiency, they have begun building a conceptual representation around the acquired knowledge of the domain as research findings. While the evolving

proficient extracting What’s interesting (reasoning backward)

identifying what’s promising (reasoning forward) expert

el)

directing observations

sle

)

Wo ods, all ri ghts reser

e Ro el Ax by 006 ©2

im plem

competent

ac t

te

ra nd

Da vid

applied

cr ea

ved.

observe

conceptual c

ice

e

ali ze

od

xp

e lor

(m

testing hypotheses

(hypot he sis sign ) de

concept u

pr t( en

eminent expert detailling responses novice

implementing change advanced beginner

FIGURE 8.3 The cyclical model of the acquisition of expertise is overlaid on the de:cycle, a cyclical model of the design process. The overlay illustrates how innovation as a result of the build up to eminent stage of expertise produces change that leads to effects that require the growth of new expertise.

Designing for expertise

231

conceptual model provides the proficient designer with explanations for what has been observed, it allows the capacity to make predictions by simulating the outcomes of proposed changes that would be part of new products. Because the conceptual model is a representation of the relationship, dependencies, and causal sequences identified during observations in the field, it allows experts to reason forward by manipulating variables in the model. Moving counter-clockwise is a process of projecting ahead toward future states. These predictions form hypotheses about how investments in additional design work will produce change in the target field of human activity. These hypotheses are tentative and open to revision as evidence comes in about the likely impact of the proposed changes. One difficulty is that getting this evidence usually requires movement into activities that make design ideas tangible in the form of sketches, mock-ups, and refined prototypes. These types of design representations allow designers to share their observations with the domain practitioners and design clients. Taking this as a starting point, let us return to the nature of expertise in a more general frame of reference. Practice is characterized by an experimental component, where the expert follows through with promising actions based on decisions that were supported by simulations performed in the framework of relationships of the conceptual model. Activities are subsequently evaluated in the context of the real situation before being adapted to unanticipated conditions and modified in a later iteration. Any expert is designing response strategies based on a hard-won conceptual understanding of the situation at hand. The eminent expert knows that, in order to identify the boundaries of their understanding, they have to manipulate the situation at hand. This alteration of existing conditions is done responsibly, as he or she can identify manipulations that would lead to unmanageable outcomes. The eminent expert also knows the scope and possible results of unmanageable states, and will avoid losing control of the situation. The stage of eminent expertise is characterized by innovation that had originated in the course of many previous observations of such experiments. These previous experiments do not need to be repeated for the current decision task. The past outcome of the original experiment or practice is sufficient for making the current decision. The eminent expert recycles past insight, accessing and applying patterns from a knowledge base of past storylines. The expert has learned to calibrate projections, based on appropriate interventions that conform to the constraint space identified as boundary conditions. Yet the eminent expert carries out an intervention to provide more insight into the situation at hand and does not rely completely on his or her knowledge of the relevant constraint space. This is the process of envisioning innovation. Eminent expertise is characterized by the continual refinement of a conceptual model of the expert’s domain. This occurs by directing observations with the intent to actively manipulate conditions in the domain in order to arrive at innovative new outcomes from given and known situations. The eminent expert does this repeatedly, as he or she purposefully steers activity away from routine. The eminent expert knows about the limits of routines and knows how to apply slight changes to the repeated activity in order to scope new boundaries. As he or she initiates new conditions in the known field, the expert will eventually find him/herself in new terrain, hence being less than an eminent expert in the newly created domain area. In a cyclical configuration of gathering understanding and generating novelty by applying the new understanding, the knowledge base of the domain expert grows with each iteration of change. As with designers who first need to learn about the domain under design before being able to initiate change, the eminent expert explores the boundaries of the current

232

Product Experience

conceptual models by purposefully manipulating conditions in his or her domain. Designing – the act of manipulating representations – is necessary in order to arrive at refined representations. Eminent expertise thus is not the mastery of analytic deductions in the found domain, but instead incorporates steering the domain by changing it. The original contribution of the eminent expert is a necessity in this process of gathering advanced understanding to correlate actual results with anticipated outcomes. This process is the engagement of the expert in reflective practice (Schön, 1983) where strategies for activities, as well as the activities themselves, are subject to adaptations in the face of a changing environment. Domain experts have gone through many iterations of reflection upon action, initiated by changes in the conditions within the domain environment and driven by results and problems that arise from these changes. These iterations are necessary to reinforce mutual dependencies between conceptual assessment and practical application on the basis of a series of interventions into ongoing activities. The eminent expert’s conceptual model of the domain has reached a level where it can represent the patterns of change that steer the adaptation of response strategies to the changing context of the situation.

4. THE IMPLICATIONS OF DIFFERENCES IN USER EXPERTISE FOR PRODUCT DESIGN Expert decision making is driven by context. From the perspective of the designer who faces the challenges of designing artifacts as decision support, this has many implications. First, the nature of expertise is rooted in the social structure of the domain and is driven by the changing nature of the domain context as captured in the TEMPEST model of expertise as a kite in the wind (Figure 8.1). Expertise is generally instantiated as an interpersonal collaboration where designers can identify experts in certain stages of the activities, while expert roles may change during different stages of tasks. This same pattern is relevant to the design process, where the lead expert roles shift between practitioners, innovators and technologists (Roesler et al., 2005). In order to conduct the design process, all three pools of expertise are necessary in parallel. No person can play all roles; designers cannot completely put themselves into the seat of practitioners, practitioners cannot take over design work, and technologists cannot replace designers – nor can they take on the role of practitioner. Problems arise in the design process when one pool of expertise attempts to stand in for others on the basis of inferences from a distance. Key to an understanding of expertise in design is coordinating the convergence of these different types of expertise as a particular product development cycle matures and restarts. There are many examples where the handover of role-specific expertise is central. Consider, for example, the changing meanings of artifacts in the course of cognitive work with the model of a door key. While a technologist is primarily concerned with designing a secure key that can reliably lock and unlock a door, the innovator might question the concept of a key to lock a door at all. The question might not stop at this point. Why a door at all, what is it for? What are other possibilities for the concept of a key? From the perspective of a practitioner, the existence of the key becomes meaningful only as specific episodes arise during the execution of tasks. On the way to the site of a task, the practitioner encounters a locked door. Unlocking the door is not the central task for the practitioner – getting to the other side of the door where the task is located is the central task.

Designing for expertise

233

Let’s put this apparently simple issue in a context: During an emergency at a power plant a pump that should provide a critical function to maintain safe conditions fails to start. The person on the scene is inexperienced and is unable to start the pump; the experienced person is in a different location in the plant (the control room) and attempts to get to the failed pump to perform the restart. However, during an emergency for security reasons physical access from one part of the plant to another is restricted. The key expert is blocked from reaching the critical location; if the pump is not restarted an extreme and untried safety procedure must be started. Can the expert find a way through the blocked doors in time to restart the critical pump in time? (He did in the actual nuclear power plant emergency in 1985 that is the basis for this example.) This unique event captures a general tradeoff between security and collaboration – blocking access when it is undesirable or blocking some groups access versus enabling collaboration and connection to enhance performance – that is a central dilemma in current debates about computer security in a world of web-based interactions at a distance. One pool of expertise is about different ways to block access; another pool of expertise is about ways to enable more collaboration at a distance. Without coordinating the two, cycles of mal-adaptation develop where for security reasons access is hobbled or blocked and people who can take advantage of the connection devise workarounds to communicate. In reverse, we can deduct levels of expertise from artifact characteristics in the work environments of experts. The expert work of aircraft pilots is mapped into the complex structure of the aviation cockpit (which in turn is a representation of the complexities underlying the task of flying a plane – a representation of the joint system of different people and devices that underlies flying; see Hutchins 1995b). There is a link between expertise and the complexity of technology applications. Alexander (1964) has argued that, for emerging specializations, the emergence of experts is the consequence of technological advances and part of the formation of new domains. Understanding the context-dependent nature of expertise is a prerequisite for designing artifacts that support decision making – that allow people to become familiar with patterns of use and that support identifying constraints necessary to detect and resolve irregularities in the course of action. From the cognitive systems design perspective, the level of expertise of domain practitioners directs how much of the guidance for use must be built into the artifacts of a system, since the training of experts may serve as a substitute in the absence of self-explanatory clues in expert environments. As an example, compare a pilot with the everyday driver of an automobile. For the former, flight training typically takes a few hundred hours, with frequent deliberate refresher sessions in the flight simulator. As a consequence, the pilot has an extensive conceptual representation of the plane as a system of artifacts, and of the plane’s boundaries of performance in the face of environmental condition changes and other events. This expertise acts as a source of resilience to overcome design deficiencies such as mode confusions so that they rarely turn into plane crashes (but see the mode confusion issue in, for example, Woods et al., 1994). Now consider the car example. Driver training is much less elaborate and consists of a rough understanding of traffic rules, performance of the vehicle, and control of the vehicle. The domain of driving a car in roadway traffic is less complex than that of flying an airplane in air traffic. Compared to the domain of flying a plane, the driving domain consists of factors such as slower speed and simpler degree of technology in the groundbound vehicle, greater visibility of traffic and obstacles, and a greater proximity margin of other traffic participants. In contrast, the airplane pilot must think further ahead, due to the higher speed of the plane, and must cope with the complex technology of a vehicle that cannot stop

234

Product Experience

without losing aerodynamic lift. In addition, the pilot is a participant in a larger collaborative organization consisting of cockpit crew, airport crew, and air traffic controller – all in different roles – where limited flight, take-off, and landing pathways must be shared with other planes within very tight schedules. Add to this complex organizational conglomerate automated on-board systems, replanning due to delays, invisible flight routes, and eventual technical problems, and it’s easy to assess why pilots are experts. We have learned how difficult the assessment of typical practitioner expertise is. Expert users can draw resources for interpretation of gaps from their individual knowledge, while novice users rely on knowledge in the world that is provided by the design of the task environment (Norman, 1986). While providing an extended range of performance, advanced technological applications introduce new forms of complexity, and with them new possible failures. During the past thirty years, advanced computerized systems in high-stakes domains have demonstrated the new realities of increased sophistication in the form of new possibilities. Unfortunately, they also have demonstrated the new realities of increased complexities that have contributed to new types of incidents and accidents in aviation, process control, monitoring, medicine, mission control, and military applications (Woods and Hollnagel, 2006). Due to the public consequences of failures, extensive studies about the nature of expertise in the monitoring and design of technology have generated invaluable insight into the relationship between domain knowledge, training, and design. In all instances, highly trained experts are paired with advanced computerized technology in joint systems (Hollnagel and Woods, 2005). In facing the challenges of designing for these expert domains, it is essential that designers realize that their objective is not to design for users, but to support experts by design. Learning how to reinforce innovation is a form of cognitive work as reflectionin-action. Design requires understanding, exploration, and experimentation; during ideation phases in design projects, this cognitive work leads to novel artifacts and new approaches to doing things, always coupled with the initiation of change. New technological artifacts will change the nature of practice (Winograd and Flores, 1986). Novel artifacts will require practitioners to engage in new types of operations that need to be learned. In the long run, innovation will form new types of expertise, while experts will make contributions that lead to new inventions (Woods and Dekker, 2000). Designers are confronted with the challenge of understanding the expertise in the domain under design; this does not require that they become experts in the domain themselves. Studies of expertise in various fields of practice, conducted during the past 50 years – among them studies on the expertise of designers by Cross (2003) – illustrate the nature of expertise as a characteristic feature in cognitive work that is tightly coupled with the nature of the field of practice. The most recent view on expertise is that it exists in the context of a specific domain and encompasses the social structure of the field, individual differences, training, and personal experience. Another important finding is that expertise is subject to continuous change. As technologies advance, they transform the field of practice and confront experts with the necessity of keeping their knowledge updated. In the face of these characteristics of expertise, it seems quite illusionary that designers could become experts in a field they encounter during a design assignment. However, designers may find it helpful to become familiar with the patterns of knowledge elicitation and work practice that instantiate expertise at work (see the earlier section on ‘What is expertise’). By designing support for expert reasoning strategies in collaboration with practitioners and technology developers, designers make their contribution to

Designing for expertise

235

expert work. Key to the design of systems that support practitioners in being experts is a design process model that allows the convergence of the perspectives of practitioners, innovators and technologists.

5. SUMMARY AND CONCLUSION After excursions through the nature, study of, and various views on expertise; through the implications of expertise research on design; and through the strategies for a design process that addresses the introduction of novelty into the domains of experts, we can now summarize expertise in a few sentences to illustrate how aspects of expert practice, decision making, and innovation-building affect designers. Our view on expertise enables designers to design with the objective of supporting the work of experts and acquiring expertise. Expertise as a focus for design presents an interesting though somewhat moving target for design activity, since expertise is driven by the context and social structure of the domain. One of the consequences of this is that expertise continuously re-invents itself in a sequence of adaptations to a changing environment. At the same time, events that drive these changes are, in part, outcomes of expert practice. We summarize expertise as follows: • Experts have observed, learned and practiced in their domain for a long time. Their expertise is domain-specific, driven by the context of the domain, and instantiated in the social structure of the domain. • Expertise is both knowledge and skill in the adaptation of understanding to observations in the context of a situation. • Expert practitioners capture and model their domain understanding and approach strategies in representations that they refine while eliciting the knowledge required to practice as experts in their domain. • Expertise changes; the agreed-upon standards of outstanding performance are subject to continuous improvement. Improvements may include refined reasoning strategies, better decision support representations, or more advanced adaptations to the changing nature of practice. In addition, social standards on how to assess expertise may change. • Expertise is a form of contextual understanding that guides the formation of strategies in making sense of observations in a given context. • Expertise is limited by the perspective of the expert. Complex decision-making tasks, such as designing, require reasoning strategies that utilize methods of coordination between the different types of expertise of design stakeholders. It is not only advisable to consult other perspectives, it is necessary (Klein, 1997). • At the eminent of level of expertise, an expert generates an original contribution to the standards of the domain. As innovators, experts shape the future of succeeding experts in their field. In the course of learning to understand their domain, the eminent experts change their domain, and the changed domain requires and forms a new generation of experts, often trained by the eminent experts, and a product of the new conditions. This eminent level shows parallels to the key challenge in design work: The generation and creation of novelty, and the responsibilities to steer the changed conditions. Expertise, in conclusion, evolves as a consequence of adaptations in a changing environment. In parallel, expertise forms the basis from where practitioners and designers envision and implement changes in the form of novelty into the domain. Novel technologies, in turn, will initiate new forms of expertise. Innovation and expertise are mutually interrelated.

236

Product Experience

Assessing the expertise of future practitioners is both a requirement and an outcome of design support. Expertise does not reside in artifacts or people, but is the result of cognitive work by practitioners who build a correspondence across artifacts with their goals, their environment, the perspectives of others in their field of practice, and the stakeholders who depend on the outcomes produced.

REFERENCES Alexander, C. (1964). Notes on the synthesis of form. Cambridge, MA: Harvard University Press. Bayazit, N. (2004). Investigating design: A review of forty years of design research. Design Issues, 20(1), 16–29. Bloom, B. S. (Ed.) (1985). Developing talent in young people. New York: Ballentine Books. Christoffersen, K., Woods, D. D. and Blike, G. T. (2007). Discovering the events expert practitioners extract from dynamic data streams: The mUMP technique. Cognition, Technology, and Work, 9, 81–98. Cross, N. (2003) The expertise of exceptional designers. In: N. Cross and E. Edmonds (Eds.) (2003). Expertise in design, creativity and cognition, pp. 23–35. Sydney: University of Technology. De Groot, A. D. (1978). Thought and choice in chess. The Hague, Netherlands: Mouton. (Original work published 1946.) Dreyfus, H. L. and Dreyfus, S. E. (1986). Mind over machine. New York: Free Press. Dreyfus, H. L. (1997). Intuitive, deliberative, and calculative models of expert performance. In: C. E. Zsambok and G. Klein (Eds.) Naturalistic decision making. Mahwah, NJ: Lawrence Erlbaum. Dreyfus, H. (1955, 1967). Designing for people. New York: Allworth Press. Dunbar, K. (1999). How scientists build models: In vivo science as a window on the scientific mind. In: L. Magnani, N. Nersessian and P. Thagard. Model-based reasoning in scientific discovery, pp. 89–98. New York: Plenum Press. Ericsson, K. A. and Smith, J. (Eds.) (1991). Toward a general theory of expertise. Cambridge: Cambridge University Press. Ericsson, K. A. and Charness, N. (1997). Cognitive and developmental factors in expert performance. In: P. J. Feltovitch, K. M. Ford, R. R Hoffmann (Eds.) Expertise in context: Human and machine. Cambridge, MA: MIT Press. Ericsson, K. A., Krampe, R. T. and Tesch-Römer, C. (1993). The role of deliberate practice in the acquisition of expert performance. Psychological Review, 100, 363–400. Ericsson, K. A., Charness, N., Hoffman, R. R. and Feltovitch, P. J. (Eds.) (2006). The Cambridge handbook on expertise and expert performance. Cambridge: Cambridge University Press. Feltovitch, P. J., Ford, K. M. and Hoffmann, R. R. (Eds.) (1997). Expertise in context: Human and machine. Cambridge, MA: MIT Press. Fiore, S. F. and Hoffman, R. R. (2007). Perceptual (Re)learning: A leverage point for human-centered computing. IEEE Intelligent Systems, 22(3), 79–83. Gruber, H. E. (1981). Darwin on man: A psychological study of scientific creativity (2nd Ed.) Chicago: Chicago University Press. Heiser, J. and Tversky, B. (2005). Characterizing diagrams produced by individuals and dyads. In: T. Barkowsky (Ed.) (2005). Spatial cognition: Reasoning, action, interaction, pp. 214–223. Berlin: Springer-Verlag. Hoffman, R. R. (1987). The problem of extracting the knowledge of experts from the perspective of experimental psychology. The AI Magazine, 8(2), 53–66. Hoffman, R. R. (Ed.) (1992). The psychology of expertise: Cognitive research and empirical AI. New York: Springer Verlag. Hoffman, R. R., Feltovitch, P. J. and Ford, K. M. (1997). A general conceptual framework for conceiving of expertise and expert systems. In: P. J. Feltovitch, K. M. Ford, R. R. Hoffmann (Eds.) Expertise in context: Human and machine. Cambridge, MA: MIT Press. Hollnagel, E. and Woods, D. D. (2005). Joint cognitive systems: Foundations of cognitive systems engineering. Boca Raton, FL: Taylor and Francis. Honeck, R. P. and Temple, J. G. (1992). Metaphor, experiences, and a PEST. Metaphor and symbolic activity, 7, 237–252. Hutchins, E. (1995a). Cognition in the wild. Cambridge, MA: MIT Press. Hutchins, E. (1995b). How a cockpit remembers its speeds. Cognitive Science, 19, 265–288.

Designing for expertise

237

Jenkins, J. J. (1979). Four points to remember: A tetrahedral model of memory experiments. In: L. S. Cermak and F. I. M. Craik (Eds.), Levels of processing in human memory. Hillsdale, NJ: Lawrence Erlbaum, 429–446. Klein, G. A. and Crandall, B. W. (1995). The role of mental simulation in naturalistic decision making. In: P. Hancock, J. Flach, J. Caird and K. Vicente (Eds.) Local applications of the ecological approach to human-machine systems, Vol. 2, pp. 324–358. Mahwah, NJ: Lawrence Erlbaum Associates. Klein, G. (1997). An overview of naturalistic decision making applications. In: C. E. Zsambok and G. Klein (Eds.) Naturalistic decision making. Mahwah, NJ: Lawrence Erlbaum. Klein, G. (1998). Sources of power: How people make decisions. Cambridge, MA: MIT Press. Klein, G. A., Ross, K. G., Moon, B. M., et al. (2003). Macrocognition. IEEE Intelligent Systems, 18(3), 81–85. Klein, G. A., Moon, B. and Hoffman, R. R. (2006). Making sense of sensemaking. 1: Alternative perspectives. IEEE Intelligent Systems, 21(4), 22–25. Mark, G., Gonzalez, V. M. and Harris, J. (2005). No task left behind? Examining the nature of fragmented work. Proceedings of CHI 2005, pp. 321–330, ACM, Portland OR, April 2–7, 2005. Minsky, M. (1997). Negative expertise. In: P. J. Feltovitch, K. M Ford, R. R. Hoffmann (Eds.) Expertise in context: Human and machine. Cambridge, MA: MIT Press. Norman, D. (1986). The design of everyday things. New York: Doubleday. Rappaport, A. T. (1997). Context, cognition, and the future of intelligent infostructures. In: P. J. Feltovitch, K. M. Ford, R. R. Hoffmann (Eds.) Expertise in context: Human and machine. Cambridge, MA: MIT Press. Roesler, A. and Woods, D. D. (2005). Inventing the future of cognitive work, Proceedings of the 6th international conference of the European Academy of Design. University of the Arts, Bremen, Germany, March 29–31 2005. Roesler, A., Woods, D. D. and Feil, M. (2005). Inventing the future of cognitive work: Proceedings of the 6th international conference of the European Academy of Design. University of the Arts, Bremen, Germany, March 29–31 2005. Schön, D. (1983). The reflective practitioner. New York: Basic Books. Shanteau, J. (1992). The psychology of experts: An alternative view. In: G. Wright and F. Bolger (Eds.) Expertise and decision support. London: Plenum Press. Simon, H. A. and Chase, W. G. (1973). Skill in chess. American Scientist, 61, 394–403. Sternberg, R. J. (1997). Cognitive conceptions of expertise. In: P. J. Feltovitch, K. M. Ford, R. R. Hoffmann (Eds.) Expertise in context: Human and machine. Cambridge, MA: MIT Press. Wallace, D. B. and Gruber, H. E. (Eds.) (1989). Creative people at work. New York: Oxford University Press. Winograd, T. and Flores, F. (1986). Understanding computers and cognition. Norwood, NJ: Ablex Publishing Corporation. Woods, D. D., Roth, E. M. and Bennett, K. B. (1990). Explorations in joint human-machine cognitive systems. In: S. Robertson, W. Zachary and J. Black (Eds). Cognition, computing and cooperation. Norwood, NJ: Ablex Publishing. Woods, D. D., Johannesen, L., Cook, R. I. and Sarter, N. (1994). Behind human error: Cognitive systems, computers and hindsight. Human Systems Integration Information and Analysis Center, WPAFB, Dayton OH. (available at http://www.hsiiac.org/hsi/products.do?action⫽detailandcode⫽HS-1994-2). Woods, D. D. (1995). Towards a theoretical base for representation design in the computer medium: Ecological perception and aiding human cognition. In: J. Flach, P. Hancock, J. Caird and K. Vicente (Eds.) An ecological approach to human machine systems I: A global perspective. Mahwah, NJ: Lawrence Erlbaum. Woods, D. D. and Dekker, S. W. A. (2000). Anticipating the effects of technological change: A new era of dynamics for human factors. Theoretical Issues in Ergonomic Science, 1(3), 272–282. Woods, D. D., Tinapple, D., Roesler, A. and Feil, M. (2002). Studying cognitive work in context: Facilitating insight at the intersection of people, technology and work. Cognitive Systems Engineering Laboratory, Institute for Ergonomics, The Ohio State University Columbus OH at url: http://csel.eng.ohio-state. edu/woodscta Woods, D. D. and Hollnagel, E. (2006). Joint cognitive systems: Patterns in cognitive systems engineering. Boca Raton, FL: Taylor and Francis. Wright, G. and Bolger, F. (Eds.) (1992). Expertise and decision support. London: Plenum Press. Zsambok, C. E. and Klein, G. (Eds.) (1997). Naturalistic decision making. Mahwah, NJ: Lawrence Erlbaum.

13 PRODUCT EXPRESSION: BRIDGING THE GAP BETWEEN THE SYMBOLIC AND THE CONCRETE THOMAS J.L.VAN ROMPAY Faculty of Behavioural Sciences, Enschede, The Netherlands

1. INTRODUCTION Browsing through a home improvement magazine, overhearing a conversation about the latest design trend, spotting an ad for the new Peugeot; it is all too clear that those qualities we refer to as symbolic (i.e. qualities that are not ‘literally’ part of product appearance such as distant, inviting and firm) take on increased importance in today’s market. In design and marketing literature, this trend is often traced to the fact that many (or most) products appearing on the market are very similar in function and price, making it hard or simply irrelevant for people to differentiate products on such primary criteria (Postrel, 2003; Veryzer, 1995). As a result, products are ever more evaluated in terms of their experiential benefits: ‘What does this product say about me?’, ‘Is this really me?’, etc. Indeed, a recent study confirmed such speculations by revealing that consumers’ preferences for product appearance are by and large motivated by symbolic (product) meaning (Creusen and Schoormans, 2005). Although we all perceive objects as expressive of symbolic meanings, it does not follow that we understand what it is that makes us, for instance, experience a certain product as adventurous, modest or trustworthy. On the contrary, attempts to relate symbolic meanings to formal product features often reach no further than global attributes such as size or color. And even with respect to stereotypical, seemingly obvious relations between form and expression, one may find it is not that easy to explain why, for instance, organic or rounded form features are generally perceived as secure or emotional. In other words, although perceiving what products express comes most natural, Product Experience Copyright © 2008 Elsevier Ltd.

333

334

Product Experience

accounting for a product’s expression is less straightforward. Arguably, it is precisely this ‘accounting for’ that is a prerequisite for successful design. In this chapter, different perspectives on (product) expression or symbolic meaning (synonyms used alternately) will be presented. Although different in scope and focus, they all share the, sometimes hidden, assumption that an object’s perceived expression results from the interaction between object and perceiver. However, in accounting for symbolic meanings, researchers usually stress the role of either the perceiver or the object perceived (Crozier, 1994; Crozier and Chapman, 1984; Dewey, 1934). In order to provide a rough classification of relevant studies in expression (varying widely in focus and scope) they will be presented along these lines. Another reason for doing so is to clarify the rationale underlying a third approach, discussed consecutively. This approach underlines the necessity to simultaneously and explicitly take into account the mutual contributions of both perceiver and object perceived. In the first part of this chapter we will discuss studies in which the object is at the center of investigation. In this type of study, relations between formal object features and symbolic meanings are explored. In the second part of this chapter we will review research that places primary emphasis on the role of the perceiver in the coming about of an object’s expression. According to this approach, symbolic product meanings can be traced to cognitive or biologically centered processes. Following our discussion of object- and individual-centered perspectives, we will present a third perspective, originating in the writings of the philosophers John Dewey (1934) and Merleau-Ponty (1962). Both stress the fact that (symbolic) meaning can only be studied in the light of interactions between individual and environment. By consequence, explicit and equal emphasis should be placed on the interdependent contributions of both object and perceiver. Since object-perceiver interactions are constrained by the peculiarities of the human body, both view meaning as essentially embodied. In the last two decades, this approach has ‘resurfaced’ in cognitive psychology (e.g. Johnson, 1987; Lakoff and Johnson, 1999), and has proven to be very successful in accounting for symbolic meanings of all kinds. In line with recent studies in cognitive psychology, we will argue that symbolic meanings exemplified by products are rooted in our own embodied experiences arising from interactions with the environment. Before launching our review it should be stressed that it is not our intention to attest to the merits of one approach or perspective over another. In the course of this chapter, it will become clear that the studies under review vary widely with respect to the type of symbolic meanings addressed, withholding us from any such endeavors. Nonetheless, the relative lack of studies addressing product expression from an interactional perspective justifies, we feel, the more elaborate discussion of this approach presented in this chapter.

2. THE EXPRESSIVE OBJECT At the far end of the spectrum, theories stressing the object perceived in order to account for its expression embrace the assumption that our world and its objects are intrinsically meaningful. As such, expression resides in the object and can be described without taking into account the role of the perceiver, except to say that he or she can be more or less receptive or equipped to perceive an object’s expression. The artist Wassily Kandinsky, for instance, considered visual elements such as point, line, and plane (the basic elements of abstract painting) to be fully ‘alive’, and therefore expressive in and of themselves. In discussing the expressive characteristics of the basic plane, Kandinsky (1926) argues that its four lines, i.e. the two verticals and the two horizontals, are bound up with different

Product expression: Bridging the gap between the symbolic and the concrete

335

‘sounds’. Whereas the ‘above’ and, although to a lesser extent, the ‘left’ are expressive of lightness, emancipation, and freedom, the ‘below’ and the ‘right’ express condensation, heaviness, and constraint (Kandinsky, 1926). In line with his claim that the basic elements of painting are ‘alive’ in and of themselves, Kandinsky motivates his assertions in terms of the presumed ability of every great artist to ‘feel the breathing of the still untouched plane’ (Kandinsky, 1926, p. 116). Discussing the works of Kandinsky immediately brings to mind the name of Paul Klee; artist, visionary, and, like Kandinsky, teacher at the Bauhaus, the famous school of design in Weimar. Klee considered communication with nature as the most essential condition for the artist. Forms, Klee argued, are important in so far as they symbolize man’s relation to the cosmos (Klee, 1953). In symbolizing this relationship, Klee considered line, tone value (the degrees of shading between white and black), and color as the basic formal elements an artist has at his disposal: Certain proportions of line, the combination of certain tones from the scale of tone values, certain harmonies of color carry with them at the time quite distinctive and outstanding modes of expressions (Klee, 1948, p. 37).

Dependent on the combination of these factors, objects strike us as ‘serene or severe, tense or relaxed, comforting or forbidding, suffering or smiling’ (Klee, 1948, p. 33). In many of Klee’s sketches his attempts to create what he refers to as ‘expressive contrasts’ can be witnessed. Straight rigid lines, for instance, may be combined with curved smooth lines. The use of medium shades of grey, implying weakness, may be alternated with the wide use of all tones from black to white, implying strength, and with regard to color; ‘what tremendous possibilities for the variation of meaning are offered by the combination of colors’ (Klee, 1948, p. 39). Although Klee regarded the origin of artistic expression as intrinsically linked to human life and the relation of man to the cosmos, Klee, in his role as teacher, discussed the symbolic characteristics of his sketches primarily in terms of their formal properties. Likewise, the perspectives to follow, although acknowledging the role of the perceiver, primarily stress the object perceived in order to account for its expression.

2.1. Bodily expression Without doubt, the most basic form of expression is the kind we refer to as ‘body language’. Varying from facial expressions to bodily postures, the ways in which people express themselves through their ‘body language’ are numerous. The posture of an old man may express defeat and resignation, the movements of a child playfully walking over a thin line victory and temptation, and the facial expression of an abandoned lover despair and sorrow. But apart from bodily states or activities in relation to which expression is not an end in itself, but a natural autonomic reflection of one’s state of mind, at other times people deliberately engage in bodily activities to express themselves. The type of bodily expression that comes most readily to mind in this context is dance. In general terms, form (i.e. bodily shapes or postures) and motion (i.e. bodily movements) are the main carriers of meaning in dance (Kreitler and Kreitler, 1972). Kandinsky (1926) regarded a dance as an uninterrupted composition of lines and shapes, and, therefore, subject to the same ‘rules’ applying to formal elements of works of art. But what distinguishes a dance most clearly from, for instance, a painting is its dynamic character; the interaction of form and movement. Most researchers have, for that reason, studied the ways in which dancers express meanings through bodily movement. Oskar

336

Product Experience

Schlemmer, Kandinsky’s colleague and director of the Bauhaus stage in Dessau (1925– 1929) is particularly known for his stage plays and workshops in which the relations between stage, body, and space were explored: We shall observe the appearance of the human figure as an event and recognize that at the very moment it has become a part of the stage, it is a ‘space-bewitched’ creature, so to speak. With a certainty that is automatic, each gesture and each movement is drawn into the sphere of significance (Schlemmer, 1927. Cited in Wingler, 1974, p. 474).

More recently, Sawada, Suda and Ishii (2003) looked into relations between armmovement characteristics (i.e. speed, force, and directness) and emotional expression, based on Laban’s classification of movement in terms of time, weight, space, and flow (Laban, 1988). They showed that dancers’ expressions of anger are reflected in arm movements fast in velocity and strong in force, as opposed to slower and weaker arm movements indicative of sadness and joy. Expressions of joy differ from those of sadness in that the former are characterized by a longer traveled distance with the arms (i.e. more ‘expansiveness’) and a more varied trajectory (i.e. more ‘indirectness’). In a similar study, De Meijer (1989) demonstrated that expressions of positive emotions, e.g. joy, are characterized by upward directed movements (i.e. stretching trunk movement), whereas downward directed movements (i.e. bowing trunk movement) are characteristic of negative emotions. These results are in line with Osgood’s finding that people across the world perceive downward-directed curves as sad and upward-directed curves as merry or joyful (Osgood, 1960). Earlier on, we noted that Kandinsky proposed a similar structuring when relating the ‘above’ to positive connotations like lightness and freedom, and the ‘below’ to negative connotations such as heaviness and constraint (Kandinsky, 1926). Although the studies discussed in the remainder of this chapter all center on static, non-moving objects such as products and works of art, aforementioned studies are of relevance in so far as both moving and static objects are perceived as infused with symbolic qualities, as will be shown next. A second distinction concerns the difference between two-dimensional and three-dimensional forms, discussed alternately in the remainder of this chapter. Whereas human bodies, products, and sculptures may literally enclose space or convey depth, two-dimensional forms such as paintings are never literally ‘open’ or ‘closed’, ‘near’ or ‘distant’. It should be kept in mind, however, that our interest is not in what products literally convey, but in their symbolic or expressive meaning. In that regard, products and two-dimensional objects are alike; they are both perceived as expressive of symbolic meanings through their visual appearance.

2.2. Arnheim and the Gestalt school In the early twentieth century, Gestalt psychology developed as a response to the traditional method of scientific analysis advocating the analysis of complex phenomena in terms of their separate parts. According to the Gestalt psychologists ‘the whole is more than the sum of its parts’. A triangle, for instance, is perceived as an independent entity (i.e. a whole), and not just as a collection of three lines plus three angles. In order to account for such primary perceptual phenomena, the Gestalt psychologists proposed a number of innate tendencies, i.e. ‘Gestalt principles’, guiding our perception of the world and its objects (e.g. Koffka, 1935; Wertheimer, 1938). The ‘similarity principle’, for instance, predicts that things that share visual characteristics (e.g. shape, size, color, and orientation) are perceived as belonging together

Product expression: Bridging the gap between the symbolic and the concrete

337

(i.e. forming one whole). The ‘continuity principle’ reflects the finding that perceptual organization favors continuity over abruptness. Next to proposing gestalt principles, the Gestalt psychologists also assumed a preference for good or ‘prägnant’ gestalts, a term used to designate those gestalts which are the best organizations of stimuli in a given situation. In general terms, good gestalts are characterized by regularity, symmetry, inclusiveness, unity, harmony, maximal simplicity, and conciseness. Perhaps the most famous proponent of the Gestalt school is Rudolf Arnheim, best known for his enlightening discussions of works of art. He demonstrated that works of art are perceived as gestalts that can be more or less ‘balanced’ dependent on a wide variety of factors (Arnheim, 1974). Paintings, for instance, may express a sense of stillness or striving dependent on the placement of their constitutive elements within the frame (i.e. dependent on their composition). Whereas compositions properly balanced are generally perceived as still, compositions lacking balance are perceived as restless or as striving towards equilibrium. Arnheim’s conception of symbolic or expressive qualities motivates this brief discussion of his writings under an object-centered perspective: ‘Expressive qualities are authentic and objective qualities conveyed by perceptual shape, size, movement, intensity, rhythm, and so on’ (Arnheim, 1992, p. 205). The ability to directly perceive expressive characteristics is explained in terms of the organization of the nervous system, a notion referred to as ‘isomorphism’: ‘If gestalt processes are observed in perceptual experience, analogous processes are likely to account for them in the brain’ (Arnheim, 1992, p. 201). It is this claim, however, that has been subject to criticism for lack of empirical support (e.g. Berlyne, 1971; Crozier and Chapman, 1984). Nonetheless, many of Arnheim’s predictions on the perception of form, and in particular on the role of perceived balance and movement herein, have been supported by controlled studies (e.g. Locher, Gray and Nodine, 1996; Locher and Stappers, 2002). In his works ‘The dynamics of architectural form’ (1977) and ‘The power of the centre’ (1988) Arnheim also (explicitly) acknowledges the role of the perceiver in the experience of objects and architectural spaces. In the former, he introduces the concept of anisotropy to explain that different directions in space are perceived unequally (Arnheim, 1977). According to Arnheim, this perceived inequality relates to experiences that arise from moving through space; going up, for instance, takes more effort than going down since we have to overcome the force of gravity, or in Arnheim’s words: The symbolic endowment of architectural shape is compelling only because the humble daily experience of climbing stairs reverberates with the connotations of overcoming the weight of gravity and rising victoriously toward the heights (Arnheim, 1977, p. 210).

As a result of this anisotropy, directions in architecture and works of art are also perceived ‘unequally’; an extension in the vertical dimension is perceived as more pronounced in comparison to an equal extension in the horizontal dimension. It is for this reason that a tree or skyscraper standing up straight ‘looks’ more impressive than the same one brought down. Locher and Stappers (2002) demonstrated that anisotropy also influences the perception of visual displays; designs with greater ‘weight’ above the horizontal were rated as significantly more dynamic than designs with greater ‘weight’ below the horizontal.

2.3. The ecological approach The ecological approach to perception originates in the writings of J. J. Gibson (1979). According to Gibson, perception of the environment is direct and unmediated. In this

338

Product Experience

regard, the ecological approach can be seen as congruent with the notions put forward by the Gestalt school. But whereas the Gestalt psychologists explained this capability in terms of the functioning of the nervous system, Gibson assumed meanings are conveyed through information in the light, reflecting textures in the optic array. He considered this hypothesis radical ‘for it implies that the values and meanings of things in the environment can be directly perceived’ (Gibson, 1979, p. 127). Central to the theory of direct perception is the concept of an ‘affordance’, defined as a specific combination of invariant properties of an environment relative to a particular organism (Gibson, 1979). Gibson argued that people perceive things in terms of what actions they afford, i.e. in terms of what we can do with them. A chair, for instance, affords ‘sitting’, a cup affords ‘holding’, and a tree affords ‘climbing’. Clearly, for a bird or elephant none of these affordances holds; it is in this sense that affordances are relative to a particular organism. The notion that objects are perceived in terms of what they afford is reminiscent of the Gestalt school (Bruce, Green and Georgeson, 1996), and in particular of Koffka’s concept of ‘demand characteristics’: To primitive man each thing says what it is and what he ought to do with it: a fruit says ‘Eat me’; water says ‘Drink me’; thunder says ‘Fear me’, and woman says ‘Love me’ (Koffka, 1935, p. 7).

Although Gibson was not primarily interested in objects’ symbolic or expressive characteristics, his writings are of interest for the present context in so far as affordances may guide the attribution of symbolic meanings. Upon perceiving a chair, for instance, an individual may not just realize that it affords sitting; based on specifics of the chair he may understand that it affords a particular type of sitting. A chair with extended armrests, for instance, affords a more relaxed type of sitting than a chair with no armrests. Accordingly, the former may be perceived as more dignified or stately than the latter. In a similar fashion, distinguishing between motorbikes in terms of symbolic qualities such as toughness or ferociousness may be based on appreciation of the bodily position afforded by shape and positioning of the motorbike seat (Kreuzbauer and Malter, 2005).

2.4. Discussion The studies on expression reviewed in this section place primary emphasis on the object perceived. Although at times certainly successful in relating symbolic or expressive characteristics to the visual appearance of objects, a difficulty of studies originating in this approach is that they often fall short in explaining the relations uncovered. Arnheim’s assertion, for instance, that the proposed relations can be accounted for in terms of the workings of the nervous system is hard to sustain in the absence of empirical data. It is only when he explicitly acknowledges the role of the perceiver in the experience of space and objects, that one ‘understands’ why, for instance, the vertical connotes qualitatively different meanings than the horizontal. From a philosophical point of view, object-centered theories on expression are problematic in so far as they embrace the assumption that the world ‘possesses’ fixed properties that can be ‘picked up’. When looking at our own experience it may indeed seem natural to consider objects, for instance, impressive, natural or playful in and of themselves. But as it turns out, attempts to account for such expressive or symbolic qualities are frustrated as long as one concentrates primarily on the object of perception. To address the difficulties encountered by an object-centered approach, both practical and philosophical, researchers increasingly shifted the emphasis of their projects from stimulus properties of objects to the processes underlying their perception and understanding.

Product expression: Bridging the gap between the symbolic and the concrete

339

3. THE CONSTRUCTIVE INDIVIDUAL Paralleling the rise of cognitive science, researchers became increasingly interested in the ways in which processes related to the workings of body and mind partake in the establishment of meaning. Whereas in object-centered studies symbolic qualities were primarily studied in terms of objects’ formal properties, in the perspective discussed next emphasis is on the individual and all that influences his or her experience of the world.

3.1. Arousal and the aesthetic experience By far the most influential biologically centered theory on the aesthetic experience of visual stimuli is Berlyne’s psychobiological approach (Berlyne, 1971). Assuming people prefer an optimal level of arousal engaging the nervous system to the right extent, Berlyne considered the potential to introduce a rise in this arousal level the most characteristic feature of works of art. This potential, Berlyne argued, is contingent on stimulus properties such as ‘perceived novelty’, ‘surprisingness’, and ‘complexity’, properties he refers to as ‘collative’. Asymmetrical and irregular forms, for instance, are more ‘complex’ in comparison to symmetrical and regular forms and, therefore, have greater arousal potential. Berlyne was primarily interested in stimulus qualities capturing attention, increasing arousal, and affording exploration, and not so much in symbolic or expressive meanings of stimuli. Berlyne’s findings, however, are of relevance to our project in so far as they suggest a relation between the arousal potential of stimuli and their expressiveness. Underscoring this relation, Berlyne argues: Art biased towards heightening arousal is called dramatic, dynamic or stirring. If the arousalmoderating devices have the upper hand, art is said to be static, harmonious or serene (Berlyne, 1971, p. 254).

Obviously, people may prefer different levels of arousal, and dependent on the context in which the stimulus is perceived its arousal potential will vary. In paintings, for instance, a visual element is always embedded within a complex structure comprising both stylistic and semantic layers that codetermine its arousal potential, and hence to some degree its symbolic meaning or expressiveness (Cupchik, 1994). With respect to product design, Berlyne’s writings inspired Coates (2003) to explore the relation between perceived novelty and product expression. Departing from Osgood’s framework for the measurement of meaning (Osgood, 1957), Coates showed that higher degrees of novelty trigger perceptions of products as more ‘potent’ and ‘active’, as opposed to products presenting less novelty. Hence, they are perceived as expressing higher degrees of related characteristics such as excitement and emotionality (‘activity’), and dominance and toughness (‘potency’).

3.2. Dynamization and empathy Another biologically anchored phenomenon relevant to our discussion is the ‘dynamic’ or ‘dynamizing’ mode of response (Werner and Kaplan, 1963). Dynamization originates in enhanced kinesthetic or motoric reactivity, and relates to the fact that people, when asked to explain why they relate a certain form to a particular feeling or concept, often refer to a hypothetical quality of motion along its lines. One may, for instance, relate a circle to the concept of infinity because movement along its contour is endless and repetitive. Likewise, a sharp angle may be referred to as shocking or thrilling for its acute

340

Product Experience

and sudden change in direction of motion. And long curves may be said to express calm because their ‘motion’ is slow (Kreitler and Kreitler, 1972). In contrast to studies on bodily expression discussed earlier, in this line of reasoning the origin of symbolic expression is not sought in properties of the object or (simulated) shape as such, but primarily in the motoric activity associated with the stimulus. German psychologist Theodor Lipps (1897), for instance, discusses dynamization in relation to ‘empathy’ or ‘einfühlung’. According to Lipps, people show a tendency to imitate perceived movements or dynamic postures of people and objects. Since specific motoric movements automatically give rise to their emotional counterparts, Lipps argued, we come to feel or experience the emotion of another person (i.e. empathize with another person) or experience an object as expressing a particular symbolic concept. Sadness, for instance, is commonly reflected in curved, earth-bound bodily postures. Upon observing a like-formed object, Lipps claimed, we ‘unknowingly’ imitate its posture. And since a curved, earth bound posture gives rise to the emotion it corresponds with, i.e. sadness, we come to feel the emotion and locate its origin in the object perceived, whereby it becomes expressive (Lipps, 1897). Although research indeed shows that people tend to imitate behaviors, e.g. movements and emotional expressive behaviors of others (Bandura, 1969), Lipps’ claim that the emotional experience ‘automatically’ follows from such simulations has not withstood the test of time. The works of Schachter and Singer (1962), for instance, clearly show that physiological changes alone do not suffice to evoke a full blown emotional experience; of equal importance are the ways in which these sensations are interpreted.

3.3. Metaphor and analogy In interacting with products, people frequently draw implicit comparisons between products belonging to different categories, or between products and other phenomena (e.g. objects of nature, people or animals). The reason for doing so is that one may learn about a particular object by relating it to another. This ability is referred to as metaphorizing, defined as experiencing or understanding one thing, i.e. the target domain, in terms of another, i.e. the source domain (Lakoff and Johnson, 1980). The most popular computer interface is the ‘desktop’ on which the user drags items into folders, moves these to a desired location, or drops them in the trashcan (just to name a few of the actions the ‘desktop’ affords). The ‘desktop’ is successful because virtually everyone knows how to operate it by relying on existing knowledge from daily office work involving real desktops. For instance, one easily understands the purpose of placing related Microsoft Word-files in one folder, since placing paper documents in (physical) folders is a familiar ‘office activity’. It is in this sense that designers can allow their users to understand a relatively new or complex domain more easily by presenting it in terms of a domain they are familiar with. Next to using metaphor foremost as a means to render a new or complex product intelligible, i.e. to reduce the cognitive workload, designers may also employ a metaphor in order to promote other kinds of user experiences. At Delft University, for instance, a project was initiated addressing the design of a copier (Figure 13.1), departing from the metaphor ‘Interacting with a machine is a dance’ (Hekkert, Mostert and Stompff, 2003). One aspect of a ‘dance for two’ is that the participants feel and respond to each other’s moves, an aspect labeled ‘resonance’. The designers mapped this aspect onto the copier by reconsidering the (traditional) ways in which copiers react to user behavior. Agitated movements, for instance, cause this copier to offer more resistance in handling its different parts, whereas smooth movements evoke less resistance. In resonating with

Product expression: Bridging the gap between the symbolic and the concrete

341

FIGURE 13.1 Copier (from Hekkert et al., 2003).

FIGURE 13.2 CD player (IDEO).

its user, the product breaks free from inflexible interaction patterns that so often make traditional copiers a burden to deal with. Hence, the copier is no longer a frustrating machine but rather a dance partner and, therefore, experienced in terms of those qualities people commonly ascribe to dancers (e.g. supportiveness, flexibility, and sensitivity). Whereas metaphor involves the transfer of meaning from one domain to another, a product analogy typically involves a functional similarity between an object and another object or phenomenon. In contrast to metaphor, the transfer of meaning is usually not considered a defining characteristic of a product analogy. IDEO’s CD player may serve as an example (Figure 13.2). The analogy with a traditional light switch is clear; pulling the chord will either turn on or off the CD player similar to how a light may be switched on or off. But this is

342

Product Experience

where the similarity stops; it is, supposedly, not the designer’s intention to map meanings from the domain of lights or light switches onto his product, but merely to point out a surprising or witty functional similarity. One could argue, however, that the difference between metaphor and analogy is not as straightforward, or generally agreed upon, as perhaps suggested in this brief discussion. The designer of IDEO’s CD player, for instance, might object that his design was in fact centered on the transfer of meaning, grounded in the metaphor ‘Music is light’ (i.e. ‘music’ is a source of ‘light’ in the lives of people). Based on such considerations, several authors have pointed out that our understanding of a symbolic expression rests on the intention we ascribe to its creator, i.e. on our figuring out what it is the designer tries to communicate (Cupchik, 2003; Forceville, 1996; Gibbs, 1999; Sperber and Wilson, 1995). In case we assume, for the moment, the designer of IDEO’s CD player did in fact have the metaphor ‘Music is light’ in mind (and not just a functional analogy), his product may very well acquire a different, perhaps ‘deeper’, meaning, changing the experience of the product as a whole. Of particular interest in this regard are Cupchik’s writings on metaphor: Metaphors are generated through intentional acts and presume a point of view on the part of their creators, prompting readers to adopt a perspective in their ‘effort after meaning’ (Cupchik, 2003, p. 19).

Readers adopting an appropriate perspective, Cupchik argues, spontaneously experience the unity of the metaphor, brought about by a merging of the two domains involved (Cupchik, 2003). Arguably, this ‘merging’ takes on special significance in relation to product design where source and target literally merge, i.e. they literally occupy the same space (cf. Forceville, Hekkert and Tan, 2006). As a result of this merging, we usually do not distinguish between source and target in the experience of products; what is experienced is an integrated, seemingly novel phenomenon. Elaborating on the aforementioned ‘desktop metaphor,’ Fauconnier and Turner (2002) point out: The user manipulates this computer interface not by means of an elaborate conscious analogy but, rather, as an integrated form with its own coherent structure and properties. From an ‘objective’ point of view, this activity is totally novel – it shares very few physical characteristics with moving real folders … Yet the whole point of the desktop interface is that the integrated activity is immediately accessible and congenial (Fauconnier and Turner, 2002, p. 24).

Fauconnier and Turner (2002) refer to integrated forms of the kind discussed, i.e. novel structures that arise from the merging of two or more domains, as ‘blends’. Although this brief discussion might suggest metaphors are special, stylistic devices a designer has at his or her disposal, in the next section we will discuss a more fundamental way in which metaphor partakes in our experience of the world.

3.4. Learned meanings Product experience is also influenced by conventions specific to a particular group of people, culture or geographical region; conventions that we learn and familiarize with through interacting with our environment and others within our culture. The role of conventions is particularly stressed in ‘semiotics’, the theory of signs, defined as: ‘Everything that, on the grounds of a previously established social convention, can be taken as something standing for something else’ (Eco, 1976, p. 16). I have learned, for instance, that red flashing lights and high beeping sounds stand for, i.e. signal, danger. Likewise, specific gestures stand for anger or contempt (I gave him the finger), and specific kinds of

Product expression: Bridging the gap between the symbolic and the concrete

343

clothes or haircuts for membership of, or adherence to, a specific social class or ideology. As indicated by these examples, semiotics is basically concerned with communication between people living within a society by means of some system of shared conventions (Smythe, 1984). As for industrial design, the theory of signs gave rise to a discipline called product semantics, defined by Krippendorff and Butter (1984; chapter 14, this volume) as: An effort to understand and to take full responsibility for the symbolic environment into which industrial products are placed and where they should function by virtue of their own communicative qualities (Krippendorff and Butter, 1984, p. 4).

For the study of these communicative qualities, it is constructive to distinguish between ‘denotation’ and ‘connotation’. Whereas the former refers to the communication of a product’s primary function, the latter reflects the communication of socio-cultural values and behavioral standards (Muller, 2001). A chair’s primary function, for instance, is ‘sitting’; the communication of this function by means of the conventionalized form elements laid down in the prototype of the category is what we refer to as ‘denotation’. Next to communicating its primary function, a chair also communicates, i.e. connotes, supplementary or secondary meanings. Dependent on features of its design, a chair may be typified as an everyday office chair or a majestic throne. Conventions play a role herein in that particular materials or forms are readily associated with specific sociocultural values. Materials like silver and gold, for instance, are easily associated with luxury and refinement, recycled materials and plastic, on the other hand, are generally perceived as cheap or trashy. Designers, for this reason, may be considered communicators who have at their disposal a repertoire of forms and materials connoting agreed upon, shared meanings. Following this conceptualization of design as a process of communication, designers and researchers at times advocate the development of a suitable form language in which to talk about the symbolic qualities of products (e.g. Krippendorff and Butter, 1984). In car and theater design, for instance, designers often rely on the idea that consumers recognize a certain number of established, standard characters. Organic form features, for instance, are readily recognized as expressing cuteness and friendliness within Western culture, granting a car an overall friendly ‘character’ (Janlert and Stolterman, 1997).

3.5. Discussion The studies reviewed in this section place primary emphasis on biological or cognitive processes involved in the experience of objects, hence the emphasis on biologically motivated levels of arousal in the nervous system, learned meanings, metaphorizing, etc. In its strong form, the individual-centered perspective considers an object’s expression to be foremost a construct of the mind; objects merely present ‘information’, what really matters are the ways in which we process that information. It is this assumption underlying a so-called ‘idealist’ approach, one could argue, that is at times problematic. For by focusing primarily on the perceiver, they often fall short in providing us with the means to trace the symbolic or expressive meanings people give to objects. Regardless of the emphasis on either object or perceiver, it is of course true that the studies discussed so far in one way or the other acknowledge the contributions of both. In the perspective discussed next, however, it is stressed that explicit and equal emphasis should be placed on the interdependent contributions of both object and perceiver by focusing on the way they interact. This third perspective, coined ‘embodied realism’

344

Product Experience

(Lakoff and Johnson, 2002), originates in the writings of the American philosopher John Dewey and the French philosopher Merleau-Ponty. Both stress the fact that meaning can only be studied in the light of the interactions between individual and world. Since these interactions are constrained by the peculiarities of the human body, both view meaning (in opposition to theories in which meaning has nothing or little to do with the human body) as essentially embodied. Although notions that conceive of meaning as embodied are thus not ‘new’, it is only in the last two decades that a comprehensive scientific theory centered on embodiment and its relation to symbolic meaning has gradually emerged within the field of cognitive semantics.

4. THE INTERACTIONAL STANCE After reviewing both object- and individual-centered perspectives bearing on the subject of expression, Dewey concludes: ‘Both of the theories considered separate the live creature from the world in which it lives; lives by interaction through a series of related doings and undergoings’ (Dewey, 1934, p. 103). According to Dewey, meaning is not a more or less fixed property of world or mind, but arises in interactions between individual and environment. Accordingly, Dewey argues, in order to account for an object’s expression, one has to focus on the fusion of qualities directly present in the object and meanings extracted from prior experience: Different lines and different relations of lines have become subconsciously charged with all the values that result from what they have done in our experience in our every contact with the world about us. The expressiveness of lines and space relations in painting cannot be understood upon any other basis (Dewey, 1934, p. 101).

In stressing interactions between people and their environment (i.e. ‘our every contact with the world about us’) as the focal point of expression, Dewey acknowledges the role of the body in shaping our experience of the world. In a similar fashion, MerleauPonty coins the term ‘motor intentionality’ to characterize activities that involve a ‘bodily’ understanding of objects (Merleau-Ponty, 1962). An implication of the writings of both Merleau-Ponty and Dewey concerns their emphasis on knowledge as an ongoing process, rather than a fixed or static ‘thing’. In concordance with Merleau-Ponty, Dewey argues that we do not learn about our world by intellectually figuring out how things are (i.e. by forming mental representations), but through interacting with it. According to both, it is only through these interactions that we have a world to begin with, or in the words of Mark Johnson: ‘We are what we are at this instant, and our world is what it is at this instant, only because of our embodied interactions’ (Johnson, 1991, p. 11). And since these interactions are ever changing, so are the kinds of understanding they give rise to. The notions put forward by Dewey and Merleau-Ponty form the basis for the research reviewed in the remainder of this section. As indicated by these studies, the kinds of bodily understanding discussed by Merleau-Ponty and Dewey are not trivial or limited to specific, concrete situations, but influence our experience of the world in general.

4.1. Lakoff and Johnson on metaphor In the early 1980s, Lakoff and Johnson brought metaphor back in fashion by the release of their book Metaphors we live by (Lakoff and Johnson, 1980). Next to reestablishing

Product expression: Bridging the gap between the symbolic and the concrete

345

metaphor as an interesting phenomenon, they convincingly showed many of the traditional views on meaning and understanding to be mistaken. Until recently, for instance, metaphor was primarily considered a linguistic stylistic device. What Lakoff and Johnson showed was that metaphor is not just a linguistic device, but lies at the basis of thought in general. A specific class of metaphors, referred to as ‘spatial-relational’ (Johnson, 1987; Lakoff and Johnson, 1980), is of direct relevance to the subject under discussion. As Lakoff and Johnson demonstrated, spatial-relational metaphors are grounded in bodily interactions between individual and environment, and structure the way we understand symbolic expressions of all kinds. In My spirits were high again, for instance, a sense of happiness is conveyed in terms of being ‘high’ above the ground. In I am down, on the contrary, sadness is associated with being in a ‘low’ position. Apart from happiness and sadness many other concepts may also be defined in terms of the same spatial-relational concept. One may, for instance, also ‘use’ the notion of ‘being high’ to convey a sense of achievement or success as in I am on top of the world. Next to being defined in terms of a ‘low/ high’ orientation, symbolic linguistic concepts may also receive their structuring from spatial-relational aspects such as ‘inside/outside’, ‘front/back’, ‘balanced/unbalanced’, ‘near/distant’, ‘center/periphery’, etc. A not so friendly or talkative person, for instance, may be described as distant, or a psychotic patient as unbalanced or unstable. Although Lakoff and Johnson were the first to study this type of metaphor in a systematic way, Arnheim had pointed out their existence as early as 1977: All genuine metaphors derive from expressive shapes and actions in the physical world. We speak of ‘high’ hopes and ‘deep’ thoughts, and it is only by analogy to such elementary qualities of the perceivable world that we can understand and describe non-physical properties (Arnheim, 1977, pp. 208–209).

As suggested by Arnheim, defining concepts in terms of specific spatial-relational aspects is not arbitrary but relates to ‘actions in the physical world’. Lakoff and Johnson refined Arnheim’s claim by arguing that the structuring in question is grounded in experiences arising from repeated embodied interactions with the environment. Of special importance to their theory is the notion that embodied interactions may share a similar structure, referred to as an image schema.

4.2. Image schemas Image schemas are spatial-relational structures manifest in our everyday interactions. The ‘verticality schema’, for instance, arises from our ambition to achieve an erect, upright position. In our attempts we continually experience the effects of gravity pushing us down. As we grow older, we find we can resist gravity and achieve and maintain an upright position, an effort that requires power and control. We find we have control over people and things in our environment when we are literally higher (e.g. surveying a crowd of people from up ‘high’, or manipulating objects from ‘above’). Conversely, we may feel threatened or vulnerable when people or things rise above us (e.g. lying on the floor while others surround us standing up straight, or looking up at a skyscraper reaching ‘high’). What these examples have in common is that they all reflect embodied interactions between an individual and his environment that share a similar spatial-relational structure (i.e. all interactions involve an individual’s bodily orientation in the vertical plane). Similarly, in all of our interactions we are either in or out of containers of various kinds. At this moment, I am inside my office. As I walk outside, I head for my car that is another type of container. And at night I step in and out of bed, yet another type of

346

Product Experience

container. In other words, this containment structure (i.e. image schema) figures as a constant throughout our everyday interactions, or as a ‘pattern in the ongoing flow of our experience of a world’ (Johnson, 1991, p. 12). Phenomenologically, we find that various degrees of containment give rise to different types of feelings. Higher degrees of containment generally bring about higher degrees of experienced security and protection, but at the same time constrict our freedom to move, making us feel trapped or isolated. Such interactions (and the experiences they give rise to) are embodied in that they are constrained by the peculiarities of the human body. A container of any kind, for instance, is (too) small or (too) large, narrow or wide only in relation to our physical bodies. Lakoff and Johnson mostly provide linguistic evidence to demonstrate the reality and structuring role of image schemas (Johnson, 1987; Lakoff, 1987; Lakoff and Johnson, 1980). Linguistic expressions such as She is always looking down on others and We made it to the top refer to a verticality structure or schema. And because of aforementioned coupling between image schemas and experiences manifest in our own interactions, one intuitively understands such expressions as conveying a sense of success or power. In that sense, Lakoff and Johnson not only acknowledge a bodily mode of understanding underlying our experience of the physical world and its objects (as put forward by Dewey and Merleau-Ponty), but also show how such bodily ‘knowledge’ is at the basis of our understanding of symbolic or figurative language (i.e. language that is not about concrete objects or interactions with them). Image schemas not only partake in language understanding. Lakoff and Núñez (2000) demonstrated that image schemas also constrain mathematical reasoning. With respect to the ‘containment schema’, for instance, the basic fact that we can either be ‘in’ or ‘out’ of a container, but never ‘in’ and ‘out’ at the same time structures our reasoning about conceptual containers in mathematics such as closed sets of points. A point, for instance, is either ‘in’ or ‘out’ of the closed set, but never ‘in’ and ‘out’ at the same time. In a similar vein, Lakoff and Núñez demonstrate that the mathematical concept of ‘infinity’ is grounded in embodied actions that are conceptualized as cyclical or ‘not having completions’, e.g. moving or breathing (Lakoff and Núñez, 2000).

4.3. Image schemas and product expression Several authors have studied relations between image schemas and product expression (Johnson, 2002; Muller, 2001; Van Rompay, Hekkert and Muller, 2005a; Van Rompay, Hekkert, Saakes and Russo, 2005b). Van Rompay et al. (2005a,b) proposed that perceiving product expression involves recognition of spatial-relational structures in a product’s spatial gestalt. For instance, similar to how containers protect and cut people off from their environment, designed objects may do the same in relation to the contents they enclose. And since increasing degrees of closure bring about feelings of security and constriction, objects providing high degrees of closure to their contents are perceived as lending expression to these experiences to a greater extent than objects providing less closure to their contents. Similarly, following the logic of the verticality schema, the higher an object articulates a rising upward, the more likely it is perceived as dominant, impressive or proud. And in analogy to experiences we have ourselves when not properly balanced, objects perceived as unbalanced are understood as restless, unstable, and uncontrolled. To provide experimental support for these predictions, in one of our studies we designed water jugs representing the image schemas discussed (i.e. the containment schema, the verticality schema, and the balance schema) to varying degrees (Figure 13.3). Ratings of these variants on selected symbolic qualities confirmed our predictions (Van Rompay et al., 2005b). Thus, jugs providing higher degrees of closure to their contents

347

Product expression: Bridging the gap between the symbolic and the concrete

(Figure 13.3A) were perceived as increasingly expressive of symbolic qualities such as secure and constricting. Similarly, an increase in height (Figure 13.3B) resulted in higher ratings on characteristics such as dominant and impressive. And jugs improperly balanced, due to an uneven distribution of spout and handle around the vertical axis (Figure 13.3C), received high ratings on restless, unstable and uncontrolled. In short, these findings show that our understanding of product expression is grounded in embodied interactions with the environment; perceiving objects as infused with symbolic meanings rests on recognition of spatial-relational structures in product form. To use Dewey’s words, such structures ‘have become subconsciously charged with all the values that result from what they have done in our experience in our every contact with the world about us’ (Dewey, 1934, p. 101). Perceiving product expression, however, should not be considered a passive, analytical process, but rather an imaginative process in which we feel or ‘undergo’ the relations presented by the object. In this process, the perceiver ‘takes’ the perspective of the object perceived and comes to ‘feel’ the experiential consequences of the relations the object embodies. Perceiving a jug as secure or confining, for instance, involves a projection ‘into’ the jug. It is this process that enables us to ‘feel’ its (over)protective qualities. Accordingly, genuine, i.e. embodied, expression results not only from acting on, but also from simultaneously undergoing the relations presented by the object (cf. Dewey, 1934). Interestingly, ‘perspective taking’ has been extensively studied in social psychology and has been shown to underlie our capacity to make sense of other people’s thoughts and feelings (e.g. Ames, 2004). And with respect to design, Donald Schön (1988, 1992) alludes to a similar process when demonstrating how designers and architects ‘project’ themselves inside their drawings to get a ‘feel’ for the experiential consequences of their design moves: ‘Thanks to her ability to see and travel in the drawing as though seeing

(A)

(B)

(C)

FIGURE 13.3 Front views of the image schema-based manipulations: row (A) containment schema manipulation; row (B) verticality schema manipulation; row (C) balance schema manipulation.

348

Product Experience

and traveling around the building, her move experiment is also a voyage of discovery’ (Schön, 1992, p. 150). Hence, Schön claims, designers’ and architects’ design moves are ‘embodied in acts of seeing’ (Schön, 1992, p. 137). Our results suggest that not only is ‘perspective taking’ or projection essential to the creation of design, as suggested by Schön, but also to the experience of design.

4.4. Variability in the perception of product expression In presenting the embodied perspective, we suggested that it would allow us to understand those aspects of product expression that are relatively stable across situations and individuals. Human bodies, after all, show little variation across the world, irrespective of cultural origin, and the same holds, although to a lesser degree, for the environments we interact with. Therefore, the symbolic meanings discerned in products should show consistency across regions and cultures, a claim persuasively formulated by Rudolf Arnheim: The symbolism of the arts could not move us so profoundly and prevail over changes in cultural convention, were it not rooted in the strongest, most universal human experiences (Arnheim, 1977, p. 210).

In order to test the cross-cultural relevance of our findings, a replication of the study reported, using identical stimulus materials, was conducted in Brazil (Van Rompay et al., 2005b). Although the results were by and large consistent with the Dutch findings, lending partial support to our prediction, several inconsistencies emerged. Most notably, the containment schema manipulation (reflected in the degree to which the jugs enclose their contents, see Figure 13.3A) did not influence ratings on symbolic qualities as expected. For instance, the prediction that increasing degrees of containment would result in higher ratings on secure (a prediction supported by the Dutch results) was not supported by the Brazilian findings. Arguably, such discrepancies relate to differences in social interactions, climate, and/or occurrences of violence. For instance, being alone in closed isolated spaces may not be regarded as safe in Brazil, explaining why higher degrees of containment are not perceived in terms of increasing degrees of security. Closed forms, that is, may be thought of as representing isolation or retreat from the environment whereas open forms represent connectedness with, or a reaching out to, the environment. Alternatively, retreat or isolation from the environment may be more positively valued in Western European (i.e. individualist) countries whereas collectivism or connectedness to the environment is emphasized in Brazil. And on a more mundane level, differences in weather conditions might influence the connotations people ascribe to indoor and outdoor spaces across cultures. Although these are, admittedly, wild guesses, explanations of this kind may motivate why forms connote different symbolic meanings across cultures. Another source of variability pertains to the physical environment in which products reside. A product that strikes us as stately or dignified in a design store may lose its charm back home in our living room. In accounting for such contextual or environmental influences, it has to be kept in mind that products ‘acquire’ their expression in relation to their environment. Thus, a large vase may strike us as impressive because of its size relative to other objects in the environment. Accordingly, changing the measurements of the latter will bring about a change in our perception of the product’s symbolic meaning. It is in this sense that symbolic qualities are not static or fixed properties but relational; objects are not expressive of (embodied) symbolic qualities in and of themselves but only in relation to the environment or context in which they reside.

Product expression: Bridging the gap between the symbolic and the concrete

349

5. CONCLUSION In this chapter we set out to provide an overview of studies in expression with the intent to account for relationships between the concrete (i.e. the product’s formal features) and the symbolic (i.e. the product’s perceived expression). After all, that is what (some or most) designers do; they have to find a way to ‘translate’ their ideas regarding the product’s envisioned expression into (concrete) form and materials. In order to do so, we argued, it is imperative to understand how a perceived expression comes about. The answer we arrived at was not just that both perceiver and object have to be taken into account, but that we have to understand how interactions between people and their environment give rise to specific experiences. Only then can one account for symbolic meanings exemplified by product form. Of course, the type of symbolic meanings accounted for by this approach, i.e. experience-based expressions, should be distinguished from other types of symbolic meaning discussed throughout this chapter. For instance, symbolic qualities related to learned meanings clearly are not, or at least to a lesser extent, grounded in embodied interactions. Obviously then, not all symbolic meanings are embodied. But with respect to embodied expressions, are the insights presented also of practical relevance for designers? Several exploratory design exercises and workshops based on the findings presented suggest they are (Van Rompay, 2005; Van Rompay and Hekkert, 2004). In these practices, designers were instructed to design a product with a specific expression. Most striking, and frequently mentioned, was the comment that the insights had allowed them, i.e. the designers, to relate something abstract and difficult to their own experiences, thereby facilitating the translation from idea to form. It would certainly be of interest to further explore how the insights presented should be tailored to best suit designers’ needs. Such explorations in collaboration with designers might also reveal how the insights presented can be related to other formal product features, e.g. material selection, and how the findings presented apply to more detailed levels of the form giving process. In summary, the studies reviewed in this chapter demonstrate that there are many factors that underlie a product’s symbolic meaning, most of them well documented in scholarly literature. Surprisingly few studies, on the other hand, have addressed the role of embodiment in the experience of design. In this chapter, we set out to demonstrate the merits of an interactional, embodied approach to product expression. Again, this should not obscure the fact that significant parts of what products express relate to factors other than embodiment, as shown throughout this chapter. But regardless of the extent to which symbolic product meanings are embodied, it is our contention that theories on product experience should take this important factor into account. For it is only then that we come to understand how everyday interactions shape our experience of design.

REFERENCES Ames, D. R. (2004). Inside the mind reader’s toolkit: Projection and stereotyping in mental state influence. Journal of Personality and Social Psychology, 87(3), 340–353. Arnheim, R. (1974). Art and visual perception. Berkeley and Los Angeles: University of California Press. Arnheim, R. (1977). The dynamics of architectural form. Berkeley and Los Angeles: University of California Press. Arnheim, R. (1988). The power of the centre. Berkeley and Los Angeles: University of California Press. Arnheim, R. (1992). To the rescue of art: Twenty-six essays. Berkeley and Los Angeles: University of California Press.

350

Product Experience

Bandura, A. (1969). Principles of behavior modification. New York: Holt, Rinehart and Winston. Berlyne, D. E. (1971). Aesthetics and psychobiology. New York: Appleton-Century-Crofts. Bruce, V., Green, P. R. and Georgeson, M. A. (1996). Visual perception: Physiology, psychology, and ecology (3rd Ed.) East Sussex: Psychology Press. Coates, D. (2003). Watches tell more than time: Product design, information, and the quest for elegance. London: McGraw-Hill. Creusen, M. E. H. and Schoormans, J. P. L. (2005). The different roles of product appearance in consumer choice. Journal of Product Innovation Management, 22, 63–81. Crozier, W. R. (1994). Manufactured pleasures: Psychological responses to design. Manchester: University Press. Crozier, W. R. and Chapman, A. J. (1984). The perception of art: The cognitive approach and its context. In: W. R. Crozier and A. J. Chapman (Eds.) Cognitive processes in the perception of art, pp. 3–23. Amsterdam: North-Holland. Cupchik, G. C. (1994). Emotion in aesthetics: Reactive and reflective models. Poetics, 23, 177–188. Cupchik, G. C. (2003). The ‘interanimation’ of worlds: Creative metaphors in art and design. The Design Journal, 6(2), 14–28. De Meijer, M. (1989). The contribution of general features of body movement to the attribution of emotions. Journal of Nonverbal Behavior, 13, 247–268. Dewey, J. (1934). Art as experience. New York: Berkley Publishing Group. Eco, U. (1976). A theory of semiotics. Bloomington, IN: Indiana University Press. Fauconnier, G. and Turner, M. (2002). The way we think: Conceptual blending and the mind’s hidden complexities. New York: Basic Books. Forceville, C. (1996). Pictorial metaphor in advertising. Routledge: London. Forceville, C., Hekkert, P. and Tan, E. (2006). The adaptive value of metaphors. In: U. Klein, K. Mellman, and S. Metzger (Eds.) Heuristiken der Literaturwissenschaft. Einladung zu disziplinexternen Perspektiven auf Literatur, pp. 85–109. Paderborn: Mentis. Gibbs, R. W. (1999). Intentions in the experience of meaning. Cambridge, MA: Cambridge University Press. Gibson, J. J. (1979). The ecological approach to visual perception. Boston: Houghton Mifflin. Hekkert, P., Mostert, M. and Stompff, G. (2003). Dancing with a machine: A case of experience-driven design. Proceedings of the 2003 International Conference on Designing Pleasurable Products and Interfaces, Pittsburgh, pp. 114–119. Janlert, L. E. and Stolterman, E. (1997). The character of things. Design Studies, 18, 297–314. Johnson, M. (1987). The body in the mind: The bodily basis of meaning, imagination, and reason. Chicago: Chicago University Press. Johnson, M. (1991). Knowing through the body. Philosophical Psychology, 4(1), 3–18. Johnson, M. (2002). Architecture and the embodied mind. Oase, 58, 75–93. Kandinsky, V. (1926). Point and line to plane. New York: Dover Publications. Klee, P. (1948). On modern art. London: Faber and Faber. Klee, P. (1953). Pedagogical sketchbook. London: Faber and Faber. Koffka, K. (1935). Principles of Gestalt psychology. London: Routledge and Kegan Paul. Kreitler, H. and Kreitler, S. (1972). Psychology of the arts. Durham, NC: Duke University Press. Kreuzbauer, R. and Malter, A. J. (2005). Embodied cognition and new product design: Changing product form to influence brand categorization. Journal of Product Innovation Management, 22, 165–176. Krippendorff, K. and Butter, R. (1984). Product semantics: Exploring the symbolic qualities of form. Innovation: The Journal of the Industrial Designers Society of America, 3(2), 4–9. Laban, M. (1988). The mastery of movement. Plymouth, MA: Northcote House. Lakoff, G. (1987). Women, fire and dangerous things: What categories tell us about the nature of thought. Chicago: Chicago University Press. Lakoff, G. and Johnson, M. (1980). Metaphors we live by. Chicago: Chicago University Press. Lakoff, G. and Johnson, M. (1999). Philosophy in the flesh: The embodied mind and its challenge to Western thought. New York: Basic Books. Lakoff, G. and Johnson, M. (2002). Why cognitive science needs embodied realism. Cognitive Linguistics, 13, 245–263. Lakoff, G. and Núñez, R. E. (2000). Where mathematics comes from: How the embodied mind brings mathematics into being. New York: Basic Books. Lipps, T. (1897). Raumästhetik. Leipzig, Germany: Barthes. Locher, P., Gray, S. and Nodine, C. (1996). The structural framework of pictorial balance. Perception, 25, 1419–1436. Locher, P. and Stappers, P. J. (2002). Factors contributing to the implicit dynamic quality of static abstract designs. Perception, 31, 1093–1107.

Product expression: Bridging the gap between the symbolic and the concrete

351

Merleau-Ponty, M. (1962). Phenomenology of perception. London: Routledge and Kegan Paul. Muller, W. (2001). Order and meaning in design. Utrecht, The Netherlands: Lemma Publishers. Osgood, C. E. (1957). The measurement of meaning. Urbana, IL: University of Illinois Press. Osgood, C. E. (1960). The cross-cultural generality of visual-verbal synesthetic tendencies. Behavioral Science, 5, 146–169. Postrel, V. (2003). The substance of style. New York: Harper Collins. Sawada, M., Suda, K. and Ishii, M. (2003). Expression of emotions in dance: Relation between arm movement characteristics and emotion. Perceptual and Motor Skills, 97, 697–708. Schachter, S. and Singer, J. E. (1962). Cognitive, social and physiological determinants of emotional state. Psychological Review, 69, 379–399. Schlemmer, O. (1927). ‘Bühne’. In: H. M. Wingler (Ed.) The Bauhaus: Weimar, Dessau, Berlin, Chicago, pp. 474. Massachusetts, MA: MIT Press. Schön, D. A. (1988). Designing: Rules, types and words. Design Studies, 9, 181–190. Schön, D. A. (1992). Kinds of seeing and their function in designing. Design Studies, 13, 135–156. Smythe, W. E. (1984). Psychology and the traditions of symbolization. In: W. R. Crozier and A. J. Chapman (Eds.) Cognitive processes in the perception of art, pp. 45–63. Amsterdam: North-Holland. Sperber, D. and Wilson, D. (1995). Relevance: Communication and cognition (2nd Ed.) Oxford: Blackwell. Van Rompay, T. (2005). Expressions: Embodiment in the experience of design. Delft University of Technology, The Netherlands. Van Rompay, T. and Hekkert, P. (2004). Designing embodied expressions. In: J. Redmond, D. Durling and A. de Bono (Eds.) Futureground: Proceedings of the Design Research Society, Vol. 2, pp. 230–237. Melbourne: Monash University Press. Van Rompay, T., Hekkert, P. and Muller, W. (2005a). The bodily basis of product experience. Design Studies, 26(4), 359–377. Van Rompay, T., Hekkert, P., Saakes, D. and Russo, B. (2005b). Grounding abstract object characteristics in embodied interactions. Acta Psychologica, 119(3), 315–351. Veryzer, R. W. (1995). The place of product design and aesthetics in consumer research. Advances in Consumer Research, 22, 641–645. Werner, H. and Kaplan, B. (1963). Symbol formation: An organismic–developmental approach to language and the expression of thought. New York: Wiley. Wertheimer, M. (1938). Laws of organization in perceptual form. In: W. Ellis (Ed.) A source book of Gestalt psychology, pp. 71–88. London: Routledge and Kegan Paul.

11

AESTHETICS IN INTERACTIVE PRODUCTS: CORRELATES AND CONSEQUENCES OF BEAUTY MARC HASSENZAHL University of Koblenz-Landau, Landau, Germany

1. INTRODUCTION Beauty matters. Certainly, most people would agree. Beauty is an important ingredient of our daily lives. We admire and praise the beauty of nature, architecture, music, other people – an ugly color or an awkward form easily repels us. Given its pervasiveness, the lack of research addressing beauty (or aesthetics) in Human-Computer Interaction (HCI) is striking. HCI seems a ‘science of design’ (Carroll, 1997) that long neglected beauty. A cursory search of the keyword ‘aesthetics’ in the Association of Computing Machinery’s (ACM) Digital Library (www.acm.org/dl), for example, showed 25% of all retrieved papers to be published in 2005. This is so far the culmination of a growing interest in the topic, which had its starting point about 10 years ago. In an attempt to define criteria for the ACM interaction design award, Alben (1996) emphasized the importance of aesthetics by making it one out of eight aspects contributing to a quality user experience. Today, aesthetics are viewed as a non-instrumental quality, forming an important aspect of product appeal and experience (e.g. Hassenzahl and Tractinsky, 2006). However, empirical research addressing questions such as how to ‘measure’ aesthetics; whether aesthetics can be reliably differentiated from other aspects, such as usability; how important is beauty as a part of experience; what is the value users attach to it; and what are ‘consequences’ of beauty, is sparse and results are inconsistent. I believe that inconsistencies in findings can be at least partially resolved by distinguishing three different approaches to the study of beauty: A normative; an experiential; and a judgmental. The normative approach defines particular descriptive attributes of the Product Experience Copyright © 2008 Elsevier Ltd.

287

288

Product Experience

interactive product as expressing more or less beauty (e.g. Ngo and Byrne, 2001). Such attributes can be, for example, symmetry or proportion. This approach further assumes symmetry to be more beautiful than asymmetry, particular proportions to be better than others, and so forth. Based on the objective configuration of attributes, it can then be decided whether the product is beautiful or ugly. The normative approach is primarily design-oriented. Thus, it starts from the materials (e.g. color, layout, form, movements) and attempts to provide a ‘recipe’ of how to design something beautiful. The experiential approach focuses on all-embracing, holistic aesthetic experiences (e.g. Frohlich, 2004; McCarthy and Wright, 2004) marked by an altered perception of one’s surroundings or a scene – a heightened sense for objects, persons, the environment, which creates and attaches new, yet unthought meaning to things. The experiential approach is primarily concerned with preserving the complexity and richness of an aesthetic experience. Beauty is rather thought of as something rare, outstanding – a ‘design prize’. Finally, the judgmental approach is concerned with what users judge to be beautiful or not (e.g. Tractinsky, Katz and Ikar, 2000; Hassenzahl, 2004a). This approach is foremost interested in the consistency of beauty judgments among individuals and how fast and easy those judgments are (e.g. Lindgaard et al., 2006). In addition, it addresses the question of how beauty relates to other product attributes, such as novelty or usability. The present chapter focuses on the judgmental approach to the study of aesthetics/ beauty. It starts with an attempt to define beauty in a way which lends itself to its empirical/quantitative study in the context of HCI. This is followed by a review of research addressing correlates of beauty, primarily focusing on the relation between beauty and usability. After this, three consequences of beauty are considered in detail, namely beauty as added value, beauty as a way to accomplish self-referential goals and, finally, beauty as a way to work better. The chapter ends with a summary and conclusion.

2. BEAUTY ‘DEFINED’ The appreciation and evaluation of a piece of art, such as Velázquez’s ‘Las Meninas’, is a complex process. Leder and colleagues (Leder et al., 2004) suggest the process to start with perception, followed by the implicit memory integration of the artwork with previous experience; the artwork’s explicit classification in terms of style and content, its interpretation (cognitive mastering) and evaluation. The output of this process is an aesthetic judgment of the artwork. Clearly, such a judgment goes way beyond the mere appreciation of the artwork’s visual quality. In other words, a piece of art can be good, without being necessarily visually pleasing (e.g. ‘DaDa’). Or it can be visually pleasing, without much quality (e.g. ‘kitsch’). In this sense, aesthetic judgment is very broad. It becomes aesthetic mainly because the judgmental object is a piece of art. In the context of HCI or interactive product design, such a broad definition of aesthetics can be problematic. Interactive products mostly serve purposes; they embody mundane action goals, such as ‘making a telephone call’, ‘purchasing a flight ticket’, or ‘ordering a book’. This distinguishes them from pieces of art, which per definition do not serve personal goals other than enjoyment or creation of new insights. The appeal of an interactive product – its evaluation – can be the exclusive consequence of its functionality or usability. Being able to send email via Microsoft’s Outlook is surely a useful and, thus, appealing possibility. If Outlook would be the only available tool to send mail, you surely would appreciate it, that is, you would appreciate the opportunity to send electronic mail. But would you consider Outlook as particularly beautiful, just because it enables you to send electronic mail?

Aesthetics in interactive products: Correlates and consequences of beauty

289

Some researchers, nevertheless, adopt a relatively broad definition of aesthetics in the context of interactive products, i.e. ‘non-aesthetic’ objects (e.g. Lavie and Tractinsky, 2004; Park, Choi and Kim, 2004; De Angeli, Sutcliffe and Hartmann, 2006). Park and colleagues (2004) explicitly state that aesthetic impressions can include qualities such as ‘adorable’, ‘cool’, or ‘strong’, which are rather general evaluative terms. Unfortunately, this approach to beauty may lead to a substantial overlap with broader models of user perceived quality. Take Lavie and Tractinsky’s (2004) two-component model of aesthetics as an example. They distinguish classical aesthetics (e.g. clear, symmetric, clean) and expressive aesthetics (e.g. creative, original, fascinating). The expressive aesthetics dimension, however, perfectly maps on the hedonic quality–stimulation dimension of my multi-component model of appealing products (e.g. Hassenzahl, 2003). Stimulation is the product’s ability to satisfy human needs for novelty and curiosity. It is measured by attributes such as creative, original, innovative. It is impossible to decide which of the two labels (expressive aesthetics vs. hedonic quality–stimulation) attached to the same underlying dimension (novelty, originality) is the more correct. The point is, however, that broad approaches to aesthetics may blur potential differences between general measures of appeal and aesthetics. By that, the concept of aesthetics loses some of its discriminant power. In addition, it must compete with existing multidimensional models of overall product appeal. A more concise definition of aesthetic judgment may stress the sensory – primarily visual – nature of input to judgment (e.g. Bloch, Brunel and Arnold, 2003; Lindgaard and Whitfield, 2004; Hekkert, 2006). In other words, aesthetics may be narrowed down to the (mainly visually mediated) physical attractiveness of the product – its beauty. This has three advantages. First, the restriction of aesthetic judgment to judgments of beauty closely corresponds to laypersons’ understanding of aesthetics. A study of the associative words individuals generate for ‘aesthetics’ (Jacobsen et al., 2004) showed ‘beautiful’ and ‘ugly’ to be the most prototypical for aesthetic judgment. Ninety-two percent of the 311 participants produced ‘beautiful,’ 42% ‘ugly’ and 27% ‘pretty’ as association to ‘aesthetics’. This is in contrast to only 5% who associated ‘aesthetics’ with ‘attractive’, ‘enjoyable’ or ‘cool’. Secondly, beauty in this visual, physical sense can now differ from overall appeal. We may like products just because of their utility or usability, maybe even despite their obvious ugliness. Beauty discriminates. Hassenzahl (2004a), for example, studied judgments of goodness and beauty for mp3-player skins. Although both judgments were correlated (study 1: r ⫽ φ.54, study 2: r ⫽ φ.53 [pre-use], r ⫽ φ.39 [post-use]) – as would be expected for any evaluative construct – they also showed interesting differences. Goodness (as opposed to beauty) was related to usability (pragmatic quality), mental effort (induced by experiencing of usability problems), and was affected by usage. Note the reduction of the correlation between goodness and beauty after having used the products (from φ.53 to φ.39). Especially this dissociation, that is, the differential effect of product usage (as opposed to first impressions or in other words apparent versus experienced) on goodness and beauty judgments, justifies the more restricted definition. Just to give a further example, Mahlke (2002) found four independent aspects to contribute to the intention to use websites. Among those, one was labeled ‘perceived visual attractiveness’ (what I call beauty). Together with utility, usability and stimulation, it explained 72% of the total variance. Thus, beauty can be distinguished from other quality aspects, and plays a significant role in the overall evaluation of the product. Thirdly, only the restriction of aesthetic judgment to beauty makes it a truly neglected aspect in HCI. General user satisfaction (e.g. Lalomia and Sidowski, 1990) and technology acceptance (e.g. Davis, 1993) are well-researched topics. However, beauty in its

290

Product Experience

narrow sense never played an explicit role in according models, not even in the presumably most comprehensive model of user acceptance, the unified model of user acceptance (UTAUT, Venkatesh et al., 2003). Beauty judgments – in a narrow sense – are remarkably stable over time. Tractinsky and colleagues (Tractinsky, Cokhavi and Kirschenbaum, 2006) as well as Lindgaard and colleagues (2006) consistently found a high retest-reliability for beauty judgments, even if the exposure time was as short as 50 ms. This may be due to the fact that participants in both studies saw only images of products (websites) without having the possibility to interact with them, i.e. to experience them. However, Hassenzahl (2004a) showed a high retest-reliability (r ⫽ φ.87) of beauty judgments before and after having used a product, which was not mirrored by judgments of goodness (r ⫽ 0.46). The stability of beauty judgments, even in the face of actual product experience, supports the visual nature of beauty. Beauty – other than, for example, usability – can be more immediately assessed on the basis of the product’s visual Gestalt. Moreover, key visual aspects are unlikely to change greatly over time. Both features make judgments of beauty fast and stable. Some authors do not explicitly distinguish between affect and beauty. Zhang and Li (2005), for example, subsume van der Heijden’s (2003) ‘perceived visual attractiveness’, Tractinsky et al.’s (2000) ‘perceived aesthetics’ or Schenkman and Jönsson’s (2000) ‘first impressions’ as affective constructs. In this sense, beauty judgments become almost indistinguishable from perceptions of affective quality (Russell, 2003), i.e. an object’s ability to impact a person’s affective state. I wouldn’t go this far. Surely, integral affective responses to objects, i.e. feelings either produced by the object’s percept or its mental representation (Pham et al., 2001), play an important role in any judgment. However, following an ‘affect-as-information’ approach (e.g. Schwarz and Clore, 1983), those affective responses may rather serve as an input to the judgmental process than as the outcome. In this view, affective responses must, for example, be attributed to the visual Gestalt of the object, and not to its content to be considered relevant for a beauty judgment. To give an example: A movie can feature a repelling story with beautiful images. Even if the initial affective response to the movie had been negative (triggered by the repelling story a friend told us about), this affective response is – if correctly attributed to the story – worthless for a judgment of beauty, and its influence may be discounted or even corrected (e.g. Wegener and Petty, 1997). Nevertheless, although it may seem useful to separate affective responses from judgments of beauty, we may assume that beauty is more likely to be affect-driven than other more general, evaluative constructs. In other words, individuals may find it more adequate to allow for an affective basis of judgments of beauty compared to judgments of goodness. This has interesting implications. If we assume a simple two-stage judgmental process for product beauty, which uses immediate affective responses attributed to the percept of the product as an input, which then can – but must not necessarily – be further modified by previous experience, domain specific knowledge or personal tastes (see Leder et al., 2004), beauty judgments may show a higher interpersonal consistency than commonly expected. Pham and colleagues (2001) found individuals to agree more about their feelings toward targets (photographs, commercials) than about their overall evaluation. If, for example, beauty is more affect-driven than goodness, one should expect a higher interpersonal consistency for judgments of beauty. And indeed, in a meta-analysis of 102 studies (total number of participants ⫽ 15,681) of the ratings of human (facial) beauty, Langlois and colleagues (2000) report a surprisingly high interrater-agreement, within and across cultures. They came to the conclusion that the maxim ‘beauty is in the eye of the beholder’ is a myth; people agree on the beauty (physical attractiveness) of other people. Whether the same holds for the beauty of products has still to be demonstrated. In addition,

Aesthetics in interactive products: Correlates and consequences of beauty

291

the suggested two-stage model also implies that the consistency of beauty judgments maybe the higher, the less solid the judgmental basis. In other words, individuals confronted with only a 50 ms exposure to a product’s visual Gestalt may more heavily rely on their initial affective response, and thus, may be even more interpersonally consistent. Let me summarize this section by a definition of beauty judgments, which I find appropriate as basis for further research in the context of HCI: A judgment of beauty is a predominantly affect-driven evaluative response to the visual Gestalt of an object. It takes the percept of the object and the integral (i.e. attributed) affective response as input. This input may be further modified by classification and comparison processes. Beauty’s relative reliance on integral affect makes it faster and more consistent than complex judgments of goodness.

In the following, I will first review literature on correlates of the beauty judgment. Based on this, three consequences of beauty are further discussed: Beauty’s value; beauty’s self-referential nature; and the impact of beauty on task performance.

3. CORRELATES OF BEAUTY 3.1. What is beautiful is usable: — Myth or truth? Noam Tractinsky’s (1997, Tractinsky et al., 2000; Lavie and Tractinsky, 2004) claim ‘What is beautiful is usable’ received considerable attention in the HCI community. Norman (2004a), for example, took these findings to support his notion of ‘attractive things work better’ (see Section 4.3). Since then, Tractinsky’s claim has been repeatedly challenged. Hassenzahl (2004a), for example, found no correlation between usability (pragmatic quality) and beauty ratings for an mp3-player with different skins (mean r ⫽ φ.07 [Study 1], mean r ⫽ φ.14 [Study 2, pre-use], mean r ⫽ φ.08 [Study 2, post-use]). In his own study of skins, Tractinsky and Zmiri (2006) obtained the same result (r ⫽ φ.03). Van der Heijden (2003) was able to separate visual attractiveness (beauty) from ease-ofuse (usability) using principal components analysis with an orthogonal rotation, which implies independency of components (see also Mahlke, 2002). Lindgaard and colleagues (2006; see also Lindgaard and Dudek, 2002) studied the relation between beauty and ratings of other design characteristics. They found high correlations (smallest r ⫽ φ.93) with ‘interesting–boring’, ‘good design–bad design’, ‘good color–bad color’, ‘good layout– bad layout’, ‘imaginative–unimaginative’, but not for ‘simple–complex’ (r ⫽ φ.10) and lower for ‘clear–confusing’ (r ⫽ φ.63). The latter are clearly usability-related attributes (Hassenzahl, Platz, Burmester and Lehner, 2000; Hassenzahl, 2002). To conclude, whether what is beautiful is really usable remains inconclusive. Potential reasons for this lack of correspondence are manifold and may lie in differences of product genres (automated teller machines versus mp3-player skins or web sites) or differences in methods employed (see for an extensive debate Frohlich, 2004; Hassenzahl, 2004a; Monk, 2004; Norman, 2004b; Overbeeke and Wensveen, 2004; Tractinsky, 2004). Let me elaborate a bit further on two particular aspects that may influence the observed correlation between beauty and usability, namely the (1) type of correlation and (2) attribute overlap. Two ways to correlate In studies of beauty and usability two types of strategies can be distinguished (Monk, 2004). The one aggregates (averages) ratings across individuals to derive a score on each attribute for each product and then correlates attribute scores (materials analysis, for an example see Tractinsky, 1997). The product serves as the focal point of analysis; variance introduced by individuals is discarded. A high correlation as result of this type of

292

Product Experience

analysis indicates a sample of products that predominantly consists of high usability/high beauty and low usability/low beauty products. If stable, this may, for example, point at successful/unsuccessful underlying design practices, which contribute simultaneously to beauty and usability. The other type of strategy aggregates (averages) ratings across products to derive a score on each attribute for each individual and then correlates attribute scores (subjects analysis). The individual serves as the focal point of analysis; variance introduced by products is discarded. A high correlation in this type of analysis indicates that individuals who find something usable may also find it more beautiful and vice versa. If stable this may, for example, point at naïve theories of attribute covariation (akin to implicit personality theories, e.g. Schneider, 1973), which in turn serve as a basis for inference processes (e.g. Kardes, Posavac and Cronley, 2004). To my mind, both strategies have their merits and should be addressed in future studies. Monk (2004), however, strongly argued for the materials strategies as the one better able to inform designers’ work. Although I already took up this issue elsewhere (Hassenzahl, 2004b), let me devote some space to the question, why subjects analysis may be important, too. The goal of subjects analysis is to describe individual’s naïve theories of attribute covariation. If beauty and usability correlate in a sample of individuals, that is, individuals who rated an object high on usability did the same for beauty and individuals who rated an object low on usability did the same for beauty, one may argue that both attributes belong together. People believe them to covary. Why is this important? As already pointed out, there is a striking difference between beauty and usability judgments. Because of its visual, affect-driven nature, beauty is much easier to assess than usability. Assessment of usability attributes, such as whether a product is ‘predictable’ or ‘manageable’ needs experience, i.e. usage of the product. Beauty does not. Thus, beauty – as based on immediately accessible information about the product (i.e. its visual Gestalt) – may be used to infer other, less immediately accessible qualities, such as usability or utility. In other words, individuals may infer unobservable attributes from beauty. Literature on consumer inference (see Kardes et al., 2004, for an overview) reports many types of inference strategies. Imagine being exposed to the – to you unknown – homepage of a new bank and someone asks you to judge its usability on a 0–100 scale, without being allowed to browse the page. One strategy is to estimate usability as discounted average on the basis of its known distributional properties (e.g. Johnson and Levin, 1985). In other words, you may know something about the distribution of usable banking websites (e.g. most of them are fairly usable) and, thus, infer that the website at hand might be similar (e.g. is fairly usable, too). You may further adjust this inference to the fact that you lack actual experience with the website at hand. A second strategy is evaluative consistency (e.g. Lingle and Ostrom, 1979). Here an overall favorable impression derived from all observable attributes leads to a favorable impression on the unobservable attribute. If the banking website is beautiful, was recommended by a friend, and the bank is well known, usability might be good, too. Note, that no direct causality is assumed, i.e. in the sense that beauty causes good usability. It is rather a ‘halo’effect (Thorndike, 1920). The third strategy, however, implies causality. Probabilistic consistency inferences assume a causal link between an observable and the unobservable attribute (e.g. Ford and Smith, 1987). It would be given, if individuals assign a good usability to the website based on their assessment of beauty, because they believe beauty to cause good usability. A final possible strategy is compensatory inferences (Chernev and Carpenter, 2001). Individuals may believe that there ain’t no such thing as a free lunch, and, thus, if a bank invested in its website’s beauty other important attributes may not

Aesthetics in interactive products: Correlates and consequences of beauty

293

have received the same attention. In this case, beauty would lead to the inference of low usability. In a preliminary study of inference processes, I gave each of the 100 participants a list of 30 (hypothetical) mobile phones, each described on three attributes: Price (range €40 to €400); functionality; and beauty (both on a scale from 0 to 100, actual range 10 to 100). All three attributes were uncorrelated (no interattribute correlation larger than φ.08). The participants were asked to estimate each mobile phone’s usability (0 to 100) on the basis of the available information. Correlations and regressions were calculated for each individual separately to get an idea of personal strategies. Interestingly, 53% of the participants showed a significant positive relation between beauty and their assessment of usability. Only 3% exhibited the inverse relationship of beauty and usability (‘the dark side of beauty’), and the remaining 44% failed to show any correlation. Thus, for slightly more than half of the participants, beauty and usability were related. Of the 56 participants, who took beauty into account to predict usability, only two solely relied on beauty alone. The remaining relied on either two or three attributes. A stepwise group regression model showed functionality to be the best predictor of usability ratings (explained variance ⫽ 14%), followed by beauty (⫹3% explained variance). Price explained virtually no additional variance (⫹1% explained variance). All in all, individuals seemed to rely on an evaluative consistency strategy, where the quantity of an unknown attribute is inferred from the quantity of other attributes. Beauty played an – albeit compared to functionality minor – role in the inference of usability, which points at least to the possibility that beauty contributes to value and that from this value usability is inferred. To me, studies of inference processes involving usability and beauty seem a promising way to study the perceived relation between both constructs. Obviously, knowledge of these processes would not only be interesting, but also valuable to designers. Attribute overlap Another feature of usability and beauty, which may make correlation likely, is attribute overlap. Some attributes such as ‘clean layout’ may contribute to both beauty and usability. Indeed, I would attribute the observed high correlation between ratings of beauty and usability in, for example, Tractinsky’s 1997 study (but only partially in Tractinsky et al., 2000) to this type of attribute overlap. In this study, 26 different layouts of automated teller machines (ATM, adapted from Kurosu and Kashimura, 1995) were used as products. Each layout consisted of the same basic elements (e.g. a number block, display). Variations in beauty were solely due to variations in the spatial layout. One-hundred and four participants rated each of the 26 layouts, and ratings were averaged across participants (materials analysis). Beauty and usability was correlated. Without variation of other design dimensions, such as form, color, interactional style, etc., the results can easily be interpreted as the consequence of a particular property of the spatial layout dimension, namely to impact both beauty and usability. For example, Gestalt psychology’s laws of organization (e.g. ‘proximity,’ see Goldstein, 1989, pp. 192) are regarded as both, a general theory of aesthetics and a central way to a ‘simple and natural dialog’ with an interactive product (Nielsen, 1993, p. 118). In other words, correlation stems from the fact that distinct aspects, such as beauty and usability, partially rely on the same objective properties, such as layout.

3.2. Other correlates In HCI, research on beauty and its correlates almost exclusively focused on beauty and usability (e.g. Tractinsky, Katz and Ikar, 2000), with some notable exceptions.

294

Product Experience

I (Hassenzahl, 2004a) found a substantial relation between judgments of beauty and what I call hedonic product attributes in mp3-player skins, such as ‘inclusive’, ‘presentable’ or ‘brings me closer to people’. This group of attributes was labeled ‘identification’. It captures the product’s perceived ability to communicate a favorable Self to relevant others. This mirrors results by Tractinsky and Zmiri (2006), who found a high correlation (φ.72) between beauty and what they call ‘symbolism’ (e.g. ‘positive message about user’, ‘communicates desirable image’). A more solid body of knowledge on beauty judgments resulted from the study of the physical attractiveness stereotype in social psychology. Starting with the seminal paper of Dion, Berscheid and Walster (1972) entitled ‘what is beautiful is good’, literally hundreds of studies explored further the basic finding that the physical attractiveness of a person (beauty) leads to more favorable evaluations on a variety of other, unobserved, yet relevant dimensions. Two meta-analyses concisely subsume the findings, one exclusively focusing on the attribution of unobservable, favorable traits to more or less beautiful strangers (Eagly et al., 1991), the other (Langlois et al., 2000) extending findings by including actual behavioral consequences (i.e. number of sex partners, income) for beautiful and not so beautiful individuals, differences in treatment by others (i.e. giving recommendations for hiring, social attention), and self-perception (e.g. perceived happiness, perceived susceptibility to mental illness). Eagly and colleagues (1991) found a considerable effect of beauty on ratings of social competence (d ⫽ φ.68), but only a mid-size effect on intellectual competence (d ⫽ φ.46). General evaluations of persons were substantially affected by beauty (d ⫽ φ.57). First, these results may point at the primarily social nature of the beauty stereotype, because judgments of intellectual competence, i.e. the rational, pragmatic side of individuals, were less affected by beauty than judgments of social competence (but see Langlois et al., 2000, for less pronounced results in their review). Secondly, beauty substantially affected the general evaluation of a person, which in turn may lead to a more favorable judgment on any other attribute – at least if evaluative consistency is used as an inference strategy (see section ‘Two ways to correlate’ above). Although it may seem questionable whether results from the research on the beauty of persons generalize to the beauty of interactive products, the findings in HCI seem to mirror findings from social psychology where beauty is correlated with self-presentation (i.e. identification, symbolism). As I put it elsewhere: ‘Beautiful people are assumed to better get along with others, to be more popular, etc. Maybe people apply this general notion to objects as well, in the sense, that possessing beautiful things will help one to better get along with people and will make its owner more popular’ (Hassenzahl, 2004b, p. 380). To summarize, research on correlates of beauty in HCI is sparse and inconsistent. This may be due to the different approaches taken. From the research so far, we may cautiously conclude that beauty has self-presentational (i.e. social) implications. Its relationship to usability is inconclusive. From a materials perspective (i.e. focus on products), it seems likely that both attributes correlate mildly, because (1) they partially overlap and (2) good designers may provide in general better quality than bad designers, that is, someone who cares about beauty may also care about usability. From a subject’s perspective, beauty and usability may correlate as a consequence of a ‘halo’-effect. Specifically, people may infer a higher quality of the product from its beauty, which in turn implies a better usability. Note that some may also hold a ‘what is beautiful is unusable’ stereotype based on the idea that beauty is shallow and deliberately used to conceal deeper deficits (e.g. Russo and De Moraes, 2003). Literature on the physical attractiveness of people (Langlois et al., 2000), however, failed to find strong evidence for such a ‘dark side’ of beauty (e.g. the ‘dumb blonde’ stereotype).

Aesthetics in interactive products: Correlates and consequences of beauty

295

4. CONSEQUENCES OF BEAUTY 4.1. Beauty as a source of value Obviously, beauty is a source of value. In one study (Bloch et al., 2003, Study 7), participants saw and rated pictures of two different toasters. While being equal in function, the toasters differed in beauty. Among other things, participants were asked to state their willingness to pay for both toasters. On average, participants were willing to spend $37.20 on the beautiful toaster, but only $24.05 on the not so beautiful toaster. In other words, beauty was worth $13.15, i.e. an increase of about 55%. In an unpublished study, I confronted participants with the following hypothetical situation: Imagine, you have just bought a mobile phone for €100. Immediately after the purchase, the dealer offers an alternative phone, which is – everything else being equal – more beautiful. Participants were then asked to state the maximum additional charge they would accept for upgrading the phone. They were further told that the dealer would either accept or reject the offer, depending on the amount offered, and that no bargaining was possible. The average premium offered was €33.67 (95% c.i.: €24.99–42.34). Albeit slightly lower than in the toaster study, this is still a significant premium for beauty. Although the notion that beauty adds value seems intuitive, studies reveal a more complex picture. Whether beauty adds value can depend on individual or situational aspects. In the toaster study already mentioned above, Bloch and colleagues (2003) identified an individual difference, the so-called centrality of visual product aesthetics (CVPA), as an important moderator of beauty’s value. CVPA subsumes three aspects: Value, acumen, and response. Individuals with a high CVPA attach personal value to beauty (e.g. ‘Beautiful product designs make our world a better place to live’); they think of themselves as connoisseurs, able to perceive the subtlest differences in beauty (e.g. ‘I see things in a product’s design that other people tend to pass over’) and they strongly respond to beautiful things (e.g. ‘If a product’s design really “speaks” to me, I feel that I must buy it’). High CVPA individuals are more prone to use a visual style of processing, they more strongly desire to acquire objects that only few others possess, and the acquisition of beautiful objects becomes a central pursuit of their lives, closely linked to happiness and success (Bloch et al., 2003, Study 4). In the toaster study, CVPA moderated the overall evaluation, purchase intention, and the willingness to pay for the two products (Bloch et al., 2003, Study 7). Whereas for low CVPA individuals neither evaluation nor purchase intention varied significantly as a function of beauty, it made a large difference for high CVPA individuals. The same pattern, but not as pronounced was also apparent for willingness to pay. On average, high CVPA individuals were willing to pay $40.09 for the beautiful toaster; low CVPA individuals paid only $34.32. This study demonstrated individual differences in the importance we attach to beauty, and consequently in the value beautiful objects have. Besides those individual differences, situational aspects can determine whether beauty is valued or not. Ben-Bassat, Meyer and Tractinsky (2006), for example, attempted to measure the perceived value of beauty and usability with the help of an auction mechanism. Participants first used and rated versions of a software for text input, which differed in their usability and beauty. In the second part of the study, participants were required to perform a task, i.e. to input items with the help of the software. Task performance (number of items) was monetarily rewarded. Before the task, participants were asked to place a bid on the version of the software they would like to use for the input task. Successful bidders were allowed to use the preferred version of the software for the task. Bids had to be paid in real money. Interestingly, bids differed largely for the medium and high usability version (NIS 6.54

296

Product Experience

[approx. €1] and NIS 20.71 [approx. €3.50], respectively), but virtually no difference was found for the less and more beautiful versions. Participants even seemed to pay slightly less for the beautiful, high usability version than for the less beautiful, high usability version of the software. Thus, whether beauty has value depends on the context in which the product is used. In a highly efficiency-oriented context (as in Ben-Bassat’s study), individuals do not seem to place much value on beauty. In an unpublished study, Dieter Rhode and I further explored the impact of situational cues on the centrality of beauty. We presented a number of laptop desktops, which were meant to differ in beauty and usability. Each participant rated the usability of each desktop (four items, e.g. easy to use, concise), their beauty (four items, e.g. well-formed, beautiful look) and their overall adequacy (i.e. not adequate–adequate). However, participants received different background stories for their rating task. One group was told to imagine that they must later use the laptop for correcting a faulty and badly designed PowerPoint presentation under time pressure. The second group was told to imagine using the laptop for a series of important conference talks. The third group was told that the desktop would be installed on their own personal laptop. As expected, in the first group (revision of the presentation), the usability ratings were the single best predictor and explained 58% of the adequacy judgments’ variance. The further inclusion of beauty into the regression model explained only an additional 9% of the variance (67% in total). Albeit significant, it seems fair to conclude that beauty did not play much of a role for the adequacy judgments in this group. This was different for the second group (conference talks). Usability remained the single best predictor, but explained only 28% of the total variance. If beauty was added to the model, explained variance was increased by 20% (48% in total). In the last group (own laptop), beauty was the best predictor and explained 17% of the total variance. Usability added another 17% (34% in total). Thus, for a highly task-related context (group 1, revision of the presentation) beauty played only a minor role, which changed clearly given the context emphasizing self-presentation (group 2, conference talks) or personal identity (group 3, own laptop). In these cases, beauty mattered. All in all, beauty can add value to a product. The magnitude of this effect is likely to be moderated by personal, as well as situational aspects. Some, more visually oriented individuals may value beauty more than others. In addition, task- and efficiency-oriented contexts may call for less importance of beauty than, for example, social contexts.

4.2. Beauty as appealing to self-referential goals Typically, I distinguish between pragmatic and hedonic attributes of interactive products (e.g. Hassenzahl, 2003), where – broadly speaking – pragmatic attributes relate to action goals (either externally given or internally generated) and resulting tasks (e.g. ‘purchasing a flight ticket’) and hedonic attributes relate to self-advancement and self-presentation (are ‘self-referential’). Or as McGrath and Tschan (2004, p.49) put it, there are goals, ‘which are clearly related to action content (‘do goals’, [go shopping])’ and ‘more general goals that are related the person’s state or condition (‘be goals’, [be competent])’. In general ‘be goals’ are more abstract and more stable over time than ‘do goals’ (Carver and Scheier, 1998). As reported above, Hassenzahl (2004a) and Tractinsky and Zmiri (2006) found a substantial relation between judgments of beauty and hedonic attributes, such as ‘inclusive’, ‘presentable’, ‘brings me closer to people’, ‘positive message about user’, ‘communicates desirable image’ (see Section 3.2). These attributes capture the product’s perceived ability to communicate a favorable Self to relevant others. Self-presentation is clearly a ‘be goal’. Thus, one may argue that beauty is related to, signals, or is even a part of

Aesthetics in interactive products: Correlates and consequences of beauty

297

hedonic quality in products, which in turn primarily appeals to self-referential goals, i.e. ‘be goals’. Importantly, people treat products differently depending on whether they are perceived to be primarily hedonic or pragmatic (utilitarian, instrumental). Literature on consumer choice (Dhar and Wertenbroch, 2000; Borcherding and Hassenzahl, 2004), for example, showed hedonic aspects to be a moderator of the so-called endowment effect (e.g. Kahneman, Knetsch and Thaler, 1990). In short, individuals who own a particular object request more money to give it up than others (or even the same individuals) are willing to pay for it. This gap between selling and buying prices is a robust finding that is neither explained by strategic considerations (i.e. ‘buy low – sell high’), experimental artifacts nor inexperience with ‘market mechanisms’. The endowment effect may be interpreted as a form of ‘bonding’ to the object. In other words, higher selling prices result from a higher reluctance to part from the object. Indeed, Dhar and Wertenbroch (2000) found a higher preference for primarily hedonic objects in a forfeiture compared to an acquisition situation. In the acquisition situation, participants could either choose between a $7 gift certificate for computer disks (primarily pragmatic) or a music CD (primarily hedonic, Study 1). Fifty-four percent preferred the music certificate. In the forfeiture situation, however, participants got both certificates first and were then informed that there had been a procedural error and one of the certificates has to be given away. Interestingly, in this situation 84% preferred the music certificate. In another study (Hassenzahl and Borcherding, in preparation, Study 3), individuals were either asked to state their (hypothetical) maximum buying or minimum selling price for a wristwatch in the range of €0 to €50. (In the selling condition, individuals were asked to imagine they would own the watch.) For half of the participants, the watch was described as primarily pragmatic (i.e. useful, effective, accurate, reliable, robust); for the other half as primarily hedonic (i.e. exclusive, sporty, fashionable, beautiful, valuable, dynamic). Note that beauty was explicitly included in the hedonic description. Participants, who found either watch relatively appealing, were willing to pay about €30 for the primarily pragmatic watch, but only about €23 for the primarily hedonic watch. This changed, however, for selling prices. Here, participants were willing to sell the primarily pragmatic watch for about €29, but asked €32 for the primarily hedonic watch. Consistent with Dhar and Wertenbroch’s (2000) findings, the primarily hedonic object led to an endowment effect (⫹€9) – an indicator for bonding – which was not apparent for the pragmatic object. In other words, individuals may not be willing to pay much for beauty, but owning it, they don’t want to give it up either. The observed reluctance to spend money on something primarily hedonic (lowest price in the watch-study) may also stem from difficulties of justifying the expenditure for hedonic quality. In one study (Okada, 2005, Study 2) a certificate for a dinner in a restaurant (hedonic) was rated as more attractive than a certificate for groceries (pragmatic), although both certificates had the same nominal value of $50. In a combined choice situation, however, participants preferred the pragmatic alternative. Okada (2005) argued this to be the consequence of a justification process induced by the comparison of objects in the joint evaluation (choice) situation. In such a situation, individuals construct reasons for justifying their choice. It may be more difficult to envision reasons for primarily hedonic objects, because their benefits are often rather difficult to state or even quantify. In addition, hedonic alternatives are often viewed as wasteful and their acquisition or consumption is more likely to induce guilt (e.g. Prelec and Loewenstein, 1998). These aspects all lead to the effect that the acquisition (buying) of a hedonic object is harder to justify, and that it may not be chosen, although it is more appealing than an available alternative pragmatic object.

298

Product Experience

To summarize, I believe beauty to contribute to the hedonic quality of an object rather than to its pragmatic quality. This self-referential, social nature of beauty has interesting implications. On one hand, it may lead to a stronger bonding, as expressed by an endowment effect. A primarily beautiful object is initially less valued compared to a more useful product; however, if in one’s possession the perceived value of beauty increases. On the other hand, the acquisition of primarily beautiful objects may be harder to justify. Thus, although preferred, people may not choose the beautiful product.

4.3. Beautiful products work better In his book, Norman (2004a) made the claim that ‘attractive things work better’ (Chapter 1). He assumes the following mechanism for this somewhat surprising consequence of beauty. Beauty leads to positive mood; positive mood facilitates creative thinking and problem solving, which in turn improves task performance. No doubt this claim is immediately appealing to researchers in the field of beauty in HCI. Nevertheless, it calls for a careful and critical examination of the underlying assumptions. Starting from the beginning of the causal chain ‘beauty–positive affect–better task performance’, we may first ask whether a link between beauty and affect is likely. I assume beauty and positive affect to be related (Section 2). Specifically, a beautiful percept may elicit an affective response, which is in turn – if attributed to the product (integral affect, e.g. Pham et al., 2001) – used as an input to judgments of beauty and overall liking. Positive affect, presumably caused by the percept of the object, but not attributed to the object (incidental affect, e.g. Pham et al., 2001) will be experienced as mood (i.e. unattributed affect, Russell, 2003). Either way, beauty will have an impact on the initial affective state of the users. The link between positive affect and task performance is more questionable. In an extensive meta-analysis of happiness and positive affect, Lyubomirsky, King and Diener (2005) reviewed a total of 92 studies, testing the effect of a mood induction procedure on particular outcomes. The outcomes were further categorized into ‘positive perceptions of self and others’, ‘sociability’, ‘negotiation’, ‘prosocial behavior’, ‘physical wellbeing’, and ‘creativity and problem-solving’. The last category comes closest to task performance. In general, the observed median effect size for this category was the smallest (median weighted d ⫽ φ.25, largest was sociability, d ⫽ φ.56). In a more detailed account, Lyubomirsky and colleagues (2005) came to the conclusion that induced positive affect no doubt fosters originality and divergence in thinking. Concerning performance on complex mental tasks, however, results are rather mixed: ‘It appears that sometimes the performance of people in positive moods is superior, sometimes equal to, and at other times inferior to […] those in a negative mood’ (p. 839). The authors try to resolve these contradictions by referring to different underlying processing styles triggered by the different affective states (see Schwarz, 2001). According to this account, positive mood signals that everything goes well. Past successful strategies are likely to work, which results in a stronger reliance on preexisting successful knowledge structures. The processing becomes heuristic and holistic. In contrast, negative mood signals problems, i.e. a failure of the actual strategy, which in turn leads to a more analytic and detailed processing. Thus, depending on the properties of the task at hand, mood can result in better performance (if the task calls for a holistic approach, such as finding a novel solution to a problem) or worse performance (if the task calls for attention to details, such as solving syllogisms). The authors conclude: ‘People experiencing happy moods have potential deficits when it comes to problem solving, but they can overcome these deficits if they are motivated to perform well at the task’ (p. 840). Thus, the second part of Norman’s claim may be

Aesthetics in interactive products: Correlates and consequences of beauty

299

viewed with some reservation. First, it seems unclear whether the handling of interruptions in goal-directed behavior (i.e. overcoming a usability problem) calls for heuristic/ holistic or analytical/detailed thinking. Norman assumes the former because holistic thinking facilitates new, creative, original solutions to problems (i.e. ‘seeing the forest for the trees’), however, reliance on preexisting knowledge structures (Bless et al., 1996), one of the consequences of heuristic thinking, may be rather detrimental for overcoming usability problems. Secondly, effects reported in the literature are often obtained with the unobtrusive induction of positive affect. This type of affect is called incidental affect. The type of affect Norman assumes for his claim, however, is integral affect – a positive affective response to the product at hand, i.e. affect attributed to the product. It seems plausible that positive integral affect leads to a better evaluation of the product (as already argued for in Section 2), and maybe also that experienced affective reaction to the product colors mood. Nevertheless, this latter step must be assumed, if Norman’s assumed mechanism is to hold. Thirdly, as already pointed out by Schwarz (2001; see also Russell, 2003), affect’s primary function is to signal an individual’s actual state. It is monitored constantly and changes over time. If, for example, barriers to goal attainment arise, negative affective reactions are at the core to make the individual aware of the problems. So, we may experience positive affect induced by beauty at the beginning of the interaction, which in turn increases likelihood of exploration; however, the moment the first severe problem arises, affect may change considerably to the negative. Consequently, individuals engage in mood maintenance strategies. Handley and colleagues (Handley et al., 2004), for example, set participants into either a happy or neutral/sad mood. Participants were then asked to rank order eight different hypothetical video films according to their liking. They were further told that their most preferred film would be watched at the end of the experimental session. All film titles contained only neutral words (e.g. ‘The walk from home’), however, some neutral words were associated with positive words, others with negative words in an according task before the mood manipulation. Participants in a positive mood more strongly preferred films with a title which had been associated with positive words. Only 5% of the participants, however, reported mood maintenance as a motive of their preference. In the same vein, individuals in a positive mood bet less on a high-risk gamble (Isen and Patrick, 1983) and reveal a higher sensitivity to losses (Isen, Ashby and Nygren, 1988). Both were explained by the mood maintenance motive, which increases the sensitivity to anything that is capable of disrupting the current positive mood. Thus, to maintain our positive mood, we may rather stop interaction with a product that starts to become troublesome than to be more willing to compensate problems as suggested by Norman. To summarize, beauty presumably has affective consequences. However, affect is dynamic and subject to self-regulatory processes, which makes its impact on task performance less straightforward. Moreover, an increase in divergent thinking and creative problem solving may not be central to overcoming usability problems.

5. SUMMARY AND CONCLUSION Beauty had been long neglected. The challenge for HCI is to integrate beauty (and other hedonic aspects) into the field’s research practices and practitioners’ approaches to an empirically driven user-centered design (i.e. processes, methods). The present chapter attempts to facilitate this integration by defining the object of study, that is, beauty judgments, their correlates and consequences. First, it seems helpful to restrict beauty judgments to evaluations of a product’s visual Gestalt to avoid overlap with more general models of appealing products, user

300

Product Experience

experience, user satisfaction, or technology acceptance. Secondly, we may view beauty as primarily hedonic, i.e. more concerned with self-referential goals (‘be’) than action goals (‘do’). Nevertheless, beauty is often perceived first and relatively stable, which makes it an important starting point for inference processes. Estimating hard to observe attributes (e.g. usability) from more readily accessible attributes (e.g. beauty) is a complex process. However, as long as beauty contributes to value and the overall liking of the product, this general ‘positivity’ is likely to spill over to other attributes, without a necessarily causal relation between attributes. Thirdly, how much beauty matters to each of us may depend on individual differences and situations. Social context may call for beauty; strongly efficiency-related may not. Nevertheless, the link between beauty and task performance is surely worthy to be considered. The study of beauty in HCI may be a big challenge; more importantly, I believe it to be a big opportunity for the field. On the one hand, it opens up a wealth of interesting, stimulating, exciting and thought-provoking research questions. On the other hand, a better understanding of beauty is required to find suitable trade-offs between all the aspects making an interactive product appealing. This may be the most important competence of future practitioners of HCI (i.e. usability professional).

ACKNOWLEDGMENTS I’m very grateful to Paul Hekkert, Sascha Mahlke, Gitte Lindgaard and Noam Tractinsky for their helpful and thought provoking comments on an earlier draft of the chapter. (They may forgive my stubbornness concerning the one or other detail.)

REFERENCES Alben, L. (1996). Quality of experience: Defining the criteria for effective interaction design. Interactions, 3, 11–15. Ben-Bassat, T., Meyer, J. and Tractinsky, N. (2006). Economic and subjective measures of the perceived value of aesthetics and usability. ACM Transactions on Computer–Human Interaction, 13, 210–234. Bless, H., Clore, G. L., Schwarz, N., Golisano, V., Rabe, C. and Wölk, M. (1996). Mood and the use of scripts: Does a happy mood really lead to mindlessness? Journal of Personality and Social Psychology, 71, 665–679. Bloch, P. H., Brunel, F. F. and Arnold, T. J. (2003). Individual differences in the centrality of visual product aesthetics: Concept and measurement. Journal of Consumer Research, 29, 551–565. Borcherding, K. and Hassenzahl, M. (2004). Hedonic aspects support the endowment effect. In: Proceedings of the 19. Conference on Subjective Probability and Utility in Decision-Making (SPUDM–19). Carroll, J. M. (1997). Human–computer interaction: Psychology as a science of design. International Journal of Human–Computer Studies, 46, 501–522. Carver, C. S. and Scheier, M. F. (1998). On the self-regulation of behavior. Cambridge: Cambridge University Press. Chernev, A. and Carpenter, G. (2001). The role of market efficiency institution in consumer choice: A case of compensatory inferences. Journal of Marketing Research, 38, 349–361. Davis, F. D. (1993). User acceptance of information technology: system characteristics, user perceptions and behavioral impacts. International Journal of Man–Machine Studies, 38, 475–487. De Angeli, A., Sutcliffe, A. and Hartmann, J. (2006). Interaction, usability and aesthetics: What influences users’ preferences? In: Proceedings of Designing Interactive Systems (DIS06), pp. 271–280. New York: ACM Press. Dhar, R. and Wertenbroch, K. (2000). Consumer choice between hedonic and utilitarian goods. Journal of Marketing Research, 37, 60–71. Dion, K. K., Berscheid, E. and Walster, E. (1972). What is beautiful is good. Journal of Personality and Social Psychology, 24, 285–290.

Aesthetics in interactive products: Correlates and consequences of beauty

301

Eagly, A. H., Ashmore, R. D., Makhijani, M. G. and Longo, L. C. (1991). What is beautiful is good, but …: A meta-analytic review of research on the physical attractiveness stereotype. Psychological Bulletin, 110, 109–128. Ford, G. T. and Smith, R. A. (1987). Inferential beliefs in consumer evaluations: An assessment of alternative processing strategies. Journal of Consumer Research, 14, 363–371. Frohlich, D. M. (2004). Beauty as a design prize. Human Computer Interaction, 19, 359–366. Goldstein, E. B. (1989). Sensation and perception (3rd Ed.). Belmont, CA: Wadsworth. Handley, I. M., Lassiter, G. D., Nickell, E. F. and Herchenroeder, L. M. (2004). Affect and automatic mood maintenance. Journal of Experimental Social Psychology, 40, 106–112. Hassenzahl, M. (2002). The effect of perceived hedonic quality on product appealingness. International Journal of Human–Computer Interaction, 13, 479–497. Hassenzahl, M. (2003). The thing and I: understanding the relationship between user and product. In: M. Blythe, C. Overbeeke, A. F. Monk and P. C. Wright (Eds.) Funology: From Usability to Enjoyment, pp. 31–42. Dordrecht: Kluwer. Hassenzahl, M. (2004a). The interplay of beauty, goodness and usability in interactive products. Human Computer Interaction, 19, 319–349. Hassenzahl, M. (2004b). Beautiful objects as an extension of the Self. A reply. Human Computer Interaction, 19, 377–386. Hassenzahl, M. and Borcherding, K. (in preparation). The endowment effect for hedonic and utilitarian objects. Hassenzahl, M., Platz, A., Burmester, M. and Lehner, K. (2000). Hedonic and ergonomic quality aspects determine a software’s appeal. In: T. Turner and G. Szwillus (Eds.) Proceedings of the CHI 2000 Conference on Human Factors in Computing, pp. 201–208. New York: ACM, Addison-Wesley. Hassenzahl, M. and Tractinsky, N. (2006). User experience – a research agenda [Editorial]. Behavior and Information Technology, 25, 91–97. Hekkert, P. (2006). Design aesthetics: Principles of pleasure in design. Psychology Science, 48, 157–172. Isen, A. M., Ashby, F. G. and Nygren, T. E. (1988). Influence of positive affect on subjective utility of gains and losses: It is just not worth the risk. Journal of Personality and Social Psychology, 55, 710–717. Isen, A. M. and Patrick, R. (1983). The effect of positive feelings on risk taking: When the chips are down. Organizational Behavior and Human Decision Processes, 31, 194–202. Jacobsen, T., Buchta, K., Köhler, M. and Schröger, E. (2004). The primacy of beauty in judging the aesthetics of objects. Psychological Reports, 94, 1253–1260. Johnson, R. D. and Levin, I. P. (1985). More than meets the eye: The effects of missing information on purchase evaluations. Journal of Consumer Research, 12, 169–177. Kahneman, D., Knetsch, J. L. and Thaler, R. H. (1990). Experimental tests of the endowment effect and the coase theorem. Journal of Political Economy, 98, 1325–1348. Kardes, F. R., Posavac, S. S. and Cronley, M. L. (2004). Consumer inference: A review of processes, bases, and judgment contexts. Journal of Consumer Research, 14, 230–256. Kurosu, M. and Kashimura, K. (1995). Apparent usability vs. inherent usability. In: Proceedings of the CHI 1995 Conference on Human Factors in Computing. Extended abstracts, pp. 292–293. New York: ACM Press, Addison-Wesley. Lalomia, M. J. and Sidowski, J. B. (1990). Measurements of computer satisfaction, literacy, and aptitudes: A review. International Journal of Human–Computer Interaction, 2, 231–253. Langlois, J. H., Kalakanis, L., Rubenstein, A. J., Larson, A., Hallam, M. and Smoot, M. (2000). Maxims or myths of beauty? A meta-analytic and theoretical review. Psychological Bulletin, 126, 390–423. Lavie, T. and Tractinsky, N. (2004). Assessing dimensions of perceived visual aesthetics of web sites. International Journal of Human–Computer Studies, 60, 269–298. Leder, H., Belke, B., Oeberst, A. and Augustin, D. (2004). A model of aesthetic appeciation and aesthetic judgements. British Journal of Psychology, 95, 489–508. Lindgaard, G., Fernandez, G., Dudek, C. and Brown, J. (2006). Attention web designers: You have 50 milliseconds to make a good impression. Behavior and Information Technology, 25, 115–126. Lindgaard, G. and Dudek, C. (2002). User satisfaction, aesthetics and usability: Beyond reductionism. In Proceedings 17th. IFIP (International Federation for Information Processing) World Computer Congress, pp. 231–246. Dordrecht: Kluwer. Lindgaard, G. and Whitfield, T. W. A. (2004). Integrating aesthetics within an evolutionary and psychological framework. Theoretical Issues in Ergonomics Science, 5, 73–90. Lingle, J. H. and Ostrom, T. M. (1979). Retrieval selectivity in memory-based impression judgments. Journal of Personality and Social Psychology, 37, 180–194. Lyubomirsky, S., King, L. and Diener, E. (2005). The benefits of frequent positive affect: Does happiness lead to success? Psychological Bulletin, 131, 803–855.

302

Product Experience

Mahlke, S. (2002). Factors influencing the experience of web site usage. In: Proceedings of the CHI 2002 Conference on Computer–Human Interaction. Extended Abstracts, pp. 846–847. New York: ACM Press, Addison-Wesley. McCarthy, J. and Wright, P. C. (2004). Technology as Experience. Cambridge, MA: MIT Press. McGrath, J. E. and Tschan, F. (2004). Temporal matters in social psychology. Washington, DC: APA. Monk, A. (2004). The product as a fixed-effect fallacy. Human–Computer Interaction, 19, 371–375. Ngo, D. C. L. and Byrne, J. G. (2001). Application of an aesthetic evaluation model to data entry screens. Computers in Human Behavior, 17, 149–185. Nielsen, J. (1993). Usability engineering. Boston, San Diego: Academic Press. Norman, D. (2004a). Emotional design: Why we love (or hate) everyday things. New York: Basic Books. Norman, D. (2004b). Introduction to this special section on beauty, goodness, and usability. Human–Computer Interaction, 19, 311–318. Okada, E. M. (2005). Justification effects on consumer choice of hedonic and utilitarian goods. Journal of Marketing Research, 42, 43–53. Overbeeke, C. J. and Wensveen, S. A. G. (2004). Beauty in use. Comment on Hassenzahl’s ‘The interplay of beauty, goodness and usability in interactive products’. Human–Computer Interaction, 19, 367–369. Park, S., Choi, D. and Kim, J. (2004). Critical factors for the aesthetic fidelity of web pages: empirical studies with professional web designers and users. Interacting with Computers, 16, 351–376. Pham, M. T., Cohen, J. B., Pracejus, J. W. and Hughes, G. D. (2001). Affect monitoring and the primacy of feelings in judgment. Journal of Consumer Research, 28, 167–188. Prelec, D. and Loewenstein, G. (1998). The red and the black: Mental accounting of savings and debt. Marketing Science, 17, 4–28. Russell, J. A. (2003). Core affect and the psychological construction of emotion. Psychological Review, 110, 145–172. Russo, B. and De Moraes, A. (2003). The lack of usability in design icons: an affective case study about Juicy Salif. In: Proceedings of the 2003 International Conference on Designing Pleasurable Products and Interfaces, pp. 146–147. New York: ACM Press. Schenkman, B. N. and Jönsson, F. U. (2000). Aesthetics and preferences of web pages. Behaviour and Information Technology, 19, 367–377. Schneider, D. J. (1973). Implicit personality theory. A review. Psychological Bulletin, 79, 194–309. Schwarz, N. and Clore, G. L. (1983). Mood, misattribution, and judgments of well-being: Informative and directive functions of affective states. Journal of Personality and Social Psychology, 45, 513–523. Schwarz, N. (2001). Feelings as information: Implications for affective influences on information processing. In: L. L.Martin and G. L. Clore (Eds.) Theories of mood and cognition, pp. 159–176. Mahwah, NJ: Lawrence Erlbaum. Thorndike, E. L. (1920). A constant error on psychological rating. Journal of Applied Psychology, 4, 25–29. Tractinsky, N. (1997). Aesthetics and apparent usability: empirically assessing cultural and methodological issues. In: Proceedings of the CHI 1997 Conference on Human Factors in Computing, pp. 115–122. New York: ACM, Addison-Wesley. Tractinsky, N., Cokhavi, A., Kirschenbaum, M. and Sharfi, T. (2006). Evaluating the consistency of immediate aesthetic perceptions of web pages. International Journal of Human–Computer Studies, 64(11), 1071–1083. Tractinsky, N., Cokhavi, A. and Kirschenbaum, M. (2004). Using ratings and response latencies to evaluate the consistency of immediate aesthetic perceptions of web pages. In: Proceedings of the Third Annual Workshop on HCI Research in MIS, pp. 40–44. Tractinsky, N., Katz, A. S. and Ikar, D. (2000). What is beautiful is usable. Interacting with Computers, 13, 127–145. Tractinsky, N. and Zmiri, D. (2006). Exploring attributes of skins as potential antecedents of emotion in HCI. In: P. Fishwick (Ed.) Aesthetic computing, pp. 405–421. Cambridge, MA: MIT Press. van der Heijden, H. (2003). Factors influencing the usage of websites: the case of a generic portal in The Netherlands. Information and Management, 40, 541–549. Venkatesh, V., Morris, M. G., Davis, G. B. and Davis, F. D. (2003). User acceptance of information technology. Towards a unified view. MIS Quarterly, 27, 425–478. Wegener, D. T. and Petty, R. E. (1997). The flexible correction model: The role of naive theories of bias in bias correction. In: M. P. Zanna (Ed.) Advances in experimental social psychology, pp. 141–208. San Diego: Academic Press. Zhang, P. and Li, N. (2005). The importance of affective quality. Communications of the ACM, 48(9), 105–108.

10

PRODUCT AESTHETICS PAUL HEKKERT Delft University of Technology, Delft, The Netherlands

HELMUT LEDER University of Vienna, Wien, Austria

1. INTRODUCTION In 2003, Lidwell, Holden and Butler published a well-documented collection of 100 universal principles of design. Among these are 28 principles explaining ‘How can I increase the appeal of a design?’ These principles, laws, or guidelines deal with the Golden Ratio, similarity, savannah preference, symmetry and color; principles that will also appear in this chapter. Most of these principles have for centuries been applied in the arts, and have over the last century been uncovered and tested in psychological experiments. The authors claim that the application of such principles ‘increases the probability that a design will be successful’ (Lidwell et al., 2003, p. 11). We are tempted to adopt this claim, but want to take it a little further. Understanding why people are aesthetically attracted to some properties or patterns over others will support designers to make founded decisions on the attractiveness of their design. Over the past ten years, the first author has given many lectures on visual aesthetics to students of industrial design. The main message of these lectures always was: People may and do differ extensively in their aesthetic reactions to objects; these reactions as well as the differences are not arbitrary, but lawful. Contrary to what the popular expression ‘de gustibus non est disputandum’ holds, there is accounting for taste! Does this mean we can (already) explain all varieties in aesthetic preference? Of course we cannot. There are still many unresolved issues and unpredicted (but not unpredictable) exceptions. But, after more than 100 years of theorizing and experimentation, we have come to understand quite a bit about the drivers of people’s aesthetic responses to the things around us in general and designed artifacts in particular. This chapter aims to bring together these insights. Product Experience Copyright © 2008 Elsevier Ltd.

259

260

Product Experience

1.1. Aesthetics ‘Aesthetics’ is a very old concept, rooted in the Greek word aisthesis that can be translated as understanding through sensory perception. Only in the eighteenth century the concept started to be used in the way we will use it here, referring to sensory pleasure and delight (Goldman, 2001). Recently, the first author has argued that such a definition of aesthetics, i.e. the pleasure attained from sensory perception, is most appropriate in that it clearly separates aesthetic phenomena from other types of experience, such as the construction of meaning and emotional responses (Hekkert, 2006). In adopting this definition, some misunderstandings in the use of the concept aesthetics become salient, and these will now be briefly discussed. Aesthetic is not restricted to art or artistic expressions – Many artistic expressions, like works of art, music and designs, are aesthetic in the sense that they can evoke pleasure in the observer or user. But other, non-artistic phenomena, such as people, landscapes, and sunsets can also be aesthetic in that their appearance can strike us as beautiful or attractive. Aesthetic is not limited to the visual domain – The visual arts have clearly dominated Western art and, as a result, the concept of aesthetics has often been used as synonymous for visual beauty. If we, however, agree that aesthetics refers to sensory pleasantness in general, things can also be aesthetic or pleasant to listen to, touch, smell, or taste. In Section 4 we will discuss some aesthetic principles that apply to non-visual domains. Aesthetic is not a matter of styling (only) – In product design we often speak of aesthetics in relation to the final surface treatment of a design or its styling. The aesthetic principles in the next sections will hopefully make clear that all product properties can contribute to the sensory pleasure that is evoked. Making a product aesthetic is clearly not something you can start to work on after most of the design is finished. Aesthetic pleasure is not an emotion – This is probably the most controversial implication of our definition. Many scholars in the field of emotion have been theorizing about so-called aesthetic emotions, mostly referring to ‘normal’ emotions, like interest, fascination and surprise, that often take place in, but are not restricted to, encounters with works of art (see e.g. Silvia, 2005). Whether these emotions are a special class or no emotions at all has been subject to some debate (e.g. Frijda, 1988, 1989; Lazarus, 1991). Following our position, an emotion per se simply cannot be aesthetic. An aesthetic response is limited to the gratification that comes from sensory perception of an object, and has no implications for any of our concerns, the class of dispositional states that is so fundamental to our emotions. In short, for an emotion to be evoked, some concern, such as a goal or an expectation, must either be violated or satisfied (e.g. Scherer, Schorr and Johnstone, 2001; see also Chapter 15 for an extensive treatment of appraisal theory). An aesthetic response, however, is ‘disinterested’ (Kant, 1952) or distanced (Bullough, 1912) in that no motives other than perceiving the object of perception ‘as such’ are at stake. The pleasure ‘simply’ results from the act of perception itself. This certainly does not mean that an aesthetic experience could not result in a (positive) emotion, or that responses to art cannot be emotionally moving. Most people experience strong emotional reactions when they listen to their favourite music, as was shown in studies by Blood and Zatorre (2001). How and when aesthetic responses lead to what emotions is a complex process that requires a deeper understanding of the appraisal processes underlying emotions. Aesthetic is not an aspect, property or element of something – Following our definition, any property can elicit an aesthetic response, as long as that property is perceived as pleasant through the stimulation of one of the senses. Although we will show that some properties will more likely evoke such responses than others – and are for that reason

Product aesthetics

261

often coined ‘aesthetic’ – it is theoretically (and empirically) impossible to defend that a property or element is aesthetic.

1.2. Research in aesthetics Although our definition of aesthetics is to some degree limiting, most of the research done in the area of experimental aesthetics since the pioneering work of Fechner (1876) is relevant for our overview. Much of this research focused on finding, mostly visual, properties of objects, whether simple patterns, artworks or designed objects, determining aesthetic preference. These properties are generally classified into three classes: Psychophysical, organizational, and meaningful properties (e.g. Berlyne, 1971; see Hekkert, 1995 for an overview). The psychophysical properties are the formal qualities of objects, such as their intensity, size and color (in terms of hue, saturation, brightness), or, generally speaking, properties that can be quantified. Aesthetic effects of these properties are highly relational and contextual, as we will show in Section 2. In isolation, the most interesting findings come from color studies. It has often been demonstrated, for humans of many cultures and even for animals, that hues are preferred in the order blue, green or red, and yellow (McManus, Jones and Cottrell, 1981). Furthermore, the three color dimensions, hue, saturation, and brightness, differ with respect to their impact on aesthetic preference. Contrary to what many would suspect, variations in hue only explain a small amount of the variance in judgments of color pleasantness; brightness seems to be somewhat more important, and saturation determines by far the most variance (Smets, 1982). The two other classes of properties, organizational and meaningful properties, have been studied more extensively and will be discussed in Sections 2 and 3. In this discussion, we will confine ourselves as much as possible to studies involving design objects as stimulus material. As will be shown, findings from these studies often suggest universal agreement in aesthetic pleasure. In Section 4 we try to explain why and under what conditions people of different times and cultures aesthetically prefer the same properties, and not only visually. Despite these universal principles, people can differ considerably in matters of taste. Section 5 is devoted to some explanations that may account for this variability. Section 6 closes with some conclusions and implications for designers and the field of design.

2. ORGANIZATIONAL PROPERTIES Our visual system is tuned to organize information, to bring structure or order in the wealth of information that reaches our retina. Psychology of perception has achieved a good understanding of how our perceptual system makes sense of our environment by analyzing edges, contours, blobs, and basic geometrical shapes (e.g. Marr, 1982; Biederman, 1987). However, in order to represent what surrounds us we, for example, need to perceive which elements belong to the same object. Various principles have been proposed that seem to be fundamental to how this organization unfolds (see also Chapter 1). Elements that look similar in color, size, or shape, are seen as belonging together (principle of similarity), a line that is interrupted and continued later on is seen as one line (principle of good continuation), and we tend to make the most likely or economically efficient interpretation of a pattern (law of Prägnanz). These are examples of so-called Gestalt principles or laws of perceptual organization and these do not only explain why we see what we see, but also why we prefer to see certain patterns over others (see e.g. Hekkert, 2006; Ramachandran

262

Product Experience

and Hirstein, 1999). Simply put, we like to look at patterns that allow us to see relationships or create order. The generality of this assumption will be further investigated in Section 4. Below, we first look at some organizational properties that have been central in aesthetic research. Although the present chapter is concerned with aesthetics in the context of product design, it needs to be mentioned that researchers often tend to investigate properties of objects in isolation. While this gives them control over the source of changes in appreciation, it also leaves the question to what extent the variation of only one property, such as ‘visual contrast’, contributes to our aesthetic experience derived from encounters with everyday objects such as cars, fashion designs or sculptures. In consequence, although we will discuss some properties of objects that are preferred over well-defined others, we will particularly focus on those properties that are relevant for the perception and appreciation of products.

2.1. Unifying properties Order, balance or harmony, symmetry and ‘good’ proportion are omnipotent in products. Only rarely does a designer allow himself to challenge these unifying properties, to disrupt order, create misbalance or asymmetry, or design objects that are badly proportioned. If he does, he is either a bad designer or has very good reasons to do so (see Section 6). These principles are used to make a design coherent and orderly and, therefore, pleasant to look at. Balance Eye movement studies have shown what happens when the balance in a visual composition is distorted. Locher and his colleagues (e.g. Locher, Overbeeke and Stappers, 2005; Nodine, Locher and Krupinski, 1993; see Locher, 2006 for an overview) examined the scanpaths of people looking at original versions of paintings and versions in which the original composition was somehow altered, either by leaving out certain elements or changing the distribution of ‘weight’ in, for example, a typical Mondrian painting. Scanpaths of people looking at distorted versions revealed more eye movements (saccades) and less fixations, interpreted as an indication of the observer’s desperate attempt to detect order and balance in the distorted composition. This interpretation is supported by findings from other studies in which pictorial compositions where systematically changed (Boselie, 1992; Hekkert and van Wieringen, 1996). Both these studies showed that changing an original, and presumably balanced, painting leads to a decrease in preference ratings, especially among untrained viewers. Together these findings reveal that people do have sensitivity for a balanced composition. ‘Good’ proportion Whereas it is clear that an orderly, balanced or symmetrical design is aesthetically pleasant, it is less clear what proportion should be considered ‘good’ or aesthetically superior. For centuries, people believed that a ratio according to the golden section deserved this special status, but a wealth of empirical studies testing its special attractiveness yielded ambiguous results (see for an overview e.g. Berlyne, 1971; Hekkert, Peper and van Wieringen, 1994; McWhinnie, 1987).1 At most, ratios close to the golden section seem to 1 For those unfamiliar with the golden section ratio, this ratio is obtained when the ratio of the shortest to the longest of two lengths, such as in a rectangle or a cross, equals the ratio of the longest to the sum of the two. The numerical value of this ratio, often denoted as ␸, is approximately 1.618 (or its reciprocal ␾.618).

263

Product aesthetics

be preferred over other ratios, but this could easily be a range effect (Godkewitsch, 1974) or an effect of averaging (Plug, 1980), obscuring great intersubject variability (Hekkert et al., 1994; McManus, 1980). Next to the ratios in the vicinity of the golden section, the square was also often found to be a preferred ratio (McManus, 1980). As we concluded earlier, the golden section ratio probably has ‘obtained this special attention mainly thanks to its unchallenged mathematical beauty’ (Hekkert et al., 1994, p. 186). As noted in the beginning of Section 2, studying properties in isolation probably tells us little about the effects of these properties in the context of design objects. Given the high interrater variability, one may therefore question whether the search for proportions of special attractivity per se is worthwhile. As Hekkert et al. (1994) concluded, ‘instead of continuing the search for proportions of special attractivity in their own right, it is more valuable to study proportionality of something’ (p. 200). Following this suggestion, Hekkert (1995, chapter 3) started a series of experiments on proportion preferences in context. As could be easily predicted, he found that aesthetic preference for particular rectangular proportions highly depended on the type of object the rectangle represented, such as a window, a cabinet door, or a bathroom tile (see Figure 10.1). More interestingly, preference was linearly related to the rated commonness of the proportion (Figure 10.2), a measure of familiarity (see Section 3.1). This finding was replicated in a subsequent experiment (Experiment 2, p. 73) with three (at that time) unknown and especially designed products (a portable smoke-filter, a subwoofer, and an electromagnetic radiation reducer), for which the exposure frequency was systematically varied. Other research along these lines has been done in the area of packaging, further showing that proportions of invitation cards and packages for grocery products affect consumer perception, preferences, and purchase intentions (Raghubir and Greenleaf, 2006). Symmetry Symmetry in simple patterns can be produced quite easily; the designer has to choose one or more axes at which the design is mirrored. Objects that are mirrored along one axis can easily be recognized as being symmetrical, and indeed are often seen as pleasant. For example, symmetrical faces are preferred over non-symmetrical ones (Grammer and Thornhill, 1994) and symmetrical abstract patterns are often seen as more beautiful (Jacobsen and Höfel, 2003). The reasons for a preference of symmetry are not fully understood. ‘Reading’ a symmetrical object is much easier than reading asymmetrical ones. Once you have seen one half, you know what the other half is like. Thus, an important part of symmetry preference might be due to ease of processing (Reber, Schwarz and Winkielman, 2004). Concerning beauty of faces it has been argued that

Window (1/1)

Picture frame (8/5)

Cabinet door (1/2)

Book (4/5)

Bathroom tile (4/3)

FIGURE 10.1 Examples of rectangles representing a product (from Hekkert, 1995, p. 69).

264

Product Experience

9

Books

8

Mean ranking

7 6 5 4 3 2 1 3/8

1/2

5/8

4/5

1/1 Ratio

5/4

8/5

2/1

8/3

9 Bathroom tiles

8

Mean ranking

7 6 5 4 3 2 1 3/8

1/2

5/8

4/5

1/1 Ratio

5/4

8/5

2/1

8/3

FIGURE 10.2 Mean rankings for two of the products used in Hekkert (1995, p. 72). Solid line: commonness ratings; dotted line: attractiveness ratings.

symmetry indicates a healthy development and therefore is an indicator of positive genetic make-up (Thornhill and Gangestad, 1993). Others have argued that symmetry makes faces attractive because they are more prototypical, where prototypicality is the underlying attractive feature (Rhodes, 2006). In Section 4 we will look more closely at such explanations.

2.2. Complexity and variety If humans would just look for orderly and balanced patterns, our world and our designs would be rather simple, and presumably be experienced as boring. In some circumstances, we also seem to search for complexity and variety, a type of behavior coined diversive exploration (Berlyne, 1966). According to Berlyne’s collative-motivation model, patterns are preferred for their ability to generate arousal (Berlyne, 1971). Visual patterns with low arousal potential are not stimulating and leave the observer indifferent; patterns with very high arousal potential are too difficult to grasp and are considered unpleasant. Preferred are patterns with an arousal potential at a medium (or optimum) level, leading to the famous prediction of an inverted U-shaped function between hedonic tone (pleasantness) and arousal potential. Since collative properties, like complexity and variety, contribute most to the arousal potential of a design, they have dominated research in aesthetics.

Product aesthetics

265

Although ample evidence was found for an inverted U-shaped relationship between preference and complexity (e.g. Berlyne, 1970; Smets, 1973; Walker, 1980), other, mainly monotonic, functions between these two variables were observed as well (e.g. Frith and Nias, 1974; Walker, 1980). This was especially true when the stimulus material was more meaningful, such as real artworks, as opposed to the simple, artificial stimuli that were used in most studies in favour of Berlyne’s model. It was concluded that Berlyne’s model has limited explanatory value when ecologically valid objects, like products, are evaluated (see Hekkert, 1995; Martindale, 1984), a limitation already acknowledged by Berlyne (1971) himself. However, Berlyne’s prediction reflects a more general principle of aesthetic pleasure: Unity in variety.

2.3. Unity in variety If people are attracted to order and unity, whereas they also (occasionally) seek complexity and variety, it is easy to predict that a balance between these opposing forces would lead to maximum pleasure. This principle of unity in variety was already known to the Greeks and has been most influential in the field of aesthetics ever since (see e.g. Berlyne, 1971; Fechner, 1876). The principle holds that the greatest pleasure or beauty is arrived at by as much variety or complexity as possible with a maximum of unity or order. Attempts to formalize this principle in simple functions of order (O) and complexity (C) failed to explain preference ratings of simple polygons (see McWhinnie, 1968 for an overview of these information-theoretic approaches). In a classic study, Boselie and Leeuwenberg (1985) developed a more subtle formula, taking into account that patterns can be regular in more than one way. These additional regularities, not accounted for by the simplest interpretation of a pattern, determine a pattern’s unity (R); the free parameters that are not specified by these additional regularities represent the irregularity or variety of a pattern (P). The beauty of a pattern is arrived at by subtracting P from R. This formula proved to be adequate to predict the rated beauty of simple polygonal figures. Since products, as all real-life stimuli, embody an endless number of regularities, it is hard to predict which of them will be perceived. A mathematical description of product preference on the basis of such measures therefore seems a pointless exercise. But qualitative descriptions of a design’s unity and variety may help to see its formal attractiveness (see for examples, Hekkert, 2006). Conjunctive ambiguity Boselie and Leeuwenberg (1985) based their mathematical model on the principle of conjunctive ambiguity, which is another principle proposed to be conducive to aesthetic preference, highly related to unity in variety (e.g. Arnheim, 1974; Berlyne, 1971). When an ambiguous pattern can be visually interpreted in several ways, conjunctive ambiguity concerns the case where the separate interpretations are compatible and jointly effective. As such, it is opposed to the beauty-reducing principle of disjunctive ambiguity where alternative interpretations are mutually exclusive (as in the famous duck-rabbit drawing). Hekkert (2006) describes Jean Nouvel’s building Institut du Monde Arabe in Paris as a good design example of conjunctive ambiguity (Figure 10.3): The interpretation of this building at a global level (as an Islamic weave pattern) is different from, but fully compatible with its interpretation at a local level (seeing that the holes are actually shutters that regulate the amount of sunlight entering the building). Maximum effect for minimum means Conjunctive ambiguity can be seen as a special case of ‘maximum effect for minimum means’, a general principle that explains aesthetic quality in a wide variety of domains. The principle is economy-driven: We prefer solutions, ideas, formulas and the like that

266

Product Experience

FIGURE 10.3 Institut du Monde Arabe by Jean Nouvel.

consist of as few elements or parameters as possible, while solving or explaining a range of problems or phenomena (e.g. Boselie and Leeuwenberg, 1985). For the same reason we can also say that a particular engineering solution, like a bridge, or a car suspension, is aesthetic; it only uses a limited number of constructive elements to solve all the problems the construction was meant to overcome. The general acclaim for the original Mini is, for example, based on this principle. By literally striving for minimalism in space and material – to realize a car that would be affordable to many – the designers introduced a range of innovations, such as a transversal engine, 10-inch wheel rims, and an ultra-compact wheel suspension. Analogously, an explanation or theory can be more attractive than others, and will therefore be selected, when it uses fewer parameters to explain the same phenomenon or more phenomena, a principle also known as Occam’s razor (or the principle of parsimony). Since these aesthetic solutions and formulas are, by definition, also more economical or efficient – clever we could say – aesthetic sensitivity is important for scientists, engineers, and designers to create and recognize the most beautiful idea or solution. Designers often refer to this principle in preferring minimal solutions as exemplified by the iPod shuffle, an MP3 player in a tiny white box that only has a connector for an earplug, a USB connection for battery power and uploading songs, and a clickwheel for navigation, but no display at all (Figure 10.4).

3. MEANINGFUL PROPERTIES Whereas the organizational properties in the previous section always require a beholder to perceive the extent to which they are present in a design, they can in principle be measured and formalized. The properties considered in this section are by definition subjective and are thus not properties of things, but rather properties as we perceive them. Based on our knowledge and previous experiences, we qualify something as familiar or novel, typical or strange, original or outdated. Since people in a particular culture may have considerable overlap in their backgrounds, the formal attributes on which these meanings are based may be rather consistent over people. As a result, we tend to attribute the meaning perceived to these characteristics. Logically, however, the degree to which something is perceived as novel or familiar is independent of the presence of an attribute as such. As we will show here, these meaningful ‘properties’, determining the so-called diagnosticity of a pattern, have a big impact on our aesthetic preference.

Product aesthetics

267

FIGURE 10.4 The Apple iPod shuffle.

3.1. Familiarity and prototypicality Whenever we encounter an object, and this also holds for design objects, we (try to) classify it by comparing it to objects we know or have seen before. The idea that we like what we know has had an apparent appeal to psychologists for a long time. However, there are different ways in which familiarity might affect the aesthetic appeal of an object. In the next paragraphs we discuss those ways that have found empirical confirmation and were shown to be important in the appreciation of complex, real-life objects. Familiarity While William James and Gustav Fechner, both pioneers of psychology in the nineteenth century already assumed that ‘familiarity breeds liking’, it was in 1968 that Robert Zajonc provided a systematic empirical study of this phenomenon. In a seminal paper he reported evidence, from a number of sources, that mere exposure to a stimulus increases its aesthetic appreciation. Not only did he show that words with a positive connotation are far more frequent in language, he also experimentally varied the number of times that faces, Chinese characters or pseudo-Turkish words were repeated, and found that with increasing repetition the objects were liked more. He discussed his findings as a general principle of aesthetic appreciation that can explain why we often like the people we know, why we feel comfortable in our homes, and stick to the brand of a car we own. Thus, in order to create objects that people like, a straightforward recommendation could be to refer to existing, familiar solutions. Preferring things that are familiar obviously has evolutionary advantages in that it leads to safe choices. In a world full of inherent dangers it might be sensible (or adaptive; see Section 4) to stick to the familiar and not expose oneself to strange and maybe harmful and threatening alternatives (Bornstein, 1989). Recently, an alternative has been proposed to this evolutionary explanation. Repeated exposure changes the way things are processed,

268

Product Experience

the way they are perceived, classified, and recognized. Simply spoken, repetition or familiarity makes perceptual and cognitive processing easier and somehow more fluent, and this fluency is intrinsically pleasant (Reber et al., 2004). The more fluently perceivers can process an object, the more positive their aesthetic response will be. The important implication of this explanation is that fluency increases liking, not because it is a property of the stimulus, but because it is a property of the processing dynamics of the perceiver. Reber et al. (2004) thus believe that we somehow ‘perceive ourselves’ when we perceive and evaluate the objects around us, and attribute this ease of processing to the appreciability of the object. However, repeated exposure has its limitations and will at a certain point (often after 20 repetitions) lead to over-exposure and saturation, and, consequently, boredom (see also Section 5.4). Furthermore, Bornstein’s review (1989) showed that the effect of repeated exposure depended on the type of stimulus, being strongest for simple patterns, weak for real objects/persons, and was often not found with artworks and complex drawings. As already discussed under ‘Good’ proportion above, Hekkert (1995) demonstrated a strong linear relationship between attractiveness ratings and exposure frequency for three unknown products. This effect was, however, less strong when the product was presented in an ‘aesthetic’ (conspicuous) context as opposed to a ‘neutral’ (inconspicuous) one (see Figure 10.5). These results suggest that the mere exposure effect is not equally strong for all objects and in all conditions. Nevertheless, the fact that the ‘mere-exposure effect’ is strongest when people are not aware that the stimulus to evaluate has been shown several times (e.g. Murphy and Zajonc, 1993) indicates that the effect is automatic and difficult to suppress. Prototypicality In order to recognize things, we tend to classify all things into groups of objects which share some properties. For those object categories for which there are many exemplars, such as human faces, cars, toasters, or cubist paintings, it seems that through experience we build so-called prototypes. These are typical representations which allow us to trigger appropriate responses and which summarize information that all objects of that 5.0

Context

Aesthetic Neutral

4.8

Mean attractiveness

4.6 4.4 4.2 4.0 3.8 3.6 3.4 0

2

4

8

16

Frequency

FIGURE 10.5 Mean attractiveness ratings of objects as a function of exposure frequency and context (from Hekkert, 1995, p. 77).

Product aesthetics

269

class have in common. This is not to say that the prototype is represented by a certain category member; a prototype is ‘simply a convenient grammatical fiction; what is really referred to are judgments of degree of prototypicality’ (Rosch, 1978, p. 40). Whitfield and his colleagues (1983; Whitfield and Slatter, 1979) carried out pioneering work concerning the effect of prototypicality on preference. They directly tested a preference-forprototypes model (see also Martindale, 1984) against Berlyne’s ‘collative-motivation’ model predicting an inverted U-shaped relationship between preference and novelty/ complexity. They measured appreciation for different kinds of chairs that varied in prototypicality as a result of belonging to different styles, assuming that ‘Georgian chairs’ are more prototypical than ‘Modern style’ chairs, and these more prototypical than ‘Art Nouveau’ chairs. Moreover, the authors directly measured subjective impressions of typicality (as well as complexity and novelty) for all chair models investigated. As expected, more prototypical chairs were liked better, and typicality was negatively correlated with novelty, indicating that prototypicality is opposite to novelty. Contrary to what Berlyne’s model would have predicted, ‘complexity’ did not account for differences in aesthetic appreciation. Subsequent studies in which both models were empirically tested against each other were performed for diverse categories such as houses (Purcell, 1984), cubist paintings (Hekkert and van Wieringen, 1990), and musical performances (Repp, 1997), all confirming a linear relationship between preference and prototypicality. Although familiarity is not the only defining variable of (proto)typicality (Barsalou, 1985), the two concepts are clearly related. They both find their aesthetic attractiveness in ease of classification or processing (Reber et al., 2004). But ease of processing is not what people are always after. At various occasions people look for novel or original instances and especially children have a bias towards novelty in their early ages (e.g. Uehara, 2000).

3.2. Originality, novelty and innovativeness Biederman and Vessel (2006) claim that as our brain has evolved in order to understand the world, it derives pleasure from processing new and unfamiliar objects. They showed that new pictures of scenes and objects were preferred over pictures shown repeatedly. Though this seems to contradict the above described mere-exposure hypothesis, this finding is very much in accordance with our everyday experience. We are often attracted by new, unusual and innovative products (Veryzer and Hutchinson, 1998). However, the visual pleasure proposed by Biederman and Vessel (2006) only emerges when we are able to identify and successfully process what we see or, in other words, when the new thing is not frighteningly unfamiliar. A related argument has also been proposed in explaining the aesthetic appeal of modern art, which allows us to experience that we master the ‘new’, and gain aesthetic pleasure through a subjective state of successful classification, interpretation and understanding (Leder et al., 2004). Nonetheless, from everyday experience it is apparent that novel or innovative products are often not liked immediately. Although innovative products seem to be essential for companies in competitive markets, this initial dislike poses them with a serious problem. In a series of experiments concerning the role of innovativeness in car interior design, Leder and Carbon (2005) varied stimuli according to innovative features, such as curvature and complexity. When participants were asked to indicate how much they liked each version, they preferred the curved versions to the edged versions, a finding in accordance with a recent study by Bar and Neta (2006). Most importantly, Leder and Carbon (2005) found that their participants did not appreciate innovative versions. As we know that novelty – and innovativeness as a special case – is often initially unappreciated, the

270

Product Experience

authors further aimed to understand what variables could increase liking for innovativeness. From our daily experience we know that appreciation often changes over time. Take for example the ‘edgy’ backside of the Megane, recently introduced by Renault. Now that the car has been on the road for a while, to most perceivers it no longer seems aversive, but gains in appreciation. This brought Carbon and Leder (2005) to think of a setting which allowed them to realistically measure not only appreciation and innovativeness in a single shot measure, but to understand the changes over time, i.e. the dynamics of innovation. For these purposes, Carbon and Leder developed the repeated-evaluation technique (RET). In their study they used drawings of stylized car interiors, which systematically varied in innovativeness (as well as in other variables). When perceivers in a first block were asked to indicate how attractive the different versions were, they preferred the classical version. Next, participants took part in a session in which they were asked to rate all designs according to several dimensions, which made them actively deal with the designs for about half an hour (Carbon and Leder, 2005). When the participants afterwards rated all stimuli again according to liking and innovativeness, it was found that the more innovative designs were now seen as more attractive, while they still preserved their level of apparent innovativeness. From these results it can be concluded that actively evaluating the stimuli somehow reveals the possible advantage of innovativeness: The design becomes more attractive and still preserves aspects of being new, distinctive, and thus innovative.

3.3. ‘Most advanced, yet acceptable’ So far we have two seemingly contradictory hypotheses, ‘we like what we know’ versus ‘we sometimes appreciate the new’. How do these fit together? Is it possible that both are true at the same time? These two seemingly contradictory aspects are brought together in the famous MAYA principle proposed by Raymond Loewy (1951), MAYA being an acronym for Most Advanced, Yet Acceptable. Designers need to find a balance between innovation and novelty (advanced) and a certain amount of typicality (acceptable). Is such a balance possible and do objects that correspond to the principle indeed produce a high level of appreciation and pleasure? Hekkert, Snelders and van Wieringen (2003) provided a strict empirical test of this MAYA assumption. They selected various products, such as telephones and teakettles, which differed along the dimensions of typicality and novelty. Participants rated all objects according to typicality, novelty, and aesthetic preference. As expected, novelty and typicality highly intercorrelated and each correlated poorly with preference. The trick of the study was to analyze the effect of both variables on aesthetic preference independently, by keeping the other variable (statistically) constant. In full accordance with the predictions of the MAYA principle, Hekkert et al. found independent effects on aesthetic preference of both novelty and prototypicality, and these effects were nearly equally strong. Thus indeed, attractive designs comprise a thoughtful balance between novelty and typicality.

3.4. Product expression and association So far, the meaningful properties we discussed are rather unspecific and mainly require a generalized comparison with things we have encountered before. However, we tend to attribute many more differentiated meanings to a product, that sometimes rely on deeper levels of processing. We could, for example, see a product as feminine, easy to use, or friendly, or associate it with products from the 1950s or Italian design. Although the perception and identification of such connotative meanings are treated elsewhere in this volume (see Chapter 13) and go beyond the scope of the present chapter, identifying such

Product aesthetics

271

meaningful properties may also have an impact on a product’s perceived attractiveness. When, for example, Volkswagen introduced the New Beetle, much of its aesthetic appeal was due to the fact that the unusual shape referred to the old model by Ferdinand Porsche. Given the narrow definition of aesthetics as adopted in this chapter, however, it remains to be seen whether such effects should be coined aesthetic. Take for instance the finding that people prefer products that express a personality that matches their own personality, also referred to as the self-congruency effect (Govers and Schoormans, 2005). The explanation for this effect is that people prefer congruent products because they are an extension of the self, and thereby contribute to one’s identity. The liking in this case is based on an external motive, which renders it less aesthetic, or even non-aesthetic. Next to such direct effects of meaning on liking, attributed meanings can also affect familiarity or originality and, thereby, have an indirect effect on a product’s aesthetic appeal. In an attempt to identify the determinants of product originality, Snelders and Hekkert (1999) asked participants to indicate to what extent a set of telephones could be associated with things you encounter in other domains, such as ‘in the bedroom’, ‘in church’, ‘while shopping’, or ‘in a fairy tale’. They showed that measures based on the relative uniqueness of these associations (i.e. a product has few associations in common with other products from the same category) were good predictors of a product’s originality. Since, as we have seen, originality highly contributes to aesthetic preference, these meaningful associations have an indirect effect on a product’s aesthetic appeal. If and how product expression and association contribute to the aesthetic quality of a product remains both a theoretical and empirical question. It is nevertheless clear that a designer should consider what kinds of associations and relations to general or specific knowledge people might make, since these clearly affect the way a product is perceived and appreciated.

4. UNIVERSAL AESTHETIC PRINCIPLES 4.1. A study on cross-cultural aesthetic universals At the end of the twentieth century a study was carried out that perfectly conforms to the law of maximum effect for minimal means. In this study, Hardonk (1999) adopted Fechner’s (1876) method of production (i.e. people show their aesthetic preferences in the artifacts they produce) to find aesthetic universals: If such universals exist, they must be present in artifacts from all cultures. Hardonk found the perfect, ecologically valid, object of study – band patterns. Decorative band patterns, a motif that is repeated in one direction, are produced in all cultures and can be found on items such as vases and curtains, weapons and clothes (Figure 10.6). A further advantage of such patterns, unlike works of art, is that they are relatively simple and can therefore be objectively described and compared. But as simple and elegant as the idea of the study was, as complex was its execution. Hardonk first started to define culture (based on independence, language and territory) and then selected a stratified sample of 20 cultures from a total of 294 in the ‘Hardonk sample frame’. Next, before the selection of bands, he developed a descriptive system with properties that could, in principle, occur in bands. This system was based on people’s perceptual interpretation of a band, not on its formal characteristics. This means that the system takes into account how people would describe a band in terms of objects on a background, and objects in terms of, for example, orientation and whether they are entirely visible or partly occluded (see the discussion on perceptual organization in Section 2). From each culture, 40 bands were taken randomly and put into drawings.

272

Product Experience

FIGURE 10.6 Examples of decorative band patterns in an eighth-century amphora from Attica, Greece (reprinted with permission from Biers, 1996, p. 124).

Finally, all bands were described using the descriptive system, resulting in 74 independent universal properties, properties that occur to the same degree in all cultures. These universals varied from absolute (admitting no fluctuations between cultures), to strong (minor fluctuations) to weak (somewhat more fluctuations); most were positive universals (occurring in all cultures), whereas some were negative (occurring in none of the cultures). It is of course impossible to list them all here, but let us try to summarize the most interesting findings. Band patterns from all cultures contain one or more regularities, such as symmetry, parallelism, and equality of sides and angles. Both at the level of objects, and in the band as a whole, mirror symmetry occurred much more often than rotation symmetry, and vertical symmetry is much more prominent than horizontal symmetry. Most objects in bands consist of simple shapes, such as triangles, but in all cultures we also find patterns with more complex objects. If objects are grouped, most groups contain only two different objects (e.g. a circle and a square). Results like these support the contention that we like simplicity and order (Section 2.1), but that we (occasionally) also need some variety in this unity (Section 2.3). Furthermore, in all cultures, most bands contain no occlusions (a negative universal) and one or more ambiguities, of which object-background is the most frequent. Surprisingly, however, only disjunctive ambiguities were found and no conjunctive ambiguities (see Conjunctive ambiguity in Section 2.3).

Product aesthetics

273

FIGURE 10.7 A universally accepted band with many universals, such as various forms of symmetry, simple shapes, parallelism, and object–background ambiguity (from Hardonk, 1999, p. 192).

Another elegant way to summarize the universals is by combining as many as possible in a single band pattern (Figure 10.7). Despite the limitations related to using band patterns as an object of study – band patterns are after all rather simple 2D patterns and often applied for other than aesthetic (e.g. symbolic, communicative) purposes – the study confirms that many of the properties discussed earlier can be regarded as universal aesthetic principles. The question then remains, where does this universality come from? How can we explain that people of all times and cultures prefer the same properties?

4.2. Evolutionary aesthetics The most likely candidate to explain universal patterns in aesthetic preference is human evolution. Explanations along this line have been extensively proposed over the last two decades with the advancement of evolutionary psychology. The attractiveness of these theories is their ability to explain why general patterns in human behavior and their underlying psychological mechanisms are the way they are. As one of their most prominent proponents argues, ‘In the study of humans, there are major spheres of human experience – beauty (our italics), motherhood, kinship, morality, cooperation, sexuality, violence – in which evolutionary psychology provides the only coherent theory’ (Pinker, 2002, p. 135). As Darwin (1859) himself already predicted, humans have not only physically, but also mentally adapted to the challenges posed by their environments. Faced with adaptive problems, such as finding a mate, hiding from enemies, or understanding intentions, psychological mechanisms have evolved that are perfectly fit to solve such problems. As a result, we have acquired adaptations like sensory systems, a language capacity, and a trait for emotional communication, and … an aesthetic sense. One may now ask oneself, what on earth can be adaptive about finding someone attractive or something beautiful? Has it not always been argued that art and aesthetics are intrinsically useless? Art may be so, although some will certainly dispute this and argue that art is an adaptation itself (e.g. Dissanayake, 1992), but aesthetic preference certainly is not, as we will see next. To explain the evolutionary basis of aesthetic preference, one major hypothesis can be coined the ‘transfer-hypothesis’ (see Rhodes, 2006), based on principles stemming from mate selection. The basic idea is that certain characteristics in attractive people, such as symmetry, are indicators of good health (e.g. an absence of parasites) and hence, may refer to reproductive fitness, the ability to produce healthy offspring (e.g. Grammer and Thornhill, 1994). The attractiveness of such features is abstracted and somehow transferred to other objects that, as such, have no biological relevance. Thus, according to this view, we have come to like symmetrical patterns, not only in humans but also in artifacts. Others however argue that our (aesthetic) preferences are domain dependent (see Tooby and Cosmides, 2001) and related to domain-specific properties having survival value. Take for instance our preference for (properties in) landscapes. According to Wilson’s biophelia-hypothesis (1984), we prefer savannah-like landscapes: Open grasslands with

274

Product Experience

trees, water, animals, and plants, because these signal fertility, and thus abundance of food, as well as provide safety, in that they both offer means for hiding and spots that give an overview of the surroundings (see also Orians and Heerwagen, 1992; Pinker, 2002). It may indeed be difficult to ‘translate’ such specific preferences to other domains. Nevertheless, in line with other scholars such as Ramachandran (2004; Ramachandran and Hirstein, 1999), we believe there are domain-independent aesthetic universals that can be explained by looking at the evolutionary origin of our information processing system. As argued elsewhere, this explanation is often referred to as the ‘by-product’ hypothesis (Pinker, 2002; Hekkert, 2006). According to this hypothesis, our aesthetic sense is a by-product of other adaptations, primarily of our sensory systems and brain.2 Because certain patterns or features in the environment were functionally beneficial to these systems, we (have come to) derive an aesthetic pleasure from perceiving them. In other words, ‘it is brains that have evolved to generate pleasant and unpleasant feelings to those aspects of the environment that were a consistent benefit or threat to gene survival in ancestral environments’ (Johnston, 2003, p. 173). In order to find these patterns or aspects, we thus have to look at the functions of our sensory modalities (Hekkert, 2006). Whereas some of these functions are modality specific, possibly leading to domain-specific aesthetic preferences, there are certain functions that apply to most or all sensory domains (see Chapter 5). All sensory domains play a role in the identification of things or signals, whether it is a form, a sound, a texture, or a smell. Given the wealth of information in our surroundings and our limited capacity to process information, patterns or structures that support such identification are generally preferred over others (see also Ramachandran and Hirstein, 1999; Reber et al., 2004). From this basic ‘law’, we can explain many of the aesthetic principles discussed above and even predict some new ones. Both Martindale (1990) and Ramachandran and Hirstein (1999) consider the peak shift principle a prominent principle underlying our aesthetic experience. Peak shift is a well-known phenomenon from behavioral learning and refers to the inclination of animals to respond more strongly to stimuli that go somewhat beyond the one the animal has learned to be rewarding. ‘Because of peak shift, female birds that prefer to mate with males with bright rather than dull plumage will show even greater preference for males with supernormal or above-average brightness.’ (Martindale, 1990; p. 47). Our liking of caricatures, for example, can be explained along similar lines in that they amplify the – already attractive – ‘very essence’ or prototype of a face (Ramachandran and Hirstein, 1999). Figure 10.8 is an example of peak shift in product design. In this lamp for the Italian manufacturer FLOS, the designer Achille Castiglioni in 1972 has amplified the essence or ‘lampness’ of a lamp by putting an enormous bulb on a pedestal. In proposing that ‘all art is caricature’ (p. 18), Ramachandran and Hirstein give a number of examples in which artworks show such amplifications. By isolating and amplifying the ‘essence’, peak shifts contribute to ease of recognition and are therefore advantageous to our brain’s limited capacity. Many of the principles proposed by Ramachandran and Hirstein are related to the unity in variety principle as discussed in Section 2.3. Since unifying or organizing mechanisms, such as grouping, symmetry, closure, and contrast, allow us to see what belongs together (or not), detecting such structures is rewarding. They all contribute to ‘binding’, i.e. making connections, and the creation of order and, as such, facilitate economic processing of information. This not only holds for seeing the unity, but also for the process of 2 Our aesthetic sense is not alone in this. Many psychological phenomena that come so natural to us humans, such as religion (e.g. Dawkins, 2006), are most probably non-adaptive by-products of adaptations that do have survival value in and of themselves.

Product aesthetics

275

FIGURE 10.8 Lamp designed by Achille Castiglioni.

detection itself, as in solving puzzles (see also Hekkert, 2006). The rewarding effect of seeing relationships is furthermore not restricted to formal qualities, but can also result from connections made at a semantic level. This is for example the case in metaphors, where meaning is efficiently added to a product by a reference to something else (Forceville, Hekkert and Tan, 2006). Take for example Philip Starck’s famous toilet brush Xcalibur, that by its name and shape refers to the sword used by King Arthur (Figure 10.9). Through this reference, a playful and adventurous meaning is added to the, for most, not so exciting task of cleaning a toilet. It is as if the product says ‘let’s go and attack the dirt!’. It has been argued that such metaphors can be effective even when we are not (yet) consciously aware of them (Cupchik, 2003).

276

Product Experience

FIGURE 10.9 Toilet brush Xcalibur by Phillippe Starck.

The ‘by-product’ hypothesis is thus capable of explaining a number of aesthetic phenomena and there is no reason why they should be restricted to the visual realm. In the next section we will briefly speculate on how this hypothesis could account for aesthetic preferences related to the other senses.

4.3. Cross-sensory aesthetic principles When aesthetics is defined as sensory gratification – as we do – it makes sense to speak of auditory aesthetics, tactual aesthetics, and olfactory and gustatory aesthetics, next to the traditional domain of visual aesthetics. It even seems logical to regard a feeling of comfort as an aesthetic response, a sort of proprioceptive aesthetics, and it also may look plausible to use the phrase ‘aesthetics of interaction’, as is popular in the field of interaction design (e.g. Dunne, 1999; Overbeeke et al., 2003). However, in order to please the senses, interaction with an object is conditional, making the expression ‘aesthetics of interaction’ somewhat tautological. What makes a product good to listen to, pleasant to touch or use, and nice to smell (or even taste)? Following the argument introduced in the previous section, product properties are reinforcing, and thus, aesthetically pleasing if they facilitate the adapted function of the sensory systems. Hekkert (2006) started to list these functions and proposed some first and tentative predictions as to their aesthetic consequences. As argued, all of our senses can play a role in the identification of objects. When it comes to this primary function, aesthetic principles should therefore hold cross-sensory. Just as people like to see patterns that allow them to detect relationships, people like to detect organization in sounds, and feel structure in a surface. Moreover, people like these various sensory messages to be mutually consistent and appropriate for the product conveying them. The product may display such an ‘optimal match’ with respect to its utilitarian function, its intended experience, and/or the associations it evokes (Hekkert, 2006).

Product aesthetics

277

Next to these cross-sensory similarities of aesthetics, some functions are unique to a particular system and may lead to sensory domain-specific aesthetic principles. Take for example our sense of touch. It not only functions to provide us with information about the world, such as the shape, temperature, and weight of things, it also makes us aware of having a body and thus enables us to experience ourselves (Bermudez, Marcel and Eilan, 1995). We might therefore predict that products (or product features) contributing to this self-experience are considered pleasant. The seemingly endless and repetitive manipulations babies employ on some of their toys may be evidence for this prediction. See Chapter 2 for further examples in this domain. As universal as these evolutionary explanations are, this is not to say that evolutionary theories are deterministic. Most evolutionary psychologists endorse the view that these psychological mechanisms can manifest themselves differently across cultures, and even across individuals, as a result of interactions with the environment. How such interactions affect the way the aesthetic principles work for each one of us will be explained next.

5. CULTURAL AND INDIVIDUAL DIFFERENCES If aesthetic principles are universal, how come we also see many differences over cultures and individuals? The important thing to see here is that a universal principle does not automatically lead to universal agreement in people’s aesthetic choices of objects. Take for instance the components that come together in the MAYA principle. Although we all (seem to) like products that are as novel as possible, while we still see them as typical of their kind, what is considered novel/typical will differ substantially over people. For example, Hekkert et al. (2003, study 2) showed that experts and non-experts weakly agreed on the typicality of the car models judged, but for both groups typicality and novelty jointly predicted aesthetic preference. People differ with respect to the things they perceive and attend to, people differ as to their previous experiences within a domain, and people differ with regard to many other background variables, and these differences may lead to a variety in aesthetic preferences, despite the universality of the underlying principles.

5.1. Sensitivity For many of the above-described principles aspects of stimuli have been identified, but in order for these to have an effect the perceiver has to perceive them. So, you need to see (be sensitive to) order or relationships in order to appreciate it. Simply put, if you cannot detect the symmetry, closure, or any other organizational structure, you cannot like the object on these grounds. People who are not able to see the order in an abstract painting or hear the structure underlying a modern musical composition, have difficulties liking it. To them, the painting or composition is predominantly chaotic. These problems most likely play a lesser role in product design, where most designers do their best to make their designs comprehensible and easy to understand. Nevertheless, some sensitivity may be required to see all subtle ordering principles applied in, for example, a car design and to appreciate it in full. This sensitivity can of course be trained, but as in all areas of human performance, some people are (just) more receptive to or better equipped to develop such sensitivity.

5.2. Knowledge and experience In the example of the MAYA principle described above, we have explained how differences in expertise may result in different aesthetic choices while the principle still holds.

278

Product Experience

As a result of their background or previous experience, people may perceive the degree of typicality, novelty and the like in a design differently. Similarly, because some product properties are familiar for one, but novel for another person, this may result in differential effects of fluent processing; what is easy to assimilate for one may be difficult to assimilate for another. In the domain of art, Leder (2002) has proposed a kind of higher-level fluency that accounts for such differences. When it comes to higher-order levels of dealing with objects, when interpretation and understanding come into play, expertise and experience become more important. For example, though a Picasso portrait might elicit aesthetic responses in all of us through the use of color and shapes, the cubist style Picasso adopted leaves little doubt that the depictions of persons he used is far from easy to read. Noses might be beside both eyes, and other elements are at unnatural locations in the head, and differ from nature in coloring, shape and size. Thus, how could one explain that such strongly alienated portraits (Leder, 2002) are liked, or even aesthetically appreciated? The processing of cubist portraits presumably becomes much easier and more fluent with experience and knowledge. This pattern holds for all domains of objects and thus also influences design appreciation. An example is the ‘organic’ design style by Phillippe Starck. This style is applied to numerous design objects such as cutlery, TV-sets, water-kettles, bathtubs, and even houses. For the appreciation of these products, it is essential that the ‘style’ is perceived and recognized, an ability that increases with experience (Cupchik and Laszlo, 1992; Augustin and Leder, 2006). Once a person can make the required stylistic discriminations, s/he will also recognize it in other objects of the same designer and like them even so. In this case, the pleasure is indirectly derived from knowledge about design, and constitutes a kind of higher-order cognitive fluency, which is quite pleasing to the perceiver. Developing such sensitivity for style and other organizational patterns normally comes with training and experience. Experienced viewers make finer aesthetic discriminations (Winner, 1982); they attend to and perceive properties of objects, such as lines, shapes, and textures, which remain unnoticed to the untrained eye. In sum, experienced observers discover features and higher order structures to which untrained observers are insensitive, and this allows them to enjoy different features and, hence, different and more complex objects than novices. Possibly, the increasing ‘aesthetization’ of our designed world, where even boilers, door handles and bath-tubs become decorated design objects, will enhance the general experience and sensitivity among the audience as it contributes to people’s ‘aesthetic view’ on everyday objects.

5.3. Culture A variable held responsible for many of the differences in people’s aesthetic choices is culture. Popular wisdom would even say that our taste is predominantly shaped by the culture to which we belong. Looking at the diversity among cultural expressions in art, fashion, and design, it seems obvious that culture has a big effect on our aesthetic preference. Other than highlighting and emphasizing these differences, it is more interesting to investigate where such differences originate. Robert Nisbett has addressed this issue in a comprehensive research program that aimed to find out whether people in Western and Asian culture perceive things differently (Nisbett, 2003). He found evidence for a more holistic style of perceiving scenes and objects by Asian people, while Americans tended to see objects in a more analytical mode of processing (see also Masuda and Nisbett, 2001). Such a fundamental difference in looking at the world reveals that cultural background affects the way a product is perceived and, subsequently, aesthetically appreciated.

Product aesthetics

279

Other cross-cultural studies have shown that people from different cultures may systematically differ in the values and standards they hold (e.g. Schwartz and Sagiv, 1995; Hofstede, 2001), such as the degree to which people see themselves as separate from others (individualistic) or as connected to others (collectivistic), an orientation known as self-construal (Markus and Kitayama, 1991). In a recent study, for example, it was shown that logos from predominantly individualistic cultures (e.g. United States, Germany) were more angular than those of collectivistic cultures (e.g. Hong Kong, Japan), indicating that the latter relatively prefer rounded shapes, which are considered to be more harmonious (Zhang, Feick and Price, 2006). Instead of seeing such cross-cultural differences as unique and autonomous cultural phenomena, we suggest it is more fruitful to look for the underlying psychological mechanisms that govern these manifestations and then try to explain how these variations come across. Groups of people can share defining characteristics, such as sensitivity, standards of aesthetic quality, and perceived typicality to various degrees (Hekkert, 2006). If these defining characteristics differ at the group level (as in cultures), we find cross-group differences and within-group agreement. Sharing characteristics results from having a similar background, i.e. similar experiences in the interaction with the social, natural, and artificial environment one is raised in and has to deal with. Regarded in this way, the cultural, as well as the social, are ‘nothing more’ (and nothing less) than manifestations of evolved human biology (Tooby and Cosmides, 1992), where evolvement refers to the way the psychological mechanisms have developed and operate under different circumstances. As E. O. Wilson (1998) puts it, ‘Thousands of genes prescribe the brain, the sensory system, and all the other physiological processes that interact with the physical and social environment to produce the holistic properties of mind and culture’. (p. 150). How this gene-culture coevolution takes place and determines our aesthetic preferences, as well as many other psychological phenomena, is currently explored (see Buss, 2005 for an overview) and will fundamentally change our future understanding of (the relationship between) culture and aesthetics.

5.4. The evolution of taste If universal principles guide our aesthetic preferences, but are simultaneously affected by knowledge, culture, and habituation, can we predict the development of people’s taste? Put differently, can designers/artists lawfully anticipate the ever-changing demands of their audiences? This is exactly what Martindale (1990) claims in his theory of artistic change that spans 20 years of research. Starting with a theory explaining changes in the development of literature, his research expanded to domains such as poetry, visual arts, music, gravestones, and careers of individual artists. The basic assumption underlying his theory is that through repeated presentation, an artistic stimulus such as a work of art, gradually loses its impact value or arousal potential (Berlyne, 1971; see Section 2.2). As a result, the capacity of an artwork to raise interest, pleasure, or attention will diminish, a process that has also been described as Formermüdung or ‘Form fatigue’ (Göller, 1888; cited in Martindale, 1990). To compensate for such habituation effects, successive artists need to increase the arousal potential of their works of art. Martindale demonstrated that this is exactly what artists do: Measures of arousal potential, like complexity, ambiguity, or novelty, increase monotonically over time.3 3 Martindale warns that these increments in arousal potential should not be too large. This shift in preferences is a gradual process due to effects of peak shift and the minimal effort recipients want to spend (see also Section 4.2).

280

Product Experience

These mechanisms may explain why art changes; they do not explain the direction in which art changes. To account for this, Martindale proposed a second process contributing to an increase of arousal potential. This process entails a regression from ‘secondaryprocess’ or conceptual cognition, characterized by abstract, logical and reality-oriented thought processes, to ‘primary-process’ or primordial thinking, a free-associative, concrete, and irrational mode of thought. The latter type of thinking leads to new combinations of existing elements (within a style) and results in novel ideas, hence an increase in arousal potential. ‘Across the time a given style is in effect, we should expect works of art to have content that becomes increasingly more and more dreamlike, unrealistic, and bizarre’ (Martindale, 1990; p. 61). At a certain point however, the style becomes saturated and further regression will lead to an arousal decrease, the ‘evolutionary trap’. Artists are then required to introduce a stylistic change by allowing new elements to enter the artistic lexicon or by loosening the rules governing the old style. For this to happen, artists will rely more on conceptual, secondary-process thinking. The result of this process is a cyclical change in primordial content and coinciding changes in artistic styles. These related fluctuations could also be observed in most of the domains/artists Martindale examined. Although Martindale’s theory has proven to describe historical changes in many art forms, the question is whether it can also be applied to design products. Martindale himself was not optimistic about this and thought that a product’s usefulness might put non-aesthetic pressures on the design. ‘If something has a use, people want it to work. That gets in the way of its aesthetic aspects. If something has a use, people can stop using it and destroy its aesthetic aspects altogether’ (Martindale, 1990, p. 55). However, in an age in which aesthetics plays a dominant role in design (Postrel, 2003), and since many products have reached maturity when it comes to their performance and functionality, Martindale’s theory may very well explain the changes governing the development of many present-day products.

6. CONCLUSIONS In the present chapter we have reviewed research on aesthetic appreciation and demonstrated that preferences or taste judgments obey certain rules or principles. More importantly, we have argued that many of these principles are rooted in human nature and can somehow be explained on the basis of adaptations of our sensory systems and brains to our environment. Since these adaptations are, by definition, functional, allowing us to deal with the demands put forward by the surroundings, we can conclude that having an aesthetic sense is extremely useful! It stimulates and reinforces us to look for patterns and unifying properties that support the tasks of our sensory systems. If we perceive them, we can perform optimally and are aesthetically gratified, thus explaining why beauty and perceived usability are so strongly correlated (Tractinsky, Katz and Ikar, 2000; see also Chapter 11). How are designers to deal with these principles of aesthetics?

6.1. Implications for design and designers The best recommendation one could give designers is to follow the rules and obey these aesthetic principles (Hekkert, 2006). Evaluating all the products on the market, it seems clear that they do so. All cars are symmetric,4 most mobile phones have their buttons 4 The story goes that the first Citroen BX that was put on the market had its logo not placed in the middle-front of the bonnet, as all cars used to have and still have, but just off center. The car did not sell well and only when the logo was repositioned in the middle did sales increase drastically.

Product aesthetics

281

orderly organized, and many successful designs follow the MAYA-principle. Designers do not need to know the principles to apply them; they intuitively design accordingly, since the principles are as much part of their creative nature as they are of the observer’s aesthetic perception. As Ramachandran and Hirstein (1999) claim for artists, these rules or principles are ‘a set of heuristics that artists either consciously or unconsciously deploy to optimally titillate the visual areas of the brain’ (p. 15). Thus, designers will and should follow the rules when there is no reason not to. Lidwell et al. (2003) propose a similar strategy in their book of 100 design principles, but they also end their introduction with an additional statement: ‘The best designers sometimes disregard the principles of design. When they do so, however, there is usually some compensating merit attained at the cost of the violation. Unless you are certain of doing as well, it is best to abide by the principles’ (Lidwell et al., 2003, p. 11). What reasons could these ‘best designers’ have to disregard the rules? Why make something ugly, surprising, over-the-top, incomprehensible, etc? Products are not always and only designed to be visually pleasing. At times, designers break the rules for other, non-aesthetic reasons. Let us briefly explore three such reasons. First, designers could decide to make a product stand out, to make it visually very different from competing products, if needed at the cost of aesthetics. The reason is obvious: The product will draw attention and attract interest and this can be a valuable asset in saturated markets. Moreover, when an unusual appearance is due to technical progress the resulting product might need some time to be liked for its aesthetic appeal (Carbon and Leder, 2005). Secondly, products convey meanings and deciding to express certain symbolic, cultural or personal values through a product may contribute highly to its attractiveness. It may stimulate certain people to buy a BMW that looks strong and like a predator, even if its aesthetic proportions are inappropriate. Finally, more and more designers are becoming aware of the emotive powers of designed objects. Products can raise fascination or desire, evoke surprise, and be fun to use. Since emotions are valenced reactions, the pleasure attained from the emotional response could easily outdo any limitations as to its aesthetic quality. In the case of surprise, for example, the non-aesthetic effect of incongruity between our visual impression and our tactual experience will evoke a surprise reaction that may result in interest, amusement or other positive emotions (Ludden, Hekkert and Schifferstein, 2006). These are some of the reasons to disregard aesthetic principles, and future designers will certainly come up with more. Nevertheless, carefully applying aesthetic rules is a safe way to ascertain product acceptance and appreciation. Especially in areas where production costs are high, such as the car industry, and given the high probability that other, non-aesthetic effects may fade out quickly in time, most designers are best off to stick to the rules.

6.2. Future of design aesthetics We can explain (parts of) people’s aesthetic preferences, but a lot of unresolved issues remain. Before drawing this chapter to a close, we want to look into some new developments that already have or will have an effect on research in aesthetics in the near future. So far, the history of empirical research in product aesthetics often relied on wellestablished methods from experimental psychology. Key in this type of research is the design of stimulus material. Traditionally, ‘poor’, simple stimuli, such as polygons and random dot patterns, were used because they allowed for systematic variation of the dimensions studied. Findings from such studies can, however, not be easily generalized to real-life artifacts such as artworks or products. Since the early 1970s, researchers in aesthetics moved to these ecologically valid, but complex artifacts as stimulus materials.

282

Product Experience

These real objects, however, vary on so many dimensions that it is often difficult to ascribe effects to the relevant stimulus dimension. As has often been argued (see e.g. Hekkert and van Wieringen, 1996; Whitfield, 2000), the ‘middle’ way to proceed in aesthetic research is through a systematic manipulation of relevant stimulus dimensions in real artifacts. Such techniques have been occasionally employed in the area of product design (e.g. Carbon and Leder, 2005; Van Rompay et al., 2005; Veryzer and Hutchinson, 1998), but these were still two-dimensional representations of real products. With the advancements in computer software connected to rapid prototyping, it will become much easier to make systematic variations of three-dimensional, physical objects. Further developments in research methods might also bring new insights. The increasing interest in neuropsychological measures will broaden the scope of methods to investigate aesthetic appreciation. Up to now, neuroimaging studies in the area of aesthetics mainly were concerned with determining neuroanatomical correlates of aesthetic preference. Most of these studies have investigated the perception and appreciation of paintings using functional Magnetic Resonance Imaging (fMRI). For instance, Hansen, Brammer and Calvert (2000) demonstrated that activation of primary and association visual cortices varied depending on preference judgments. Their findings suggested that quantitative changes in activation, as well as qualitatively distinct networks of brain areas in frontal and limbic areas, are associated with positive, negative, and neutral judgments to images of artworks. Vartanian and Goel (2003) compared representational and abstract paintings in different formats (original, altered, filtered) and reported neural correlates of lower and higher preference. Kawabata and Zeki (2004) also searched for neural correlates of the perception of paintings considered to be beautiful. The perception of beautiful and ugly paintings led to a different involvement of the orbitofrontal and the motor cortex. Finally, Jacobson, Schubotz, Höfel and von Cramon (2006) applied the same technique to identify neural correlates of aesthetic judgments of abstract graphic patterns. Interestingly, the most active regions they located (e.g. medial wall and prefrontal cortex) partly overlap with the ones involved in social and moral judgments. Looking at product aesthetics with such neuropsychological methods in the future presumably will reveal a deeper insight into the complex interplay between emotion and cognition which interact in aesthetic experiences (Leder et al., 2004) and might also reveal the underlying mechanisms, which differ from person to person and might be specific for classes of objects (Whitfield, 2000). Both of the above developments are related to increasing technological sophistication allowing for new ways of experimentation. When it comes to theoretical progress, we strongly believe that the emerging field of evolutionary aesthetics will bring most progress as it provides a theoretical foundation against which all the variation through culture and individual history can be tested. Along with Wilson (1998) and others (e.g. Ramachandran and Hirstein, 1999; Pinker, 2002) we predict that crossing borders between psychology, evolutionary biology, and neurology will permanently change our understanding of human behavior and culture, of which our sense for beauty is just one, albeit a very prominent, representative.

REFERENCES Arnheim, R. (1974). Art and visual perception, The New Version. London: Faber and Faber. Augustin, D. and Leder, H. (2006). Art expertise: a study of concepts and conceptual spaces. Psychology Science, 48, 135–157. Bar, M. and Neta, M. (2006). Humans prefer curved visual objects. Psychological Science, 17, 645–648.

Product aesthetics

283

Barkow, J., Cosmides, L. and Tooby, J. (Eds.) (1992). The adapted mind. New York: Oxford University Press. Barsalou, L. (1985). Ideals, central tendency, and the frequency of instantiation as determinants of graded structure in categories. Journal of Experimental Psychology: Learning, Memory, and Cognition, 11, 629–654. Berlyne, D. E. (1966). Curiosity and exploration. Science, 153, 25–33. Berlyne, D. E. (1970). Novelty, complexity and hedonic value. Perception and Psychophysics, 8, 279–286. Berlyne, D. E. (1971). Aesthetics and psychobiology. New York: Appleton-Century-Crofts. Bermudez, J. L., Marcel, A. and Eilan, N. (Eds.) (1995). The body and the self. Cambridge: MIT Press. Biederman, I. (1987). Recognition-by-components: a theory of image understanding. Psychological Review, 94, 115–147. Biederman, I. and Vessel, E. (2006). Perceptual pleasure and the brain. American Scientist, 94, 249–255. Biers, W. R. (1996). The archaeology of Greece (2nd Ed.) Ithaca: Cornell University Press. Blood, A. J. and Zatorre, R. J. (2001). Intensely pleasurable responses to music correlate with activity in brain regions implicated in reward and emotion. Proceedings of the National Academy of Science, 98, 11818–11823. Bornstein, R. F. (1989). Exposure and affect: Overview and meta-analysis of research, 1968–1987. Psychological Bulletin, 106, 265–289. Boselie, F. (1992). The golden section has no special aesthetic attractivity! Empirical Studies of the Arts, 10, 1–18. Boselie, F. and Leeuwenberg, E. (1985). Birkhoff revisited: Beauty as a function of effect and means. American Journal of Psychology, 98, 1–39. Bullough, E. (1912). ‘Psychical distance’ as a factor in art and an aesthetic principle. British Journal of Psychology, 5, 87–118. Buss, D. M. (Ed.) (2005). The handbook of evolutionary psychology. Hoboken, NJ: Wiley. Carbon, C. C. and Leder, H. (2005). The Repeated Evaluation Technique (RET). A method to measure dynamic effects of innovativeness and attractiveness. Applied Cognitive Psychology, 19, 587–601. Cupchik, G. C. (2003). The interanimation of worlds. Creative metaphors in art and design. The Design Journal, 6, 14–28. Cupchik, G. C. and Laszlo, J. (1992). Emerging visions of the aesthetic process. Psychology, semiology, and philosophy. Cambridge: Cambridge University Press. Darwin, C. (1859). On the origin of the species. London: Watts. Dawkins, R. (2006). The God delusion. London: Bantam Press. Dissanayake, E. (1992). Homo aestheticus: Where art comes from and why? New York: Free Press. Dunne, A. (1999). Hertzian tales: electronic products, aesthetic experience and critical design. London: RCA CRD Research publications. Fechner, G. T. (1876). Vorschule der Ästhetik. Leipzig: Breitkopf und Härtel. Forceville, C., Hekkert, P. and Tan, E. (2006). The adaptive value of metaphors. In: U. Klein, K. Mellman and S. Metzger (Eds.) Heuristiken der Literaturwissenschaft. Einladung zu disziplinexternen Perspektiven auf Literatur, pp. 85–109. Paderborn: Mentis. Frijda, N. H. (1988). The laws of emotion. American Psychologist, 43, 349–358. Frijda, N. H. (1989). Aesthetic emotions and reality. American Psychologist, 44, 1546–1547. Frith, C. and Nias, D. (1974). What determines aesthetic preference? Journal of General Psychology, 91, 163–173. Godkewitsch, M. (1974). The ‘Golden Section’: An artifact of stimulus range and measure of preference. American Journal of Psychology, 87, 269–277. Goldman, A. (2001). The Aesthetic. In: B. Gaut and D. McIver Lopes (Eds.) The Routledge companion to aesthetics, pp. 181–192. London: Routledge. Govers, P. C. M. and Schoormans, J. P. L. (2005). Product personality and its influence on consumer preference. Journal of Consumer Marketing, 22(4), 189–197. Grammer, K. and Thornhill, R. (1994). Human (Homo sapiens) facial attractiveness and sexual selection: The role of symmetry and averageness. Journal of Comparative Psychology, 108, 233–242. Hansen, P., Brammer, M. and Calvert, G. (2000). Visual preference for art images discriminated with fMRI. NeuroImage, 11, 739. Hardonk, M. M. (1999). Cross-cultural universals of aesthetic appreciation in decorative band patterns. Unpublished doctoral thesis, NICI, Katholieke Universiteit Nijmegen. Hekkert, P. (1995). Artful judgments. A psychological inquiry into aesthetic preference for visual patterns. Unpublished doctoral thesis, Delft University of Technology. Hekkert, P. (2006). Design aesthetics: Principles of pleasure in design. Psychology Science, 48, 157–172. Hekkert, P., Peper, C. E. and van Wieringen, P. C. W. (1994). The effect of verbal instruction and artistic background on the aesthetic judgment of rectangles. Empirical Studies of the Arts, 12, 111–124.

284

Product Experience

Hekkert, P., Snelders, D. and van Wieringen, P. C. W. (2003). ‘Most advanced, yet acceptable’: Typicality and novelty as joint predictors of aesthetic preference in industrial design. British Journal of Psychology, 94, 111–124. Hekkert, P. and van Wieringen, P. C. W. (1990). Complexity and prototypicality as determinants of the appraisal of Cubist paintings. British Journal of Psychology, 81, 483–495. Hekkert, P. and van Wieringen, P. C. W. (1996). The impact of expertise on the evaluation of original and altered versions of post-impressionistic paintings. Acta Psychologica, 94, 117–131. Hofstede, G. H. (2001). Culture’s consequences: Comparing values, behaviors, institutions and organizations across nations. Thousand Oaks, CA: Sage. Jacobsen, T. and Höfel, L. (2003). Descriptive and evaluative judgment processes: Behavioral and electrophysiological indices of processing symmetry and aesthetics. Cognitive, Affective and Behavioral Neuroscience, 3, 289–299. Jacobson, T., Schubotz, R. I., Höfel, L. and von Cramon, D. Y. (2006). Brain correlates of aesthetic judgment of beauty. NeuroImage, 29, 276–285. Johnston, V. S. (2003). The origin and function of pleasure. Cognition and Emotion, 17, 167–179. Kant, I. (1952). The critique of judgment. (J. C. Meredith, Trans.). Oxford: Clarendon Press. (Original work published 1790.) Kawabata, H. and Zeki, S. (2004). Neural correlates of beauty. Journal of Neurophysiology, 91, 1699–1705. Lazarus, R. S. (1991). Progress on a cognitive-motivational-relational theory of emotion. American Psychologist, 46, 819–834. Leder, H. (2002). Explorationen in der Bildästhetik. Lengerich: Pabst. Leder, H., Belke, B., Oeberst, A. and Augustin, D. (2004). A model of aesthetic appreciation and aesthetic judgements. British Journal of Psychology, 95, 489–508. Leder, H. and Carbon, C. C. (2005). Dimensions in appreciation of car interior design. Applied Cognitive Psychology, 19, 603–618. Lidwell, W., Holden, K. and Butler, J. (2003). Universal principles of design. Gloucester, MA: Rockport. Locher, P. (2006). The usefulness of eye movement recordings to subject an aesthetic episode with visual art to empirical scrutiny. Psychology Science, 48, 106–114. Locher, P. J. and Nodine, C. F. (1987). Symmetry catches the eye. In: J. K. O’Regan and A. Levy-Schoen (Eds.) Eye movements: From physiology to cognition, pp. 353–361. Holland: Elsevier Science. Locher, P. J., Overbeeke, K. and Stappers, P. J. (2005). Spatial balance of color triads in the abstract art of Piet Mondrian. Perception, 34, 169–189. Loewy, R. (1951). Never leave well enough alone. New York: Simon and Schuster. Ludden, G. D. S., Hekkert, P. and Schifferstein, H. N. J. (2006). Suprise and emotion. In: P. M. A. Desmet, M. A. Karlsson and J. van Erp (Eds.) Design and Emotion 2006, pp. 1–13. Göteborg: Chalmers University of Technology. Marr, D. (1982). Vision: a computational investigation into the human representation and processing of visual information. San Francisco: Freeman. Martindale, C. (1984). The pleasures of thought: A theory of cognitive hedonics. The Journal of Mind and Behavior, 5, 49–80. Martindale, C. (1990). The clockwork muse: The predictability of artistic change. New York: Basic Books. Markus, H. R. and Kitayama, S. (1991). Culture and the self: Implications for cognition, emotion and motivation. Psychological Review, 98, 224–253. Masuda, T. and Nisbett, R.E. (2001). Attending holistically versus analytically: Comparing the context sensitivity of Japanese and Americans. Journal of Personality and Social Psychology, 81, 992–934. McManus, I. C. (1980). The aesthetics of simple figures. British Journal of Psychology, 71, 505–524. McManus, I. C., Jones, A. L. and Cottrell, J. (1981). The aesthetics of colour. Perception, 10, 651–666. McWhinnie, H. J. (1968). A review of research on aesthetic measure. Acta Psychologica, 28, 363–375. McWhinnie, H. J. (1987). A review of selected research on the golden section hypothesis. Visual Arts Research, 13, 73–84. Murphy, S. T. and Zajonc, R. B. (1993). Affect, cognition, and awareness: Affective priming with optimal and suboptimal stimulus exposures. Journal of Personality and Social Psychology, 64, 723–739. Nisbett, R. E. (2003). The geography of thought: How Asians and Westerners think differently … and why. New York: The Free Press. Nodine, C., Locher, P. and Krupinski, E. (1993). The role of formal art training on perception and aesthetic judgment of art compositions. Leonardo, 26, 219–227. Orians, G. H. and Heerwagen, J. H. (1992). Evolved responses to landscapes. In: J. Barkow, L. Cosmides and J. Tooby (Eds.) The adapted mind, pp. 555–579. New York: Oxford University Press. Overbeeke, K., Djajadiningrat, T., Hummels, C., et al. (2003). Let’s make things engaging. In: M. A. Blythe, K. Overbeeke, A. F. Monk and P. C. Wright (Eds.) Funology: From usability to enjoyment, pp. 7–17. Dordrecht: Kluwer Academic Publishers.

Product aesthetics

285

Pinker, S. (2002). The blank slate. New York: Viking Penguin. Plug, C. (1980). The golden section hypothesis. American Journal of Psychology, 93, 467–487. Postrel, V. (2003). The substance of style. New York: Harper Collins. Purcell, A. T. (1984). The aesthetic experience and mundane reality. In: W. R. Crozier and A. J. Chapman (Eds.) Cognitive processes in the perception of art, pp. 189–210. Amsterdam: North-Holland. Raghubir, P. and Greenleaf, E. A. (2006). Ratios in proportion: What should the shape of the package be? Journal of Marketing, 70 (April), 95–107. Ramachandran, V. S. (2004). A brief tour of human consciousness. New York: PI Press. Ramachandran, V. S. and Hirstein, W. (1999). The science of art: A neurological theory of aesthetic experience. Journal of Consciousness Studies, 6, 15–51. Reber, R., Schwarz, N. and Winkielman, P. (2004). Processing fluency and aesthetic pleasure: Is beauty in the perceiver’s processing experience? Personality and Social Psychology Review, 8, 364–382. Repp, B. H. (1997). The aesthetic quality of a quantitatively average music performance: Two preliminary experiments. Music Perception, 14, 419–444. Rhodes, G. (2006). The evolutionary psychology of facial beauty. Annual Review of Psychology, 57, 199–226. Rosch, E. (1978). Principles of categorization. In: E. Rosch and B. B. Lloyd (Eds.) Cognition and categorization, pp. 27–48. Hillsdale, NJ: Lawrence Erlbaum Associates. Scherer, K. R., Schorr, A. and Johnstone, T. (Eds). (2001). Appraisal processes in emotion: Theory, methods, research. New York: Oxford University Press. Schwartz, S. H. and Sagiv, L. (1995). Identifying culture – specifics in the content and structure of values. Journal of Cross-cultural Psychology, 26, 92–116. Silvia, P. (2005). Cognitive appraisals and interest in visual art: Exploring an appraisal theory of aesthetic emotions. Empirical Studies of the Arts, 23, 119–133. Smets, G. (1973). Aesthetic judgment and arousal: An experimental contribution to psycho-aesthetics. Leuven: Leuven University Press. Smets, G. (1982). A tool for measuring relative effects of hue, brightness and saturation on color pleasantness. Perceptual and Motor Skills, 55, 1159–1164. Snelders, D. and Hekkert, P. (1999). Association measures as predictors of product originality. Advances in Consumer Research, 26, 588–592. Thornhill, R. and Gangestad, S.W. (1993). Human facial beauty: Averageness, symmetry, and parasite resistance. Human Nature, 4, 237–269. Tooby, J. and Cosmides, L. (1992). The psychological foundations of culture. In: J. Barkow, L. Cosmides and J. Tooby (Eds.) The adapted mind, pp. 19–136. New York: Oxford University Press. Tooby, J. and Cosmides, L. (2001). Does beauty build adapted minds? Toward and evolutionary theory of aesthetics, fiction and the arts. Substance, 94/95, 6–27. Tractinsky, N., Katz, A. S. and Ikar, D. (2000). What is beautiful is usable. Interacting with Computers, 13, 127–145. Uehara, I. (2000). Transition from novelty to familiarity preference depending on recognition performance by 4-year-olds. Psychological Reports, 87, 837–848. Van Rompay, T., Hekkert, P., Saakes, D. and Russo, B. (2005). Grounding abstract object characteristics in embodied interactions. Acta Psychologica, 119, 315–351. Vartanian, O. and Goel, V. (2004). Neuroanatomical correlates of aesthetic preference for paintings. Neuroreport, 15, 893–897. Veryzer, R. W. and Hutchinson, J. W. (1998). The influence of unity and prototypicality on aesthetic responses to new product designs. Journal of Consumer Research, 24, 374–394. Walker, E. L. (1980). Psychological complexity and preference. Monterey: Brooks/Cole. Whitfield, T. W. A. (2000). Beyond Prototypicality: Toward a Categorical–Motivation Model of Aesthetics. Empirical Studies of the Arts, 18, 1–11. Whitfield, T. W. A. (1983). Predicting preference for familiar, everyday objects: An experimental confrontation between two theories of aesthetic behavior. Journal of Environmental Psychology, 3, 221–237. Whitfield, T. W. A. and Slatter, P. E. (1979). The effects of categorization and prototypicality on aesthetic choice in a furniture selection task. British Journal of Psychology, 70, 65–75. Wilson, E. O. (1984). Biophilia. Cambridge, MA: Harvard University Press. Wilson, E. O. (1998). Consilience: The unity of knowledge. London: Abacus. Winner, E. (1982). Invented worlds: The psychology of the arts. Cambridge: Harvard University Press. Zajonc, R. B. (1968). Attitudinal effects of mere exposure. Journal of Personality and Social Psychology, Monograph Supplement, 9, 1–27. Zhang, Y., Feick, L. and Price, L. J. (2006). The impact of self-construal on aesthetic preference for angular versus rounded shapes. Personality and Social Psychology Bulletin, 32, 794–805.

12

MEANING IN PRODUCT USE: A DESIGN PERSPECTIVE STELLA BOESS Delft University of Technology, Delft, The Netherlands

HEIMRICH KANIS Delft University of Technology, Delft, The Netherlands

1. MEANING IN PRODUCT USE Users create the meanings of products in use (for example Suchman, 1987; Krippendorff and Butter, 1989). The design of products contributes something to that situation of use. But what? And how? These questions have been discussed quite controversially for roughly the past 20 years. Suchman (1987) and other anthropologists showed how people’s use of products could be quite different from the expectations of designers, technologists, and organizations. Suchman observed people using a copying machine. Her studies revealed that the users acted in ways unexpected by the machine’s designers, and that the machine was unable to detect these actions. Suchman’s studies were a starting point for discussions on what constitutes interaction in product use. Suchman contended that rather than trying to help and guide a certain model user, machines should be built to communicate their state, and to allow users to make sense of a situation. Her efforts contributed to the emergence of the user-centred stance in design (Norman and Draper, 1986). As a result of this and other work, more questions were raised on whether people were able to use the things being designed for them, especially with regard to new technology. Several authors have revealed that product use should be problematized in design (for example Norman, 1988; Moore and Conn, 1985; Clarkson et al., 2003). These authors show how people are being disabled or discouraged from accessing and using commodities and public services in several ways. For example, people might be disabled by the physical dimensions of their environments. Or products might be stigmatizing their users. Or people may simply be unable to make sense of things. This last aspect, the way people make sense of things in use, is the focus of this chapter. We focus mainly on instrumental or denotative meaning in product use. Product Experience Copyright © 2008 Elsevier Ltd.

305

306

Product Experience

Krippendorff and Butter (1993) argued that it can no longer be assumed that it is possible to force ‘correct’ usage of products onto consumers. There is great variation, and indeed creativity, in the ways that people make sense of things. This is now being discovered in the design field, and various names are being given to it: For example ‘Thoughtless Acts?’ (Fulton-Suri, 2005), ‘Non-Intentional Design’ (Brandes and Erlhoff, 2006), or ‘Wild Things’ (Attfield, 2000). The terms seek to challenge the still prevailing idea that design is the exclusive domain of professionals. Design is also in the way people use things. We too have noticed that there is a great deal of variation in the ways people use things (for example Kanis, 1998). In this chapter, we propose that designers can integrate the users’ attribution of meaning in product use into their design process. The chapter reviews some of the concepts that have been discussed in relation to meaning in product use. We begin by discussing two important concepts: Product semantics and affordances. The history of both terms goes back further than their introduction into design theory. We’ll take them up at the point where they are introduced into design theory. We point out some of the successes and shortcomings of each concept in informing design. And we introduce a perspective of our own on meaning in product use. This perspective takes the situatedness of product use as its starting point. Within it, we have developed the concept of usecues. We will argue that it can be used to investigate meaning in product use.

2. PRODUCT SEMANTICS An important concept in connection with product meaning is semantics. It is in use in human–computer interaction and in product design. What is meant by product semantics? Krippendorff and Butter (1989) defined it as ‘a study of the symbolic qualities of man-made forms in the cognitive and social contexts of their use and the application of the knowledge gained to objects of industrial design’ (p. 10). Initially derived from semiotics (the study of signs), product semantics looks at form as language-like. Krippendorff and Butter (1984) relegate graphic elements, such as a brand name, labels or written instructions, to the realm of traditional semiotics because they point at something other than a product itself. Contrarily, a product’s form, shape and texture ‘is in the true sense indigenous to that product. (…) The symbolic meanings of forms, shapes and texture are the most characteristic concern of product semantics’ (p. 6). Products should be expressive about their function and purpose through shape and texture. Krippendorff and Butter (1989) note that people always see objects in a context of other things, situations, and users, including the observing self. That is why product semantics has the potential to be a truly human-centred methodology, and why there are many potentially appropriate product forms rather than one definitive, right form. Products that communicate clearly their intended context and possible use, can be of use to people in giving shape to their preferred lifestyle (Krippendorff and Butter, 1993). As Krippendorff and Butter (1989) explain: The slogan ‘form follows function’ thus implies abstracting the ordinary (scientifically naive, nonengineering-trained) user out of the equation and discarding the meanings that users construct and see. The increasingly appealing suggestion that form may not follow function but meaning, brings the user back into the picture and strongly suggests that designers need to discuss not only the contexts in which their forms are used, but also how these forms are made sense of or what they mean to someone other than themselves. (p. 15) A suitable starting point for product semantics is the experiential fact that people surround themselves with objects that make sense to them, they can identify as to what they are, when, how, for what, and in which context they may be used. (p. 11)

Meaning in product use: A design perspective

307

Product semantics was introduced as a post-modern successor to the modernist focus on ‘form follows function’. Product semantics abandons the aim of gearing product form towards technological rationalization, as the intention was in the 1960s. Product semantics became popular in the 1980s in product design. It was adopted as a replacement for what was increasingly being regarded as the straitjacket of Modernist methodology (Brown, 2006, p. 98). Designers and manufacturers embraced product semantics at a time in which it became important to differentiate products in the market, while the products had become technologically interchangeable. In parallel, digitalization meant that product functioning became separable from product communication. The digital insides of consumer technology were no longer mechanically connected to the controls that consumers saw and used (Krippendorff and Butter, 1993, p. 31). This new freedom created new options in designing functional product meaning. The focus on product communication that semantics offered was a welcome point of departure in this respect.

2.1. Product semantics applied As an example of the application of semantics in industry, we take a look at its adoption at Philips. Product semantics was introduced at Philips after Robert Blaich became Managing Director of its Corporate Industrial Design in 1980 (Blaich, 1989, p. 1). Blaich invited Krippendorff and others to present their ideas on product semantics at Philips Design. Blaich sought an approach that would help in designing novel products with novel functions and interactions. Product semantic concepts such as self-evidency and ease of use were communicated within Philips Design through workshops. Visual metaphor was championed as the primary design tool to imbue products with social, psychological, and cultural value. Products should be able to ‘speak’ about their operation and communicate something about their owners. A successful Philips product that embodies these ideas is the 1985 Roller Radio (Figure 12.1). It was introduced to improve Philips’ image among the youth market. Blaich (1989) explains its semantic approach: It utilizes product semantics in a total sense. The metaphor is mobility. The shape coding, integrated in the fixed handle versus a folding handle, says, ‘Carry me’. The speaker grills, devoid of fake chrome which is often used in products for this market, signal sound waves, while the back of the radio clearly shows the battery storage. And the small red ball on top of the antenna (a yellow ball in Figure 12.1) reminds one of a sports car. It is a youth-spirited design metaphor that says, ‘Keep on rolling.’ This product has sold in the millions since introduced in 1985 […]. Our rolling radio’s more recent badge of success is the number of copies now on the market by our competitors. (p. 7)

FIGURE 12.1 Philips 1985 Roller radio, a successful design using a semantic approach. Available from http://www.design.philips.com/About/Design/Section-13747/Index.html

308

Product Experience

Brown (1999) reviews the Roller Radio’s design and comments that ‘the small red ball on top of the antenna, supposedly reminiscent of a sports car, is perhaps a more tenuous reference, although it is in keeping with the visual metaphor of mobility’ (p. 122). Brown elaborates on the context in which the Roller Radio was brought to market. It was designed to address an audience that did not identify with ‘the archetypal “ghetto blaster” that was popular at the time. […] Blaich’s insight into the market, however, led him to recognize that not everyone wanted to adhere strictly to this “street” image, especially if it meant hauling around a large, grey and unwieldy box of “serious” electronics. The daringly colored and innovative form of the Roller Radio therefore attempted to satisfy the demands of these uncatered for sections of the market’ (p. 122). One main motive for the adoption of product semantics at Philips was clearly its potential in the market. Blaich (1989): ‘Differentiation is the key to market competitiveness, and semantics is a new design tool for achieving differentiation’ (p. 8). While some Philips products employing a semantic approach were successes in the market, there were also less successful experiments, and interest in product semantics waned towards the end of the 1980s at Philips Design (Brown, 1999, p. 130). Brown (1999) concludes, on the basis of a literature review and interviews with designers, that the methods to arrive at products were not yet fully developed. He points out that semantics had an influence on design practice, albeit in less theoretically informed approaches than those described in the literature. The general idea that product meaning was about expression and communication remained influential in the design field. Still, Brown (2006) proposes, the full potential of semantics in design has not yet been realized. The early semantic approaches, such as the Roller Radio from Philips, tended to hint at intended possible human–product interactions via fixed product form. Lately, interest in semantics has revived, with a stronger focus on the actions and characteristics of users as a starting point. For example, a new series of conferences is dedicated to Semantics of Form and Movement (www.desform2006.id.tue.nl).

2.2. Elements of product semantics On the level of concrete application in design, product semantics can be thought of as meanings associated with product characteristics such as form, dimensions, color, graphics, texture, transparency, fragility, grouping of product parts, etc. A combination of product characteristics, supposed by designers to have a particular meaning, are encoded in a design, subsequently to be decoded by users. In this view, communication primarily consists of the exchange of mental representations in a perceptive/cognitive process, which somehow thrives on experience and learning. Product semantics is about designing messages for the user. Vihma (1997) notes that Krippendorff’s proposal of product semantics takes its basic structure from information theory (p. 31). A scala of possible types of communication is considered. Krippendorff and Butter (1989), for example, outline four different contexts in which objects may have meaning: Operational context; socio-linguistic context; context of genesis; and ecological context. Vihma (1995) cites Jochen Gros’ theory of product language as an example of applied product semantics. The most important potential of Gros’ theory, according to Vihma, lies in the notion of ‘self-explanation’ of a product. ‘In design practice this property is regarded as a quality that is difficult to describe and design. It is a nonverbal expression by which the product exhibits its practical function. It cannot therefore be a property of the product only. Rather, it is a relationship between the product and the user, a sign’ (1995, pp. 38–39). Gros’ example for this, according to Vihma, is to design the grooves of a handle to express how to grasp a product. Further primary tools in

309

Meaning in product use: A design perspective

TABLE 12.1 Examples of perceptions/cognitions by users that are different from those intended by the design Featural and functional product characteristics Why not noticed

Why not understood

Not observable directly by user (e.g. on/off control on the back of video recorder) Missed (not conspicuous, escaped attention, such as icons on a screen) Unaware of something perceivable (functionality unknown, such as ‘mute’ on a remote control, so there is no search for information1) Indicated functionality not recognized (functionality as such is known, e.g. the possibility to select program on a remote control of more than one figure1) Noticed

Too much effort/obvious alternatives (e.g. icons on nozzles1)

Noticed

Misinterpreted (e.g. the term ‘auto’ as ‘car’ on the display of an electronic suction power regulation1)

Noticed

Meaning given to characteristic(s) not meant to be meaningful in the design (e.g. parting lines)

1

Kanis, 1998

designing product semantics are visual metaphor and the appeal to shared stories and histories, or ‘mythologies’ (Krippendorff and Butter, 1989, p. 38).

2.3. Product semantics and product use In studies like Vihma’s, and in Krippendorff’s methodological texts, attribution of meaning is discussed on a general, theoretical level, rather than empirically, on the basis of the observation of user activities. In observational studies, we have identified graphics as frequently occurring product semantics. By this we mean that we have observed people use them to derive meanings from products. These graphics, however, tend to be used by designers to indicate product functions, rather than to point towards possible ways of usage (Kanis, 1998). This is an example of miscommunication between designers, striving to make functions accessible, and users, trying to make sense of what a product is and what it does. See also the miscommunications shown in Table 12.1, above.

3. AFFORDANCES 3.1. The introduction of the affordance concept in design Also in the 1980s, and partly overlapping with the discussion in the field of product design, there were developments in the field of human–computer interaction (HCI) seeking to address product meaning. These approaches were informed by the traditions of artificial intelligence research and cognitive science, rather than by the traditions of product design

310

Product Experience

theory. One of those approaches is Donald Norman’s ‘cognitive engineering’ (Norman and Draper, 1986; Norman, 1988). A cognitive psychologist, Norman diagnosed a gap between internal processes of the mind and external physical processes. This gap, he thought, was crucial in human–computer interaction and was not being bridged successfully by design. The gap consisted of a ‘gulf of evaluation’ and a ‘gulf of execution’. These gulfs meant that people could not make sense of what they saw computer systems do, and that computer systems in turn offered insufficient opportunities to act on them. The gap was caused by the different communication and interaction faculties of humans and machines (Norman and Draper, 1986, pp. 38–39). Norman found that this problem applied not just to computers, but also to everyday things. He searched for fundamental concepts that could help in designing for human– environment interaction. Norman adapted James Gibson’s concept of ‘affordances’ as a notion for the ‘perceived and actual properties of a thing, primarily those fundamental properties that determine just how the thing could possibly be used’ (Norman, 1988, p. 9). Norman’s version of the affordance concept gained acceptance in the field of software design. Product designers who were aware of Norman’s ideas also adopted the affordance concept (Moggridge, 1993). But the application of the concept was not straightforward. Norman (1999) subsequently criticized some of the attempts at application. We first review James Gibson’s original concept briefly, before tracing some of its adaptations in the design field.

3.2. James Gibson’s theory of affordances The concept of affordances originates from James Gibson’s ecological approach to perception (Gibson, 1979). We draw on a summary by McGrenere and Ho (2000) of Gibson’s approach. They tell us that Gibson, a perceptual psychologist, thought that an animal’s visual perception should not be studied through physics alone, in isolation from environments. Gibson sought to identify a more appropriate frame of reference: An ecological one. Gibson claimed that we (and other animals) perceive at the level of mediums, surfaces, and substances rather than at the level of particles and atoms. In particular, we tend to perceive what the combination of mediums, surfaces, and substances offer us (McGrenere and Ho, 2000). Gibson formulated the concept thus: ‘The affordances of the environment are what it offers the animal, what it provides or furnishes, either for good or ill’ (1979, p. 127). Affordances are perceived directly, hence Gibson’s term ‘direct perception’. We understand affordances as self-evident environmental possibilities/opportunities for living organisms (animals, humans) in being supported, protected or threatened. These possibilities/opportunities are perceivable by an organism through a direct coupling between this organism and its environment: […→ acting → perceiving → acting→ …], without a specified mental mediation. An affordance as introduced by Gibson has a simultaneous foothold in the agent organism, as well as in the environment. The environment offers affordances, and humans have ‘effectivities’ through which they can perceive these affordances in their interaction with the environment. The combination of affordances and effectivities specifies invariantly what a given environment offers to a given human. With his approach, Gibson (1979) sought to cut across the dualism that perception could only be studied as either purely subjective (phenomenal), or as purely objective (physical). McGrenere and Ho (2000) explain that Gibson’s affordances are intended as objective in that their existence does not depend on value, meaning, or interpretation. And they are subjective in that an actor is needed as a frame of reference. The combined subjective/objective nature of the affordance concept has led to much subsequent

Meaning in product use: A design perspective

311

disagreement. On the one hand, Gibson proposed a new, ecological definition of ‘what value and meaning are. (…) Any substance, any surface, any layout has some affordance for benefit or injury to someone. Physics may be value-free, but ecology is not’ (1979, p. 140). On the other hand, he also wrote that affordances ‘are in a sense objective, real, and physical, unlike values and meanings’ (1979, p. 129). What to make of this contradiction?

3.3. A contradiction in the theory and two arguments to explain it We have identified a contradiction in the theory of affordances, in that it seems to include and exclude values and meanings at the same time. We suggest that this contradiction can be understood by means of two arguments. 1. The first argument is that Gibson was somewhat over-optimistic about the possibility of describing human action entirely, including meaning, in one unifying concept. Particularly with regard to this first aspect, authors have criticized the theory. 2. The second argument is that Gibson did not actually seek to specify action, but the potential of action. This distinction was not fully understood at the time Gibson formulated the theory, and was frequently misunderstood subsequently (as argued for example by Norman, 1999). The first argument: Gibson was overoptimistic in describing action in one unifying concept With the theory of affordances, Gibson sought to unify disparate discipline views on (human) action into one concept. We argue that it was overoptimistic to think that one concept can provide this. To illustrate our argument, we focus briefly on a wellknown example given by Gibson in the Ecological Approach to Visual Perception (1979). Gibson drew on a postbox as an example of the theory of affordances (Figure 12.2). Gibson stated that anyone above the age of six would know what a postbox is for (1979, p. 139). With Gibson, one might say that the visible form of a black slot in a hollow container supports the insertion of a flat object to the extended, objectclutching hand of a person (compare Gibson, 1979, p. 133). But Gibson stated simply that the postbox affords letter-posting. Gibson’s ecological psychology mixed perceivable objects (a real postbox) with cultural conventions (a postal system). This mixing has led to many objections. For example, Palmer (1999) compared the similar physical appearance of trash bins and postboxes. Palmer commented that for postboxes to afford mailing, postal service employees must remove objects from the box and deliver them to the post office. This property of postboxes ‘cannot possibly be perceived from their projected optical structure without the mediation of associations stored in memory’ (p. 412) (Figure 12.2). Such associative influences introduce a blurriness into what exactly is an affordance and what is not. The concept misleads us in apparently suggesting that it can help predict human–environment interactions for the purposes of design. If we acknowledge the diversity of meaning in product use, which Krippendorff and Butter (1993) pointed out as being important, it becomes clear that the concept, taken on its own, cannot capture meaning in product use. Krippendorff (2006) incorporates the concept of affordances into his more encompassing framework of product semantics. Gibson based the approach of direct perception on evolutionary notions. In that view, animals (which can be humans) are seen as able to orient themselves and be active in their ecological niche, with its familiar life-supporting features. Gibson primarily derived the

312

Product Experience

FIGURE 12.2 A postbox and a trash bin. Gibson (1979) posited that humans pick up information in their environment efficiently. Approaching a postbox, it remains perceptually the same postbox, even though its shape apparently changes (images on left and middle). Picture on right: Palmer’s (1999) objection to Gibson’s idea of direct perception was that people could only distinguish between such perceptually similar objects as a postbox and a trash bin by making use of associative meanings.

theory from observations of human behavior in physical environments. He developed his main ideas throughout the 1950s to 1970s. Perhaps Gibson actively sought to develop an alternative viewpoint to an emerging emphasis on cognitive processes. Since then, however, the computerization of consumer artifacts and surroundings has permeated daily life in Western societies to an extent that was perhaps not yet foreseeable at the time. The farreaching artificiality and digitalization of many of our surroundings makes it unlikely that evolutionary notions are sufficient to explain and to design for human–product interaction today. Couplings between environmental features and the activation of functionalities can be quite arbitrary. While this is often seen as a loss to be compensated, it also conceivably opens up new possibilities for interaction. For example, Svanaes and Verplank (2000) argue that the substance of objects in terms of hardware and software may no longer be detectable in their action possibilities. They propose that this undetectability increases the need to study and use metaphors to convey product meaning in interaction. The second argument: Affordances specify the potential for action, not action Gibson’s aim was to legitimize the study of perception-in-action in relation to the real human environment. We argue that his aim was never to pre-specify interaction by means of affordances. Affordances specify the potential interactions that humans can have in their niche. The variety is huge, as Gibson acknowledged: ‘Detached objects must be comparable in size to the animal under consideration if they are to afford behavior. But those that are comparable afford an astonishing variety of behaviors, especially to animals with hands’ (1979, p. 133). Gibson sought to distinguish his theory from a phenomenology as subjective experience, where environmental features play a role only in so far as they have a ‘demand’ character. Gibson (1979) cites Koffka’s Gestalt theory, where ‘valences’ are available to a person in relation to an experienced need: ‘Koffka argued that the postbox has a demand character only when the observer needs to mail a letter’ (p. 138). Gibson, in contrast, pursued the aim of identifying measurable invariants of perception. Gibson claimed that ‘an affordance is not bestowed upon an object by a need of an observer and by his act of perceiving it. (…) I prefer to say that the real postbox (the only one) affords letter-mailing to a letter-writing human in a community with a postal system’ (1979, pp. 138–139). With this Gibson sought to make both the real postbox, and the human effectivity of perception, amenable to study. They needed to be studied in relation to each other. As Gibson (1979) explains, properties of a surface for support, such as horizontal, flat, extended and rigid, ‘would be physical properties if

Meaning in product use: A design perspective

313

they were measured with the scales and standard units used in physics. As a surface of support for a species of animal, however, they have to be measured relative to the animal. They are unique for that animal. (…) They have unity relative to the posture and behavior of the animal being considered’ (pp. 127–128). But experimenters could not hope to apply the results of such study to an (agent) observer, ‘only to make it available, for it is not a stimulus’ (p. 141). So if affordances cannot be used to predict human– product interaction, what can they be used for? Investigations of human–environment interaction potential might, for example, be useful in identifying pathologies. Joel Norman (2002) reports on experiments in which he used the concept of affordances to point out the connection between physical, physiological, and neurological entities: Patients with partial, ‘semantic’ dementia could still stack and thread things, but no longer name them or describe their purpose. Such investigations can also be conducted in order to learn more about the active, ecological perceptual faculties of humans. Gibson acknowledged that much was still unknown about this human efficiency. Gibson sought to show that humans pick up information in the ‘ambient optical array’ (1979, p. 65), a concept created to posit a truly 3D perception process. Gibson suggested that human minds do not perceive each feature of the surrounding environment as separate perceptual data to be put together into meaningful objects. Rather, the active human perception system works efficiently in identifying environmental features in terms of their shape and meaning simultaneously, while humans are moving around. Thimbleby (2002) suggested that symmetry plays a role in this efficiency of information processing. While it is still quite unclear how the ambient optic array works, it may in time become an interesting starting point for efforts to make human–computer interaction truly spatial. So far, this aspect of Gibson’s theory has been little explored in relation to design. As it leads beyond our discussion of meaning in product use, we leave it at that and turn to the impact the affordance concept has had in design. We conclude from the above reading of Gibson’s arguments that he did not set out to pre-specify action, but rather that he set out to characterize its potential and make it available for further study.

3.4. Affordances: A mixed blessing for design The concept of affordances has proven a mixed blessing for design. When Norman first popularized it in design, particularly interaction designers tried to make use of it. Affordances became controversial in design when Donald Norman (1999) complained about the misuse of the term for featural characteristics such as icons. Norman argued that designers confused perceptual possibilities with cultural conventions. Designers were designing screen objects such as buttons and wondered whether they afforded clicking. Gaver (1991) interpreted affordances in this broader way toward which Norman (1999) directed his complaint. Gaver discussed screen elements such as buttons in terms of their affordances, or their ‘clickability’ (Figure 12.3).

FIGURE 12.3 As Gaver (1991) argued, onscreen buttons seem to protrude from the screen; they afford pushing, but not moving or editing. Figure based on Gaver (1991).

314

Product Experience

Conversely, Norman (1999) argued that a mouse affords clicking anywhere on a screen because this is physically and perceptually possible, regardless of where a button might be displayed on that screen. If anything, Norman argued, such a button provided a ‘perceived affordance’ informed by cultural conventions. Gibson however, who invented the concept, may well have accepted that particular screen objects have a perceived clicking affordance for certain people if this has become part of the familiar features of their ‘niche’, of their environment, as we saw in the postbox example (Figure 12.2). Later, Norman (undated) moderated his critique somewhat and acknowledged the relevance of perceived affordances for design as a form of communication, as proposed by De Souza (2005). Understandings and misunderstandings of the concept are discussed for example by You and Chen (2007) for the design field, and for other fields by Joel Norman (2002) and in a special issue of the journal Ecological Psychology, see e.g. Chemero (2003). As our focus here is on informing actual design efforts with notions of meaning in product use, we move on to take a closer look at applications of the concept. Some designers did not adopt the concept, seeing it as overly complicated or constraining. Many product designers remained unaware of it because of a lack of communication between the human–computer interaction and product design communities. In the following, we present some analyses and design efforts that have looked at the ways affordances work out in relation to actual products.

Relation between form and p\urpose The affordance concept was intended as a means to shed light on the interaction possibilities between organisms and their environment. However, authors have struggled to come up with clear examples of effective affordances that could inform design. For example, Oshlyansky, Thimbleby and Cairns (2004) conducted a simple study on the affordances of light switches in the US and in the UK. In the UK, light switches point downwards when switched on and upwards when switched off, whereas in the US it is the reverse (Figure 12.4). The authors concluded from their empirical study that the simple household light switch provides no universally understood information as to how it changes a system’s state. People who were familiar with the light switches tended to use them more successfully. When people were shown a light switch of the unfamiliar kind, they had to guess at its functioning. Oshlyansky et al. (2004) concluded that the affordance of the switch has been learned in a cultural context – in effect, that is has no affordance. The design of the switch itself provides no affordance for its successful use.

FIGURE12.4 US style switch (left) and UK style switch (right). Figure based on Oshylansky et al. (2004).

Meaning in product use: A design perspective

315

The authors concluded that more attention needed to be paid to the effect of cultural conventions, rather than affordances proper, with regard to meaning in product use. Distinguishing between affordances and semantics Djajadiningrat invested considerable effort into explorations of how one could deliberately design affordances in a product. In Djajadiningrat et al. (2004), the result of this effort is presented. Djajadiningrat designed a video recorder using only affordances that invite effective action to convey meaning for product use. He purposely excluded a semantic approach from his design effort. For example, he endeavors to mark the position and workings of the power supply (Figure 12.5). Djajadiningrat et al. (2004) explain how this appeals to the direct perception of users: It ‘breaks the contour of the outline of the videodeck. It features a switch whose ribs either ‘allow’ or ‘block’ the flow’. ‘The mains transformer is emphasized through its ribbed housing. […] It is a fair guess that the power on/off switch is positioned close to the mains transformer and to where the mains cable enters’. Djajadiningrat et al. (2004) present no observational studies on the use of this prototype, so our analysis is based on their own description and our interpretation of it. We find that in the example, power itself is not actually made perceivable. Its functionality is shown symbolically, as a flow that can be interrupted. How would this work for someone who doesn’t know how power works? Such a person may not associate anything with the ‘flow’ symbolism. Power, to that person, could as easily be a kind of touch, or a form of feeding. Such a person might be perfectly able to live in a technologically developed society, switching things on and off using learned conventions, without ever associating power, or the form of a switch, with flow. Oshlyansky et al. (2004) made a similar observation about light switches, as seen in the example above. We conclude that Djajadiningrat’s intended affordance, expressed in the appearance of the power switch, cannot be assumed to indicate power. Indeed, Djajadiningrat et al. (2004) concluded that to invite actions from users was not a sufficient approach in itself. Instead they investigated how products can offer feedforward, communicating what the result of a user action will be. Djajadiningrat et al.’s power switch example shows that semantic and symbolic meaning comes into product meaning even when one purposely tries to exclude it. We suggest that clearly identifiable affordances, or for that matter clearly identifiable semantics, cannot reliably be designed into any real world situation of product use. Neither did Gibson intend this application of affordances, as we have seen above (1979, p. 141). Gibson only intended the concept as an aid in understanding potentials for action, not as a tool for designing (and predicting) human–product interaction.

FIGURE 12.5 The power switch, symbolizing flow. Figure based on Djajadiningrat et al. (2004).

316

Product Experience

Affordances as representations of digital information Frens (2006) documents the emergence of ‘highly interactive products’, where interface-controls and feedback elements are coupled through electronics, opening up interactive functionality in their behavior. He highlights the special new challenge of designing for interaction: ‘Interaction (…) is present only in the relation between product and man, and is therefore not directly modifiable. Instead it is only given shape indirectly, through form and product behavior’ (p. 20). Löwgren and Stoltermann (2004) describe the digital technology that is involved in this as a ‘material without properties’ (p. 3). This indirectness and lack of properties means that concepts are needed to give shape to intended interactions. The field of tangible computing, for example, explores the possible correspondences between machine or computer behaviors and human behaviors (for example Ullmer and Ishii, 2000). These efforts often draw on affordances as basic patterns of human–environment interaction possibilities. The efforts mainly result in ‘tangible user interfaces (TUIs), (which) offer dedicated physical objects that represent the digital information to control and manipulate that information’ (Frens, 2006, p. 36). The development of TUIs provides theory and building blocks for the increasing ubiquity of computing (Weiser, 1999). Frens points out that information-for-use remains as yet under-addressed by these efforts (2006, p. 46). The field remains focused on technology and data representation, rather than on human activity. Combining tangible interaction and affordances, Frens proposes an approach of ‘rich interaction’ to offer a balanced view on people’s physical, cognitive, and emotional skills (2006, p. 56). Frens demonstrates the approach of rich interaction with a variety of digital camera prototypes. Functionalities are accessible through aesthetically pleasant physical actions. Frens evaluates the users’ preferences and the observed ease-of-use of the cameras. Frens’ experiment provides pointers towards preferable interaction styles for different camera functions. Yet Frens’ evaluation of the various camera models is not conducted in relation to everyday life situations of use, but in an experimental setting and with pre-set hypotheses and tasks. The work essentially remains at a technology-centred level. A case in point is also You and Chen’s (2007) proposed framework combining affordances and semantics. They assert that ‘the affordance plays a significant role in the user–product interaction’, before ever applying their framework to any real-world situations of use. They plan to conduct studies ‘using the framework as a tool to analyze users’ reactions to product appearance’. We should point out that the challenge also remains to be attentive, first and foremost, to the situated experience of human–product interaction (for example Forlizzi and Battarbee, 2004). Wensveen, Djajadiningrat and Overbeeke (2004) acknowledge the necessity of being attentive to freedom and playfulness of interaction. We should furthermore balance technological development with critical approaches. Such approaches challenge implicit assumptions that technology design alone will eventually result in a better product milieu. For example, Hjelm (2005) urges designers to engage in critical design efforts. Digital technology innovations, Hjelm argues, should be made visible and discussable, rather than embedded unnoticeably in human– environment interactions. Pitfalls in applying affordances in design for interaction To summarize, there are pitfalls in applying the affordance concept in designing for real-world interactions. People turn out to be using highly familiar and common products without making use of affordances. Deliberately designing affordances turns out to be difficult, if not impossible. And the use of affordances in design does not lead to

Meaning in product use: A design perspective

317

attentiveness to freedom in human–product interaction. The pitfalls may be caused by these aspects of the concept: • Its linking character. In apparently suggesting invariance of meaning, it tends to explain away the situatedness of interaction – the fact that meaning arises in interaction in diverse ways. As we have explained, Gibson probably did not intend to suggest that interaction is invariant, only particular affordances in potential action. • Its two-part nature. Examples often lose the human part of the human–environment relations they describe, and end up as descriptions of object characteristics. But characteristics of products could never be taken as sufficient for describing human–product relations in terms of affordances. What is often forgotten in the design-related discussion is that a human agent’s characteristics, in terms of effectivities or capabilities (Michaels and Carello, 1980), are by definition part of the affordance concept. They need to be included in order to ‘co-explain’ the variety in user activities. Gibson would say that ‘climb-up-ability’ of a set of stairs is only an affordance for a certain set of users: Those that can climb up those stairs. • Its suggestion of a self-evident, direct coupling between agent and environment. The suggestion is that when appropriate affordances are provided, human–environment behavior is of a smooth, automatic character, in essence pre- or non-linguistic. This is illustrated by such terms as ‘walk-onableness’ of surfaces (Gibson, 1979, p. 127), or ‘sit-onableness’ of chairs, which often feature in attempts to clarify what an affordance is (for example Norman, 1988, Gaver, 1991). It is further exemplified in technologycentred efforts in tangible computing and electronic product design. While this conception may hold potential for inspiration for design, it also carries an inherent risk. It may implicitly lead to an automatic, even mechanistic view of human action that neglects the diversity and freedom of product use. For design, product meaning can be seen as an open structure We have seen that the affordance concept has pitfalls when trying to apply it. Because it can be interpreted in quite opposite directions, it is inherently blurry. Some have sought to use it to address interaction, but it has also been used in attempts to prespecify human–product interaction in design (Vera and Simon, 1993; and as criticized by Norman, 1999). Such attempts deny the insight that human–product interaction is, and remains, partly impossible to predict. Rather than to dwell on the question when something is an affordance and in what sense exactly, we want to take on board the insight that interaction itself has not yet been sufficiently addressed. To explore interaction, we take meaning in product use as an open structure – as not pre-specifiable – and look at questions arising about it in design.

4. ANTICIPATING MEANING IN PRODUCT USE 4.1. The unpredictability of use on the basis of prior theoretical considerations We approach the topic of meaning in product use by taking as a starting point the perspective of design. Design deals with ‘wicked problems’ that need to be identified and resolved discursively, and that have good or bad solutions (Rittel and Webber, 1973). Consequently, design as a practice needs discursive tools to structure the communicative process in design teams and with stakeholders (Jonas, 1997). Design can benefit from basic findings, but needs to move beyond them. The process of design is more than an

318

Product Experience

application of knowledge. It is also a generation of knowledge and a generation of facts. Many aspects of design problems cannot be resolved by prior reasoning or generalized guidelines. They can only be addressed in the course of a design process and through it. Both product semantics and affordances are primarily theoretical concepts that aim towards scientific generalizations. They were not originally conceived as pragmatic notions, to be used as tools in designing for interaction. They have mainly been used in order to pre-specify (parts of) interactions, rather than to look at interactions empirically. Of course, a notion shedding light on the diversity of users’ actions and reducing their unpredictability would be of help to designers. But observational studies (for example Kanis, 1998) show that, even though designers try to anticipate on product meaning, the intended communication frequently fails. Meanings, presumably designed into product characteristics, often remain unrecognized by users. Seeing the results of observational studies can be an unwelcome surprise for designers: Users may fail to access designed functionalities and end up frustrated. Even accidents can occur (Kanis, Weegels and Steenbekkers, 1999). Kanis (1993) found that in real product use, the relevance of human characteristics, such as sensory, mental, and physical capacities/limitations, is largely constrained to setting boundary conditions for interaction. The boundaries mainly indicate what users will not do since they are unable to. For the actual user activities within the boundaries set by human capacities/limitations, human characteristics, generally, appear to be uninformative. Our studies have shown that it is precisely around the boundaries of usability that the greatest variation in usage occurs. Even activities of individual product users tend to be characterized primarily by freedom: Using something one way at one time, and another way another time. Bernstein (1967) characterized this for movements, and von Brunswik (1952) for cognition. Von Brunswik conceives ‘vicarious functioning … as the essence of behavior’ (p. 700). He proposes that behavior should be seen in terms of the equifinality principle, which means that a final state may be reached from different initial conditions and in different ways (p. 750).

4.2. Imagining product use in design Over roughly the past 15 years, we have studied how designers and design students attempt to anticipate on meaning in product use. Recently, we conducted a study comprising eight interviews with practising designers in the Netherlands that included independent designers, designers from small design firms, and designers from a large, internationally operating design consultancy. The study was conducted in order to gain a sense of how professional designers in industry currently handle meaning in product use. The study reveals that working with meaning in product use can be a source of some confusion for designers. Some designers stated that the issue of interactive product meaning has been resolved, and is a thing of the functionalist past. Still, in describing concrete design projects, the designers often state that they use their own intuition to design for product meaning. They hardly refer explicitly to any of the theory and methodology that is available, and when they do so, then only cautiously. Rather, they tend to refer to past product examples. In order to bring something of the context of the designers’ work to this chapter, we have reworked an excerpt from the interview study into a narrative form and present it in the following. The excerpt is taken from an interview held in August, 2006 with M. R., interaction designer at Philips Design. Ideas are easy to generate, if you’re used to working creatively. ‘This could be good’ and ‘that would be handy to solve existing problems’. But after that you have to work on detailing these ideas, you have to make it logical. How exactly will it work? For example, if you’re thinking about controlling a TV through a wand-like device (Figure 12.6), you might start with some ideas based on movements

Meaning in product use: A design perspective

319

FIGURE 12.6 M. R. demonstrating and sketching the usage of the wand prototype.

with a wand. And you might create a virtual behavior of screen objects that responds to movements of the wand. You have a wheel idea, you give a push to a wheel, and objects pass before you on the screen. With the wand, you can make a swinging motion to push the wheel. Then you have to work the logic out further. For example: You want to access sets of videos that are stored on a computer, via a TV. A video data file you might be accessing is quite big, so you might say it’s quite heavy. When you give the wand a twirl, the files will pass quite slowly before you on the screen. Mp3 music files, on the other hand, are quite small. Quite light-weight, say. They will move much more quickly on the screen. So you’ll be trying to put real-world principles into the virtual world to make it more logical for a user. You’ll be trying to create a link. Those kinds of principles are not new, they have existed for quite a while. But of course, you don’t know whether the user would want it. They’re nice principles, but they’re for somewhat advanced users, people who would value it. It’s all possible, it can all be done, but will the user understand it, especially a beginner or someone less experienced? It’s nice that the folder is heavier, but you have to work harder with the device to move it. Is that desirable? Would one not simply want to move them all equally easily? It quickly becomes a gimmick. That’s one of the reasons why it wasn’t implemented in the final product. On the one hand you’d be thinking about how people might interact with it, but on the other hand, you just want to get it to work conveniently in the first place.

The excerpt shows a designer anticipating and trying to imagine how people will use a product. Making assumptions on how a product will be used is often an integral part of design. In the absence of tools and a legitimation for observing the usage of their designs, designers often rely on intuition, previous experience or corporate design guidelines when making decisions that will later influence how people can use a product. Suchman (1999) reflected on a long period of experience with various corporations and noted that testing during product development was more often a matter of intentions than of real strategies and actions. Yet much has been done to show that there would be benefits in looking at product use and its problems directly. Over the past twenty years, it has been demonstrated in various ways that there is a gap between presumed product use and real-world product use. Perhaps Norman and Draper (1986) were the first to put

320

Product Experience

this argument strongly in product design. Krippendorff and Butter (1993) and Green and Jordan (2002) suggested that consumer choice is leading to greater demands on design quality and usability. And yet, recent studies suggest that companies still do not take into account the consumer perspective on product (use) quality and reliability (Den Ouden et al., 2006). The business side of this problem is beyond the scope of this chapter. We focus in the following on the activity of product use in relation to the activity of design. Can it be made easier for designers to consider product usage while designing?

4.3. Conceptualizing product use for design: Situatedness The difficulty for designers in predicting product usage successfully, we argue, comes from the fact that user activities (perception, cognition, action including the experienced effort) tend to be largely unpredictable on the basis of prior theoretical considerations. Instead of seeking to find the right formula, the right ingredients for predictability of product usage, we have taken on board Suchman’s (1987) argument that meaning is made in an interaction, not prior to it. Suchman, a sociocultural anthropologist, was one of a number of anthropologists that joined cognitive psychologists in studying the use of high technology products in the early 1980s. Suchman’s (1987) analysis particularly concerned a copying machine that was newly designed to guide the users in a particular way, based on a plan of use. Suchman observed breakdowns in the communication between human and machine, diagnosing that the machine had only limited access to the ways in which the users were trying to make sense of it and achieve their task. Suchman proposed that such breakdowns were not given due attention in the prevailing approach to technology design of the time. In that approach, it was assumed that the design of an appropriate plan could ideally prevent unexpected situations and breakdowns. Suchman, however, argued that plans, breakdowns, and repairs were all part of behavior in realworld situations. Suchman (2007, p. 21): ‘Plans are just one among many types of discursive artifacts through which we achieve the rational accountability of action. As such, they arise through activity and are incorporated into the activities that they project’. The notion that actors in a situation seek in various ways to achieve rational accountability of action is part of a theoretical perspective that goes by the name of ethnomethodology (Garfinkel, 1967). Suchman’s recent extended second edition of Plans and Situated Actions (2007) looks at the character of such situated activity in more depth. It takes recourse to Actor Network Theory in proposing the study of how human and machine activity are in relation to each other and to a socio-material situation. Suchman (2007, pp. 257–258): ‘What if our starting place comprises configurations of always already interrelated, reiterated sociomaterial practices? What if we understand persons as entities achieved only through the ongoing enactment of separateness and always in relation with others?’ For the purposes of our studies on meaning in product use, two of Suchman’s (1987 and 2007) conclusions are particularly salient. The first is that product usage can be seen as situated and sociomaterial. The second is that insights on product usage should be derived from observing enacted product usage. Suchman’s first conclusion: Product usage is situated and sociomaterial To illustrate Suchman’s first conclusion, we refer to a story with which Suchman introduces her 1987 book. In naval navigation, the European navigators made a plan, and sailed according to plan, correcting course where necessary. Anthropologists have long been keen to study the Micronesian navigators because of their different approach (compare also Hutchins (1995), who discussed several ethnographic studies of the Micronesian approach). These navigators, Suchman (1987) recounts, had a goal, and

Meaning in product use: A design perspective

321

then interacted with the elements they encountered by observing them, interpreting them and making use of them as they sailed. This was an activity learned through apprenticeship interactions and embodied experience. The two different approaches to navigation are two different ways of acting, and they lead to different tools that each of the approaches would use. The first approach would lead to tools that measure the adherence to the overall goal, correct deviations from it, and minimize environmental influences. The second approach would lead to tools that support interpretation and extrapolation of each environmental feature encountered, and that support the monitoring of reactions to those environmental features. If we see product use as an open structure, with situated sociomaterial practices, we design for that second approach. Suchman’s second conclusion: Product usage needs to be observed in order to learn about it Suchman’s second, related conclusion is that product usage needs to be observed in the situation where it occurs. Real, enacted situations can reveal the sociomaterial strategies and practices of people, and provide starting points for product development based on the users’ approaches in making sense of things. Suchman’s 1987 work was pioneering in design in that it did not try to identify and isolate factors that could be used to predict interaction, but that it took interaction itself as a phenomenon that cannot be ‘explained away’ by means of prior reasoning. It would follow from Suchman’s conclusion that one needs to research what happens in product use by experiencing it, or by observing how someone experiences it. Several authors have argued similarly. Schön proposed in 1983 that there is much to gain for designers in studying product use (see also Bennett, 1996). Designers, Schön argued, can make and argue design decisions better, and be enriched in their design practice through the feedback they get. Plowman (2003) points to a longer history of examples of usage research for design, citing Henry Dreyfuss’ work in the 1960s as an important example. Norman (1999) pointed urgently to the necessity of doing research into real use contexts, and testing designs against assumptions. He criticized the designer-centred perspective that one could simply decide, a priori, what the affordances of a design would be. Working with situatedness in design We accept that a suitable approach has to investigate and describe how meaning in product use is established in situations, by involved actors. Usage is best seen primarily as consisting of situated activities, prompted by what people encounter from moment to moment in a partly self-made situation (compare Suchman, 1987). Suchman’s work, along with the work of others (for example Winograd, 1996; Dourish, 2001), explores situatedness and embodiment in human–product interaction. Rather than being a universal theory aiming for predictability, the concept of situatedness thrives on the absence of invariant structures applying across situations (Suchman, 1987, p. 67). So in contrast to affordances, which Gibson argued were invariant, situatedness does not have a structure that can be specified in advance. That is why Suchman (2005) refers to ‘indexical affordances’ – affordances that are in reference to the work of individuals through perceptions and actions, in situations. Notions like ‘plan’ or ‘task’ can possibly predict tendencies in human behavior (compare Activity Theory in Nardi, 1996). However, such notions appear to be largely ineffective for predicting actual user activities. Around the mid-1990s, we started working with the view of human–product interaction as situated. We sought concepts that could usefully inform design. However, Suchman’s approach did not initially place specific emphasis on human interaction with the thing world. Rather, it was oriented towards social facts, towards human actions and

322

Product Experience

communications directed at establishing social order and meaning. In 2002, Suchman, Trigg and Blomberg suggested that technology development should be done via prototypes, and via iterative explorations of usage of such prototypes. Yet in terms of meaning in product use, Suchman did not develop a particular terminology. Rather, as we have seen, she later adopted the term ‘indexical affordances’ (2005). Suchman (2007) examines the relations between humans and machines fundamentally. Still, no particular terminology is proposed. Various terms used by other authors are discussed briefly, including affordances. In our work, we sought ways to express the tangibility and materiality of situated human–product interaction. We sought to complement Suchman’s original work with a new term to denote how product meaning is generated when people use things, alone or together, rather than primarily in the interaction between people.

5. USECUES FOR RESEARCH ON PRODUCT USE We adopted the term ‘usecues’ as a conceptual and pragmatic tool to describe meaning in product use (Kanis, Rooden and Green, 2000). As the concept of usecues is aimed at interactivity, it should not be seen in a positivist way, as radiated messages just waiting to be discovered and understood by the user. Rather, the concept is intended to be useful in conceptualizing how users make meaning with objects and environments they encounter. Usecues are what users make of featural and functional product characteristics in perception, cognition, and action. A button on a remote control, or a slot in a postbox, are not in themselves usecues. Rather, form, color, texture, or a button’s amenability to rotation, can become usecues to users. A usecue is any characteristic that people use (not might use) to attribute functional meaning to a product. For example, if a button is round, ribbed, and easy to turn, then these characteristics could be applied by a user as usecues that indicate that the button can be rotated. Usecues may also work in context of each other. In the case of a control panel, for example, the relative positioning of buttons to each other can have a usecue meaning for a user (note: not the buttons, but the positioning of the buttons). Usecues are meanings that users give to product characteristics, about: • the functionalities a product has, i.e. the possibilities to support, protect, replace, extend human activities, and also, in as far as desirable; • the possibilities of activating these functionalities. Usecues are intended as a pragmatic, bottom-up notion. They are not framed as cognitive, ecological or other processes that might be regarded as fundamental. Usecues are more ‘down to earth’: The actual ‘voice’ of a product in practice in terms of its functionalities. At the design stage, usecues can best be seen as opportunities. A user may take them up in a situation of use – or not. The success of this depends on several aspects, the role of which must ultimately emerge in empirical product tests. Of course it is likely that some general phenomena do play a role in the emergence of user activities. Theories about product usage may very well be found to apply in observed product use. For example, the tendency to adopt a low level of cognitive involvement as a default, or the tendency towards skill-based actions in terms of Rasmussen’s model (1983, 1986). Other phenomena might be the occurrence of habituation (Lambert, 2002), or the fixation of users trapped by past experiences in particular ways of use (Standaert and Christiaans, 2000). Usecues can be seen to resemble what Vihma calls ‘indices’ (1995, p. 114) and what Krippendorff calls ‘affordings’ (2006, p.120). The usecues concept does not deny

Meaning in product use: A design perspective

323

the applicability of theories to usage episodes. But it gives pride of place to the aspects of interactions that are not pre-specifiable in generalities. Usecues are intended to be an open term, to be used to presuppose and observe interactional product meaning. By an open term we mean that by formulating usecues, we try to anticipate on, but not prescribe or predict interaction prior to observing it being enacted.

5.1. Empirical approach Our acknowledgement of the situatedness of product usage leads us to advocate an empirical approach to learning about meaning in product use. We have shown that designers tend to try to anticipate what product usage will be like. This can be done in a more explicit way by formulating presuppositions. Specifically, one can try to anticipate which usecues may play a role in the perceptions, cognitions and actions of people. Then one can observe how a usage episode takes its course. When one observes the use of a prototype or product, it is important to look in an open-minded and receptive way. But observation can be selective. For example, an observer might be biased and tacitly assume that users need to be advanced enough to understand a design. If such a bias is not made explicit, it can color the observation and conclusions. In order to make such assumptions explicit, one can adopt the seemingly paradoxical technique of formulating them as points of attention prior to observation. This technique can serve to ‘bracket’ the assumptions. Once one has bracketed an assumption, one can observe and ask – is this really there, what is part of it, how is it manifested? What else is there to see?

5.2. Story of use The connection to sociomaterial practices makes it useful to embed presuppositions about usecues into a story of use. In embarking on empirical research using the usecues concept, a researcher can draw up a presupposed story, for example in the form of a storyboarded scenario (see for example Figure 12.8, p. 327). Such a visualized story displays a logic and approach of human action in a situation. Having made such a story, one can then look for the presumed usecues of a product or concept design in that story.

5.3. Usecues for perception, cognition and action Usecues can be described in terms of the basic human possibilities of perception, cognition, and action. A usecue can relate to any or all of these three aspects that users may apply to specific characteristics of a product. This way, product characteristics are provisionally linked to particular ways in which they may have meaning for people (or not). Observed product use can then be analyzed by evaluating how the product characteristics have fared in it. Conclusions can be drawn on improving those product characteristics or on adopting a new style of interaction. It may be argued that perceptive and cognitive activities of human beings are difficult to disentangle. They are not, however, always mutually convertible. Think, for example, of the difference between not noticing and not understanding: In design this distinction may make all the difference. Table 12.1 gives examples of this practical understanding of perceptions and cognitions. The examples are based on findings from empirical studies (Kanis, 1998). They show how users may perceive and understand product characteristics differently than intended in a design. The users may misunderstand messages, or miss them altogether.

324

Product Experience

Not only do product users perceive and understand product meanings differently than intended. Users may also carry out rather different actions than those a designer intended. Users may prefer their own ways of operating a product, or postpone an intended action. Sometimes, users carry out actions with a product smoothly, without even noticing or understanding the cues a designer sought to give. This can even happen when someone uses a product for the first time.

5.4. Observed usage Usecues and user actions can best be investigated by means of explorative research into user–model/–prototype/–product interaction. The actions that users carry out in operating a new design can be observed (unobtrusively, if desirable and possible). In order to access the situatedness of action, explorative research into product usage should be carried out in as natural a context as possible. For the purposes of design, however, it is often preferable to test a product early in a design process. Simulation of usage situations offers ways to test early. There are many ways and degrees that everyday usage situations can be observed in simulation before a design is finalized. For reasons of observability and confidentiality, users can be asked to try out a product in an environment that can easily be recorded on video. Users can be asked to carry out simulations of real product usage, and they can be asked to try out products at various stages of design. A design can be tested at the stage of a drawing, an early model, an appearance model or a functional prototype (Rooden, 2001). All of these simulations of real usage can be evaluated at appropriate stages of a design process.

5.5. Design outcomes Observed usecues can be said to be a ‘sediment’ of use actions that have been observed. They are neither stable properties of a product or environment, nor are they stable properties of a person. They arise from the enacted, lived relation between people and things. The fact that usecues remain tied to practices means that abstraction is only possible to a limited extent. There can never be a general catalog of usecues to be applied to products. The usecues concept always remains closely tied to product usage episodes. Design outcomes can always be regarded as prototypes for a subsequent study of product usage. Design implications from studies of usage episodes may simply be to add, leave out or rearrange elements of a product. Sometimes, the whole character and the action possibilities of a design may be rethought, inspired by the observed user actions. Depending on the overall design goal, one may decide to make a product’s presumed usecues more obvious or more sophisticated, one may decide to build more redundancy into the usecues a product offers, or one may decide to develop a range of product types that each cater to different ways that people have used (parts of) a product.

5.6. Limitations Escaping determination There is some inherent difficulty in delineating what is, and what is not, a usecue. Users, in deciding what to do or not to do next, often make sense of obvious characteristics such as the sound a product makes when functioning, or that it has become warm. Such characteristics are then usecues by definition, having been used as such. They may not have been designed as intended usecues at all, but may have become part of a product for entirely different reasons. A designer may simply not have hidden them, as

Meaning in product use: A design perspective

325

opposed to having displayed them deliberately. Or a designer may have implicitly accommodated a design to (presumed) current habits, practice, customs, or cultural conventions. It appears inevitable that each designer will individually develop their own sets of ideas about what might be usecues in a product. There is nothing wrong with that, provided that usecues (designed, presumed) can be made explicit, i.e. capable of articulation, and that they are used as a pragmatic tool for drawing attention to the consequences of decisions made during a design process. Apparent utilitarian bias The usecues concept can be misunderstood as applying only to utilitarian, workrelated contexts in design. The concept does not, for example, explicitly and prominently point designers towards affective aspects of product use. Crabtree, Rodden and Benford (2005) argued that the need to achieve rational accountability of action is present in a fun-oriented activity such as gaming as much as in any other activity. They defended the broad applicability of ethnomethodological approaches. The usecues approach focuses on product use at a perceivable and tangible level. It does not help designers in conceptualizing an overall design – how it will fit in a lifestyle, which emotional responses it seeks to elicit, what kind of place a whole situation around a design should be. It does not pre-suppose wider-ranging goals or cultural contexts. But it can provide findings on interactions and meanings that can be a source of inspiration for new design ideas.

6. USECUES IN THE DELFT DESIGN COURSE We conclude this chapter with an example of how students in the Industrial Design Engineering course in Delft have been applying the usecues concept in their usage research and their design. Observation of product use using the usecues concept enjoys popularity among the students and is considered a valuable learning experience by them. What may make the term usecues attractive to industrial design students is its articulation of something from which designers cannot escape: The creation of featural and functional product characteristics which are or can be transmitters of messages (voices) for users. Even if only used by hindsight, thinking in terms of usecues has been found to facilitate the identification of possible deficiencies in a design underway, such as lacking ‘directions for use’ in a prototype, or ambiguous or misleading cues (compare Table 12.1).

6.1. A project example In a compact three-month design project at Delft University of Technology, groups of five students work on a brief to design, build, and test tabletop products with mechanical and electronic functionalities. A typical brief is, for example, to design a foam cutter, a table saw, an electric perforator or a towel dryer. We support the students in thinking their product through in terms of usage opportunities and messages it offers – in terms of the usecues and other usage aspects. With this, we intend to enable the students to then look more open-mindedly at what actually happens in the use of their product, less hindered by implicit and unrecognized expectations. We present the work in this project using the case of a student group who designed a foam cutter (Figure 12.7). The case was part of action research we recently conducted (Boess, de Jong and Kanis, 2004). While

326

Product Experience

6 2 3 4

1a, 1b 5 1c

FIGURE 12.7 Left: the foam cutter designed by the student group. The numbers point to presumed usecues. See Table 12.2 for corresponding information. Right: the machine in use in a test with four participants (two shown here). Top picture, with upright arm. Bottom picture, with tilted arm.

FIGURE 12.8 Excerpt of the storyboard drawn by the student group to anticipate on the usage of the foam cutter they designed.

the students are designing and producing prototypes, they draw up a storyboard (Figure 12.8). They make presuppositions on usecues and usage, pose research questions and set up a video-recorded usage evaluation with a small number of previously uninvolved ‘test’ users. For an overview of presuppositions and observed use of this student group’s design, see Table 12.2. The students then compile a research report in which they look back on their prior presumptions about expected product usage and about the usecues of their product. The students reflect on surprises that emerged from the research, and make redesign proposals (Figure 12.9). The student group found that the four users they observed interpreted some usecues differently than expected. As can be seen in Table 12.2, the students found that their presumed usecues turned out to be used in context of

TABLE 12.2 Usecues, presuppositions about use, and observed use as researched by the student group (shortened version)

Presumed usecues 1a. Graphics above

1b. Clustering of controls on right 1c. Led to indicate ‘on’ 2. The asymmetric placing of the arm gives a hint that the arm can tilt at an angle.

3. Placing of a degree-arc in front of the arm

4. Presumably missing usecue: what the 0-point adjustment ruler is for.

5. Holes in the work surface show that it can be removed to change the wire.

6. A graphic indication on the end of the arm shows that the cap can be taken off to replace the wire.

The students’ presuppositions about perception, cognition and action Users understand and use the controls: feedback feedforward are obvious and familiar. Graphics on white ground are easily noticed. – The on/off button is marked with a 1 and a 0. – The continuous speed dial is marked with 0, a turtle and a rabbit (as in other electric tools). We chose not to use numbers so that users can adjust speed intuitively. – On the temperature dial, three thermometers showing different levels are shown. Here too, the idea is to adjust temperature intuitively. The arm is to the right of the machine. Users will probably tend to stand where the arm is. It follows also to cluster the controls on the right. The users notice led lighting up and realize that the machine is switched on. We found no good reason why the arm of styrofoam cutters is always placed centrally. Moving it to the right creates more workspace and the arm can be tilted and folded away. We expect that users will find the asymmetry of the machine strange at first. But we think they will be able to make good use of the extra workspace. In any case we think users will not be hindered by the asymmetry. We expect that the degree-arc also hints at the fact that the arm can be tilted. The degree-arc is easy to use as it shows the angle directly when the user tilts the arm. The arc itself is conspicuous, and pretty much everyone will associate the arc with an angle. When a user tilts the arm, the point changes where the wire exits the work surface. The integrated movable ruler can be set to 0 at the wire exit point, so that exact lengths can still be cut. Users will not notice the ruler until they tilt the arm. They notice the ruler as the wire moves away from the 0-point. We opted to make the work surface removable by cutting two half circle holes in it at the sides. Users can put a finger in each hole to lift the surface out. Users will notice that the holes are for this, because it is a common feature. But they notice it only when searching for a way to lift the surface. A cap on the end of the arm conceals the mechanism for replacing the wire. We expect that users will only notice the graphic indication for this when they’re looking for a way to replace the wire. Only then will such a cap attract attention at all. The graphic indication is intended to help in this.

Observed use (selection)

Test participant 4 (Tp4) first manipulates the temperature button, expecting that this will switch the machine on. Then he tries the speed button, and only then the on/off button.

None of the four participants realized at first that the arm can be tilted. To cut foam at an angle, they held the piece of foam at an angle or tilted the work surface (see pictures: Tps 4 and 3). No-one understood the purpose of the 0-ruler. The holes (5) for fingers were used as presumed

Tp 3 replaces the wire. (Observed use:) Tps 1 and 3 find the cap, but are in doubt whether they are attaching the wire correctly.

328

Product Experience

FIGURE 12.9 Redesign proposals of the students based on the analysis of observed product use. Top: test participants did not realize that the arm could tilt. The students proposed a large arrow on the arm, to suggest that it could move along the degrees of the arc. Below: test participants were unsure how to attach the wire and whether the wire would retain the tension they gave to it. The students proposed a clearly visible hook-up point for the wire and a ratchet mechanism to indicate that the wire would keep the tension.

each other, rather than in isolation. While the students had taken care to think carefully about product meaning, they found that users acted unexpectedly in several ways: • The users did not approach the product controls as expected. Rather than first perceiving the controls as a cluster and reading and interpreting the purpose of each control, users just picked a familiar looking one and started with that without pondering for long. Then they explored the other controls one by one. The students did not make a redesign proposal on the basis of this observation. They judged the problem to be minor and transitory. • The users did not notice the tilting function of the arm. They quickly found that the work surface could be lifted out, and used it to try and measure an angle on the degree-arc. The students concluded that the degree-arc could do more to indicate that the arm could tilt. They designed a conspicuous graphic addition to the degree-arc to indicate this (Figure 12.9). The question remains whether the students were really satisfied with this redesign proposal. Often, students comment that they can easily think of graphic additions, but would rather not make them in order to keep the overall design of their product intact. • The users had difficulty in attaching the wire correctly. They easily found out that the wire ends were concealed by the cap and the work surface. But they were then unsure whether the wire was attached properly. The students’ redesign was a new mechanism

Meaning in product use: A design perspective

329

for attaching the wire (Figure 12.9). By winding the wire on a ratchet spool that only moves in one direction, they simplified attaching the wires.

6.2. Reflection Students who take part in this project often start out naively denoting any distinguishable featural or functional product characteristic as a designed usecue or state that their design has been ‘usecued’. Throughout the project, we tend to find that the students’ thinking about product use becomes more differentiated and they have found a vocabulary with which to discuss meaning in product use. However, we also note some limitations of the approach we take in the design project. Often, students fail to identify observable usage problems or only analyse them superficially. Another finding is that the students only tend to make detail or graphical suggestions in their re-design proposals following their usage research (Boess et al., 2004). One reason might be that the detail and directness of the video material has a fixating effect on them as designers. The more convincing the material, the more it fixates designers to that status quo (see also Rooden, 2001). This would suggest that usage research may need to be followed by an in-between step, in which designers can step back from the data and look at their design afresh, as a whole. This can give them freedom to again develop new design ideas – and then test them afresh.

7. CONCLUSIONS This chapter has discussed some of the terminology that is used in design to describe product meaning. We propose a perspective in which the situatedness of product use is central. In our view, such a terminology should be practical and close to the way designers work. Usecues, we argued, can only ever be presumed while designing. The preliminary results of design work should be tested in a situation of product use. This is because usecues are ultimately ‘created’ by a user, through use, in that situation of use. A designer and/or researcher can observe this and also (subsequently) discuss it with a user. Through such research, it may become apparent that people recognize and use different usecues in a product than a designer intended. That is why it is useful to embed observational studies and use testing into an ongoing design process. This also leads us to express a preference for prototyping as part of a design process. We have shown that it is important to offer design terminology and methodology that is close to the actual design process, and at the same time close to the situation of product use. While in design the argument is often raised that present-day use cannot provide an inspiration for the future, the question remains how we will give form to the transitions from the present to the future – from existing situations to preferred ones. We have argued that it is useful to start by making explicit presuppositions on what is going on in use situations, and how ethnomethods of users are connected to usecues of products and situations. This explication can be used to anticipate on future situations, through co-discovery of product use situations using prototypes.

ACKNOWLEDGMENTS We gratefully acknowledge the contribution of the designers that took part in our interview study. It helped us shape the views laid out in this chapter. Our thanks also go to the students of the Ontwerpen 4 course for doing their best to learn about product usage, and for providing us with material to think with.

330

Product Experience

REFERENCES Attfield, J. (2000). Wild things. The material culture of everyday life. Oxford: Berg. Bennett, J. (1996). Reflective conversation with materials, an interview with Donald Schön by John Bennett. In: T. Winograd Bringing design to software. Addison-Wesley. Retrieved March 20, 2007 from http:// hci.stanford.edu/bds/9–schon.html, unpaginated. Bernstein, N. A. (1967). The co-ordination and regulation of movements. Oxford: Pergamon Press. Blaich, R. (1989). Philips Corporate Industrial Design: a personal account. Design Issues, 5(2), 1–8. Boess, S. U., de Jong, A. M. and Kanis, H. (2004). Usage research in the Delft design project Ontwerpen 4. In: P. Llyod, N. Roozenburg, C. McMahon and L. Brodhurst (Eds.) The changing face of design education 2nd International Engineering and Product Design Education Conference, Delft, NLF: Fac IDE, Delft University of Technology, 577–584. Brandes, U. and Erlhoff, M. (2006). Non intentional design. Cologne: daab. Brown, C. (1999). Product semantics: the style of the information age? Unpublished doctoral thesis, Nottingham Trent University. Brown, C. (2006) Product semantics: sophistry or success? In: L. Feijs, S. Kyffin and B. Young (Eds.) Design and semantics of form and movement, DesForm 2006, pp. 98–103. Eindhoven, NL: Philips. Chemero, A. (2003). An outline of a theory of affordances. Ecological Psychology, 35, 181–195. Clarkson, J., Coleman, R., Keates, S. and Lebbon, C. (2003). Inclusive design: Design for the whole population. London: Springer. Crabtree, A., Rodden, T. and Benford, S. (2005). Moving with the times: IT research and the boundaries of CSCW. Computer Supported Cooperative Work, 14, 217–251. Den Ouden, E. L.,Yuan, P. J., Sonnemans, M. and Brombacher, A. C. (2006). Quality and reliability problems from a consumer’s perspective: an increasing problem overlooked by businesses? Quality and Reliability Engineering International, 22, 821–838. De Souza, C. (2005). The semiotic engineering of human–computer interaction. Cambridge, MA: MIT Press. Djajadiningrat, J. P., Wensveen, S. A. G., Frens, J. W. and Overbeeke, C. J. (2004). Tangible products: Redressing the balance between appearance and action. Journal for Personal and Ubiquitous Computing, 8, 294–309. Dourish, P. (2001). Where the action is: The foundation of embodied interaction. Cambridge, MA: MIT Press. Forlizzi, J. and Battarbee, K. (2004). Understanding experience in interactive systems. DIS2004, Cambridge, MA: ACM, 261–268. Frens, J. (2006). Designing for rich interaction: Integrating form, interaction and function. Unpublished doctoral thesis, Eindhoven University of Technology. Fulton-Suri, J. (2005). Thoughtless acts? Observations on intuitive design. San Francisco: Chronicle Books. Garfinkel, H. (1967). Studies in ethnomethodology. Englewood Cliffs, NJ: Prentice-Hall. Gaver, W. (1991). Technology affordances. CHI 91, New York, NY: ACM, 79–84. Gibson, J. J. (1979). The ecological approach to visual perception. Boston: Houghton-Mifflin. Green, W. S. and Jordan, P. (Eds.) (2002). Pleasure with products: Beyond usability. London: CRC Press, Taylor and Francis. Hjelm, S. I. (2005). Visualizing the vague: invisible computers in contemporary design. Design Issues, 21, 71–78. Hutchins, E. (1995). Cognition in the wild. Cambridge, MA: MIT Press. Jonas, W. (1997). Viable structures and generative tools – an approach towards ‘designing designing’. In: Proceedings of ‘Contextual design – design in contexts’, Stockholm: The European Academy of Design. Kanis, H. (1993). Operation of controls on consumer products by physically impaired users. Human Factors, 35, 305–328. Kanis, H. (1998). Usage centred research for everyday product design. Applied Ergonomics, 29, 75–82. Kanis, H., Rooden, M. J. and Green, W. S. (2000). Usecues in the Delft design course. In: P. T. McCabe, M. A. Hanson and S. A. Robertson (Eds.) Contemporary ergonomics, pp. 365–369. London: Taylor and Francis. Kanis, H., Weegels, M. F. and Steenbekkers, L. P. A. (1999). The uninformativeness of quantitative research for usability focused design of consumer products. In: Proceedings of the Human Factors and Ergonomics Society 43rd Annual Meeting, pp. 481–485. Santa Monica: Human Factors and Ergonomics Society. Krippendorff, K. (2006). The semantic turn. Boca Raton, FL: CRC Press, Taylor and Francis. Krippendorff, K. and Butter, R. (1984). Product semantics: Exploring the symbolic qualities of form. Innovation, 3(2), 4–9. Krippendorff, K. and Butter, R. (1989). On the essential contexts of artifacts or on the proposition that ‘Design is making sense (of things)’. Design Issues, 4(2), 9–39.

Meaning in product use: A design perspective

331

Krippendorff, K. and Butter, R. (1993). Where meanings escape functions. Design Management Journal, 4(2), 30–37. Lambert, J. V. (2002). Designing more habituation-resistant warning systems: Sokolov’s Cortical model. In: P. T. McCabe (Ed.) Contemporary ergonomics, pp. 437–441. London: Taylor and Francis. Löwgren, J. and Stoltermann, E. (2004). Thoughtful interaction design. Cambridge, MA: MIT Press. McGrenere, J. and Ho, W. (2000). Affordances: Clarifying and evolving a concept. In: S. Fels and P. Poulin (Eds.) Graphics interface, pp. 179–186. Montreal, Quebec, Canada: Canadian Human–Computer Communications Society. Michaels, C. and Carello, C. (1980). Direct perception. New Jersey, Englewood Cliffs: Prentice-Hall. Moggridge, B. (1993). Design by storytelling. Applied Ergonomics, 24, 15–18. Moore, P. and Conn, C. P. (1985). Disguised. US: Word Books. Nardi, B. A. (Ed.) (1996). Context and consciousness: Activity theory and human–computer interaction. Cambridge, MA: The MIT Press. Norman, D. A. and Draper, S. W. (Eds.) (1986). User centered system design: New perspectives on human– computer interaction. Hillsdale, NJ: Lawrence Erlbaum. Norman, D. A. (1988). The design of everyday things. New York: Currency-Doubleday. Norman, D. A. (1999). Affordances, conventions, and design. Interactions, (May/June), 38–42. Norman, D. A. (undated). Design as communication. Essay published on jnd.org. Last accessed 31 January, 2007. Norman, J. (2002). Two visual systems and two theories of perception: An attempt to reconcile the constructivist and ecological approaches. Behavioral and Brain Sciences, 25, 73–144. Oshlyansky, L., Thimbleby, H. and Cairns, P. (2004). Breaking affordance: culture as context. NordiCHI ‘04, Tampere, Finland: ACM, 81–84. Palmer, S. E. (1999). Vision science: Photons to phenomenology. Cambridge, MA: MIT Press. Plowman, T. (2003). Anthropology and design. In: B. Laurel (Ed.) Design research. Methods and perspectives, pp. 30–40. Cambridge, MA: MIT Press. Rasmussen, J. (1983). Skills, rules, knowledge: Signals, signs, and symbols and other distinctions in human performance models. IEEE Transactions on Systems, Man, Cybernetics, 13, 257–267. Rasmussen, J. (1986). Information processing and human–machine interaction: An approach to cognitive engineering. Amsterdam, The Netherlands: North-Holland. Rittel, H. and Webber, M. (1973). Dilemmas in a general theory of planning. Policy Sciences, 4, 155–169. Rooden, M. J. (2001). Design models for anticipating future usage. Unpublished doctoral thesis, Delft University of Technology. Schön, D. A. (1983). The reflective practitioner: How professionals think in action. New York, NY: Basic Books. Standaert, A. and Christiaans, H. H. C. M. (2000). Does fixation interfere with product usability? In: Proceedings of the XIVth triennial congress of the International Ergonomics Association and 44th Annual meeting of the Human Factors and Ergonomics Society. Santa Monica CA (USA): Human Factors and Ergonomics Society. 6-81–6-84. Suchman, L. A. (1987). Plans and situated actions: The problem of human–machine communications. Cambridge: Cambridge University Press. Suchman, L. A. (1999). ‘I have, more than ever, a sense of the immovability of these institutions’. Conversation with Lucy Suchman at Xerox PARC, Palo Alto, August 13, 1999. By Claus Otto Scharmer. Accessible at http://www.dialogonleadership.org/Suchman-1999.html. Last accessed 25 January, 2007. Suchman, L. A. (2005). A riposte to Natalie Jeremijenko. In: Electronic Book Review. Accessible at http:// www.electronicbookreview.com/thread/firstperson/sociomaterial. Last accessed 25 January, 2007. Suchman, L. A. (2007). Human–machine reconfigurations. Plans and situated actions (2nd Ed.) Cambridge: Cambridge University Press. Suchman, L. A., Trigg, R. and Blomberg, J. (2002). Working artefacts: ethnomethods of the prototype. British Journal of Sociology, 53, 163–179. Svanaes, D. and Verplank, W. (2000). In search of metaphors for tangible user interfaces. DARE2000, Elsinore, Denmark: ACM, 121–129. Thimbleby, H. (2002). Reflections on symmetry. AVI 2002, Advanced Visual Interfaces International Working Conference Trento, Italy. Available from www.uclic.ucl.ac.uk/usr/harold/srf/avi2002.pdf. Last accessed 14 February, 2007. Ullmer, B. and Ishii, H. (2000). Emerging frameworks for tangible user interfaces. IBM Systems Journal, 39, 915–931. Vera, A. H. and Simon, H. A. (1993). Situated action: A symbolic interpretation. Cognitive Science, 17, 7–48. Vihma, S. (1995). Products as representations. Helsinki: University of Art and Design. Vihma, S. (1997). Semantic qualities in design. Formdiskurs, 3(II), 28–41.

332

Product Experience

von Brunswik, E. (1952). The conceptual framework of psychology. International Encyclopedia of Unified Science, 1(10). Chicago: The University of Chicago Press. Weiser, M. (1999). The computer for the 21st century. ACM SIGMOBILE Mobile Computing and Communications Review Archive, 3(3), 3–11. Wensveen, S. A. G., Djajadiningrat, J. P. and Overbeeke, C. J. (2004). Interaction frogger: A design framework to couple action and function through feedback and feedforward, pp. 177–184. DIS2004, Cambridge, MA: ACM, Winograd, T. (1996). Bringing design to software. Boston, MA: Addison–Wesley. You, H.-C. and Chen, K. (2007). Applications of affordance and semantics in product design. Design Studies 28(1), 23–38.

15

PRODUCT EMOTION PIETER M.A. DESMET Delft University of Technology, Delft, The Netherlands

1. INTRODUCTION Emotion is a central quality of human existence, and most of our behavior, motivation, and thought is enriched with and influenced by emotions. Our relationship with the world is essentially affective, which means that all our interactions imply and involve emotions, whether these are interactions with the social world or with the material one. A product, or using a product, can elicit disappointment, attraction, shame, pride, disgust, contempt, admiration, satisfaction, fear, anger, and any other emotion a person may also experience in response to events, people, or actions of people. Ignoring the emotional side of product experience would be like denying that these products are designed, bought, and used by humans. The focus of the current chapter is on that emotional side, and the term ‘product emotion’ will be used to refer to all emotions experienced in response to, or elicited by, seeing, using, owning, or thinking about consumer products. Those who are involved in design processes may sometimes be inclined to regard the concept of product emotion as intangible, and therefore unsuitable for a structured approach. Several reasons can explain this attitude. First, the concept of product emotion is broad and somewhat indefinite, because products can elicit many different kinds of emotions. We can admire the latest stylish music player, be irritated by an annoying mobile phone, be fascinated by the transparent fragility of a porcelain cup, and so on. And although the touch of melancholy felt when coming across a long forgotten childhood teddy bear seems incomparable to the thrill of driving a motorcycle, both responses belong to the same wide spectrum of product emotions. Secondly, emotions are subjective, and individuals will differ with respect to their emotional responses to a given product. We all know from experience that one person may be attracted to the same car model that is disliked by another. Even more so, the same person may experience different emotions Product Experience Copyright © 2008 Elsevier Ltd.

379

380

Product Experience

to a given product at different points in time. Thirdly, products often evoke ‘mixed’ emotions. Rather than eliciting one single emotion, products can elicit multiple emotions simultaneously because many different product aspects can have an emotional impact, such as the general product appearance, particular details, implicit and explicit expectations, and associated, remembered and fantasized meanings. Given the subjective and mixed nature of product emotions, only few one-to-one relationships between product design and emotional responses can be identified. However, one of the main themes of my discussion is that emotions are less intangible when they are analyzed on the level of the underlying process. The process that elicits emotions is primarily universal, and lawful relationships in the conditions that underlie emotions can be identified. The current chapter discusses a model of product emotion that was developed to describe some of these universalities by specifying the structure of the process that elicits emotions. Although the focus will be on tangible products, the model also applies to intangible services. The model represents a psychological view on product emotion because it represents the cognitive mechanisms that intervene between seeing, using, owning, or thinking about a product, and the emotional outcome. By focusing on the underlying process, the model aims to impose some structure on the limitless number of possible emotion-eliciting situations associated with product design. The model builds on my previous writings about product emotions, and introduces some new insights. The main aim is to set up a general framework that might help to structure the discussion of questions related to the topic of product emotion, i.e. how product emotion can be distinguished from other product experiences, why and how product emotion is elicited, and to what extent designers can influence or ‘design for’ these emotions.

1.1. Scope and structure of the chapter The chapter consists of three sections. The first section is a general analysis of the phenomenon emotion, focusing on some basic questions: What is emotion, how can it be differentiated from other affective phenomena, what manifestations are associated with it, why we experience it, and how is it elicited. The review aims to provide some basic answers to these questions that, on the one hand, are drawn from recent insights in emotion theory, and on the other hand, are not estranged from the everyday notion of what (product) emotion is. Some of the latest insights in emotion psychology are discussed, building on fundamental contributions to the science of emotion. Besides emotion, mood and affect disposition are also briefly discussed because they influence, and are related to, emotion. Although the summary does not do justice to the complex and vigorous nature of the discussions documented in the research literature, it does provide some background knowledge, and a foundation for the model and sources of product emotions discussed in the subsequent sections. The second section introduces the key variables of a basic product emotion model that was developed on the basis of an appraisal approach to emotion. In addition, two alternative approaches to product emotion are discussed: The pleasure approach of Jordan (2000), and the process-level approach of Norman (2004). The third section discusses sources of product emotions and uses examples to illustrate the different ways in which product emotion can be elicited.

2. AFFECT AND EMOTION Human beings have always shown a passionate interest in expressions of, and reflections on, the experiential phenomenon we call emotion. There is no beginning in counting the

Product emotion

381

attempts to conceptualize and define emotion that have been introduced in the (research) literature, and every year new ones are published. We should remember that although the social sciences have embraced the concept, it originated from the domain of our everyday experience, and therefore is a layman’s rather than a scientific concept. In other words: We all have emotions, and thus, from experience we all know what they are. This applies to the emotions we have in response to products just as much as it applies to those we may have in response to significant events in our life. Our implicit knowledge of emotion comes with the advantage that design and emotion scholars do not need to overly bother themselves with theorizing or defining the phenomenon, but can instead focus their attention on design-relevant questions, such as how and when products elicit emotions, and how products can be designed to have a positive emotional impact on their users. At the same time, one should be aware of relying too much on the common sense concept of emotion because these intuitive concepts differ between people, which introduces the risk of using product emotion as a container term for anything related to product design that involves some element of subjective experience (for a discussion, see Demir, Desmet and Hekkert, 2006). Before we turn our attention to products, we will therefore focus on some basic questions about the nature of emotions: What is emotion? How can we distinguish emotion from other affective phenomena like mood? How can we differentiate between emotions? What is the function of emotion?

2.1. Core affect When describing their emotions experienced in response to buying, owning or using products, people often use words like emotion, mood, feeling, and sentiment, as if these are alike. Many designers, and design researchers, tend to do the same. Because these words do refer to different phenomena, our conceptual exploration of product emotion requires a basic understanding of their similarities and differences. A first and basic distinction is between states that are affective and those that are not (see Clore and Ortony, 1988). In emotion theory, the term affect, or affective state, is generally used to refer to all types of subjective experiences that are valenced, that is, experiences that involve a perceived goodness or badness, pleasantness or unpleasantness. This approach fits the view of Zajonc (2000), who proposed that affect is the simplest raw, and universal (nonreflective) positive or negative feeling. Emotion research shows a long tradition in using valence as a bipolar dimension to describe and differentiate between affective states (e.g. Bradley and Lang, 1994; Plutchik, 1980; Wundt, 1905). Russell (1980; 2003) introduced the concept of ‘core-affect’ by combining the affect dimension with physiological arousal in a circular two-dimensional model. According to Russell the experience of core-affect is a single integral blend of those two dimensions, describable as a position on the circumplex structure in Figure 15.1. The horizontal axis represents valence (that ranges from unpleasant to pleasant), and the vertical axis represents arousal (that ranges from calm to excitement). The words in the figure represent emotions that have been found to be often experienced in response to the appearance of consumer products (see Desmet, 2002). We always experience core affect: From the moment we wake up to the moment we fall asleep our core affect constantly responds to a wide variety of internal (e.g. hormonal changes, nutritional deficiencies) and external causes (e.g. events, people, objects, weather). Core affect can be neutral (the central point), moderate, or extreme (the periphery). Changes can be short lived or long lasting, and can be in the focus of attention (in the case of intense core affect), or a part of the background of the person’s experience (in the case of mild core affect). Core affect can be experienced in relation to a

382

Product Experience

Activated

Irritation Disgust Alarm

Astonishment Eagerness Curiosity

Disappointment Contempt Unpleasant Jealousy

Pleasant

Calm

Boredom Sadness Isolation

Inspiration Desire Love

Awaiting Deferent Calm

Fascination Admiration Joyfulness

Satisfaction Softened Relaxed

FIGURE 15.1 Circumplex model of core affect with product relevant emotions (Desmet, 2002; adapted from Russell, 1980).

particular stimulus, as in emotion, but can also be experienced without a relation to a particular stimulus, as in moods (see the discussion in the next sections). Occasionally the cause of a change in core affect is obvious, but at other times, we can undergo a change in core affect without knowing why. Core affect theory and other dimensional theories of emotion offer a simple, yet powerful way to organize product affect, because all possible affective responses to (seeing, buying, using, owning, thinking about, repairing, etc.) products can be described in terms of core affect. The activated unpleasantness for the heated irritation in response to a failing computer, the calm pleasantness for the soothing experience of sliding into a warm bath, the activated pleasantness for the exhilaration of ice skating, and the calm unpleasantness for the sadness from the memory of a broken crystal vase, can all be plotted on the circumplex model. A second important strength of dimensional structures is that they allow for dynamic continuous affect reports. Dimensional models have therefore been found to be especially useful for capturing the continuous changes in emotional experience that occur during listening to music (see Schubert, 1996), using a product (see Laurans and Desmet, 2006), or looking at a TV ad (see Aaker, Stayman and Hagerty, 1986).

2.2. Attributed affect: Emotions Core affect can account for an infinite number of emotional states and provides a basis for discussing similarities and differences among affective states. Note, however, that not all affective states are emotions. Two eliciting conditions distinguish emotions from other types of affective states. An emotion is experienced when (1) some stimulus excites a substantial and acute change, or jump, in core affect (Ekman, 1994a; Russell, 2003); and when (2) this change is attributed to some antecedent or ‘object’ (Frijda, 1994b). All emotions, including product emotions, imply and involve a relation between the person who experiences them and a particular object (Frijda, 1994b). Note that this object is not necessarily a physical object, but can also be a person, an animal, a company, etc. One is not just angry, but angry at someone, afraid of something, proud of something, in love with someone, and so on. So we are angry at our printer for a bad quality print, satisfied with our toaster for making the perfect toast, and bored by our telephone for its tedious ring tone.

Product emotion

383

The distinction between stimulus and object is important because they are not necessarily the same. One day my sister and I were picking flowers in the meadow next to our parents’ house. At one point in time my sister bent over to pick a flower and, without us realizing it, her back touched the electric cattle fence. The resulting shock gave her a big fright, and, not knowing what hit her, she became angry with me because she concluded that I must have kicked her in the back. Although the electric fence was the actual emotional stimulus, the object of her emotion was me. This example illustrates that the object is whatever one believes is making him or her experience the emotion, and that in some cases we can misattribute an acute change in core affect due to a particular stimulus to another object (see Schwarz and Clore, 1983). One can, for example, be angry at a colleague for some mistake that he was actually not responsible for. The same can also apply to product emotions. One can be angry at the television set because it appears to be broken as it shows a white signal. But the objective facts might tell another story; perhaps the television only appears to be broken because of an interruption in the signal supply; or one should actually blame oneself for accidentally disconnecting the signal cable. One can be irritated by the call centre operator when the irritation should actually be attributed to the company who hired the person. The retail industry is known for experimenting with intentionally generated misattribution effects. Supermarkets, for example, sometimes distribute the fragrance of freshly baked bread to generate pleasant feelings that the customer tends to misattribute to the bread that is for sale.

2.3. Free floating affect: Moods Mood typically refers to a diffuse affective state that is of lower intensity than emotion, but considerably longer in duration (see Davidson, 1994). Whereas emotions are acute, moods evolve gradually over time. One can be sad or cheerful for several hours or even for several days. As opposed to emotions, moods do not involve a particular attributed object (e.g. one is not cheerful at something), but are directed at the surroundings in general, or in the words of Frijda (1986) at the ‘world as a whole’. Whereas emotions are usually elicited by an explicit cause, moods typically result from a combination of causes (e.g. bad weather, and bad coffee, and traffic jam, etc.). There are at least three reasons why, besides emotion, mood is relevant for product design. The first reason is that products serve important mood-manipulating functions. People seek for opportunities to alter unpleasant and maintain pleasant moods, and will engage in activities to deliberately influence their mood state. These activities can involve and require particular products, such as shopping, taking a bath, playing a video game, making or listening to music, etc. In some cases products are literarily used as instruments for mood management because they are consumed for their direct positive effect on mood, like eating or drinking particular foods (candy, ice-cream), drugs (stimulants), or drinks (alcohol). The second reason is that mood has an influence on emotional responses. Although mood and emotion can be distinguished in terms of underlying conditions and manifestations, they are not independent. Moods involve a general tuning of emotional responses (see Frijda, 1993). This means that unpleasant moods lower the threshold for experiencing unpleasant emotions, and pleasant moods lower the threshold for experiencing pleasant emotions. A person who is assembling a wardrobe closet in a bad mood, will probably be more ready to get irritated when the task is more complex than anticipated than a person who is in a cheerful mood. Many salesmen will attempt to have a positive influence on the mood of a customer (by, for example, looking attractive, offering drinks, making jokes and compliments) because mood will most likely influence the emotional response to the actual object of purchase. The third reason is that mood

384

Product Experience

has an influence on human-product interaction, and as a consequence, also on product emotion (see Wensveen, 2005). We all know that our motivation and behavior is influenced by our mood. In the case of a good mood one will, for example, respond differently to a colleague’s request than in the case of a bad mood. The person who is assembling a closet in a bad mood may try to use additional force when confronted with ill-fitting parts, whereas a cheerful person might have taken the time to explore the situation before using force. Where the first person may break the part, and experience anger, the second may discover that he or she should try another part, and experience relief.

2.4. Dispositional affect: Attitudes Affect dispositions are relatively enduring, affectively colored beliefs, preferences, and predispositions toward objects, persons, or events (Frijda, 1986; Ortony, Clore and Collins, 1988; Russell, 2003). Some of our affect dispositions (also referred to as attitudes or sentiments) are innate, and others are acquired. Examples of innate affect dispositions are the preference for sweet and aversion for bitter tastes (Rozin and Fallon, 1987), and preferences for particular odours and for particular facial features and expressions (Etcoff, 1999). Humans also have affect dispositions that are acquired through conditioning and learning. One can acquire a taste for wines, particular fashion styles, social activities, etc. These acquired affect dispositions show substantial cultural and personal differences. The Dutch salted candy ‘drop’ (liquorice) is often experienced as disgusting by those who are not familiar with the taste. Note that affect dispositions can easily be confused with emotions because they use similar terminology (e.g. ‘I love chocolate,’ or ‘I hate jogging’). Nevertheless, being afraid of dogs (affect disposition) and being frightened by a dog (emotion) are essentially different states (Frijda, 1994b). Of course, we also have affect dispositions regarding products, such as a dispositional love for 2CVs, or product categories, such as a dispositional dislike for mobile phones. The same applies to brands or particular companies, or (consequences) of product usage (e.g. ‘I love the smell of freshly washed laundry that is drying in the afternoon sun’). These dispositions are highly relevant for product emotion because, as will be discussed later in this chapter, they are one of the key parameters in the emotion eliciting process.

2.5. Differentiating between emotions Their valenced nature distinguishes emotion from non-affective states, and the acute and attributed change in core-affect distinguishes it from other affective states. An important limitation of core affect theory and other dimensional models is that dimensions are not fully sufficient to differentiate between emotions. Emotions that are clearly different may be similar in terms of core affect, like anger and fear that are both high in activity and unpleasantness. In order to effectively distinguish between emotions, the dimensional perspective must be integrated with a categorical perspective (Russell, 1993). Researchers, working with a categorical approach, focus on the (measurable) manifestations or components that distinguish various emotions from one another. Most current categorical theories of emotions consider emotions as multicomponential phenomena that involve behavioral reactions, expressive reactions, physiological reactions, and subjective feelings. Behavioral reaction (e.g. running or seeking contact) is the action or behavior one engages in when experiencing an emotion. Emotions initiate behavioral tendencies like approach, inaction, avoidance, and attack (Arnold, 1960). Fear makes one want to run, love makes one want to approach or caress, and so on. Expressive reactions (e.g. smiling or frowning) are the facial, vocal, and postural expressions that are

Product emotion

385

part of the emotion. Each emotion is associated with a particular pattern of expressions (Ekman, 1994b). Anger, for example, comes with fixed stare, contracted eyebrows, compressed lips, vigorous and brisk movements, and, usually, a raised voice, almost shouting (Darwin, 1872; Ekman and Friesen, 1978). Physiological reactions (activation or bodily arousal, e.g. increases in heart rate) are the changes of activity in the autonomic nervous system that are part of the emotion. Emotions show a variety of physiological manifestations, such as pupil dilatation and sweat production. Subjective feeling (e.g. feeling happy or feeling angry) is the conscious awareness of the emotional state one is in, i.e. the subjective emotional experience. Although feelings are sometimes seen as a separate affective phenomenon, most researchers (Lazarus, 1991a; Ortony et al., 1988) agree that feeling is the subjective representation of emotion (or any other affective state). Note that feelings are often expressed in terms of one of the other components that constitute emotion. One can express the behavioral impact of an emotion (‘I was so angry, I felt like throwing my computer out of the window’), the expression (‘the game was so sad, I felt like crying’), or the physiological reaction (‘I was trembling from fear when I noticed the smoke coming from my kitchen’).

2.6. Function of emotion In everyday life emotions are often (mis)used as an excuse to justify irresponsible behavior: ‘I am sorry for shouting at you, but you must understand that my anger had taken control of me’, or ‘I know I shouldn’t speed, but I was so thrilled by the sports car’s power that I just couldn’t help myself’. It may therefore seem odd to discuss the function of emotions when basically considering them to be dysfunctional: As primitive responses whose purpose is much better served by our more advanced mental capacities (see Frijda, 1994b). Nevertheless, and contrary to this view, most contemporary emotion theorists view emotions as coherent, organized, and functional rather than dysfunctional systems (Smith and Kirby, 2001). In fact, the basic Darwinian presupposition that emotions fulfill some sort of function is probably shared by all psychologists (Frijda, 1994a). Darwin proposed that the best way to understand emotion is by considering its evolutionary history and contribution to the survival of the species and the individual. The key to understand why we have emotions with their particular characteristics lies in the ‘functional requirements’ created by the environments in which all animals live. ‘All organisms, in order to survive and maintain their populations, must find and ingest food, avoid injury, and reproduce their kind’ (Plutchik, 1984, p. 201). In order to survive, it is not sufficient for an organism to simply understand its situation; it has to be motivated to do something about it, and that is precisely what emotions do: They physically prepare and motivate the individual to contend with the adaptational implications of the eliciting situation. Fear comes with a tendency to flee, and anger with the tendency to attack, fascination with the tendency to explore. Frijda (1986) argues that these action tendencies, or states of readiness to respond, should be thought of as adaptive responses to events that have been important to us as a species in our evolutionary past, as well as in our current environment. The important advantage of action tendencies over rigid reflexes and fixed action patterns is that they enable flexibility, both in event interpretation and in response choice. Emotions ‘decouple’ behavior from the perception of the stimulus so that reconsideration is possible (Scherer, 1984). Fear creates a tendency to flee, but depending on the particular situation (the individual is cornered) an aggressive attitude to intimidate the attacker might be the better alternative. Emotions are functional because they establish our position vis-à-vis our environment, pulling us toward certain people, objects, actions, and ideas, and pushing us away from others (Frijda, 1986). This

386

Product Experience

basic principle applies to all emotions; the intense emotions that we may experience in situations that threaten basic survival needs, and the subtle emotions that we may experience in response to seeing or using a product. Pleasant emotions pull us to products that are (or promise to be) beneficial, whereas unpleasant emotions will push us away from those that are (or promise to be) detrimental to our well-being.

3. APPROACHES TO PRODUCT EMOTION An explanation of product emotion should meet three basic requirements. First, it should reflect both the individual and temporal variability in emotional responses elicited by products. Whereas one person may be attracted by a glass dinner table, another may feel contempt towards the same table, and whereas one person may be disappointed by the performance of a mobile phone, another may be pleasantly surprised by its innovative design. In addition, an individual’s emotional response to a given product may change over time. One may be, for example, initially satisfied with a new couch, but experience dissatisfaction after using it for some time. Secondly, the explanation should reflect the differentiated nature of product emotion. Products do not elicit mere like (attraction or pleasure) and dislike (aversion or pain) responses, but distinct emotions, such as astonishment, inspiration, fascination, boredom, sadness, jealousy, and many others. Moreover, we often do not feel a particular single emotion towards a product, but a combination of ‘mixed’ (and sometimes paradoxical) emotions. One person can be proud of a new pair of shoes and happy with the reaction of his or her partner, and at the same time irritated by the lack of comfort and afraid of damaging the delicate leather. Thirdly, the explanation should clarify the role of the product as a stimulus in the mechanisms that bring about product emotion. Although this third requirement seems obvious, it is important to mention because most emotion research is focused on the manifestations of emotions as such (e.g. facial expressions, arousal, or experience), or on the processes that underlie these manifestations (e.g. cognitive, biological, or physical), without detailing the relationship between these manifestations and the stimuli that elicit them. Although in the last decade the design research community has shown an increased awareness of the phenomenon of product emotion, the main literature sources indicate a surprisingly small interest in proposing general explanations of product emotion that meet the above stated basic requirements. Next to the appraisal approach discussed in this chapter, two basic approaches have been introduced, discussed, and applied: The pleasure approach (introduced by Jordan, 2000), and the process-level approach (introduced by Norman, 2004). Jordan uses a psychological pleasure-framework to explain various types of product pleasure, and Norman explains product emotion with a neurobiological emotion-framework that distinguishes several levels of information processing. Before introducing the appraisal approach, I will first briefly review these two alternative approaches to give a broader overview of theoretical perspectives in the design and emotion research domain. Readers will find that although they show some essential differences, these approaches are not mutually exclusive and share some basic assumptions and theoretical considerations.

3.1. Pleasure approach to product emotion In the field of human factors, Patrick Jordan (2000) introduced an influential pleasurebased approach. His main proposition is that traditional usability-based design approaches are limited and even dehumanizing because they tend to focus only on the fit of a product

Product emotion

387

to a person’s physical and cognitive characteristics. As an alternative, he advocates addressing the relationship between people and products holistically, judging the quality of designs on the basis of the wider relationships between products and the people for whom they are designed. Given this intention, he proposed a pleasure-based approach to human factors, in which pleasure with products is defined as the emotional, hedonic, and practical benefits associated with products (Jordan, 1999). The approach draws on a pleasure framework introduced by Tiger (1992) that distinguishes four conceptually distinct types of pleasure that people may seek: Physical; social; psychological; and ideological pleasure. Physio-pleasure has to do with the body and with pleasures directly derived from the sensory organs (such as touch, taste, and smell). Products are perceived with the sensory organs and therefore a direct source of physio-pleasure or displeasure. For example, a mobile phone can generate physio-pleasure because of its soft touch and elegant appearance. Socio-pleasure is the enjoyment derived from relationships with others. This type of pleasure is relevant for those products that facilitate social interactions. Some of Jordan’s examples are products that attract comments (like a piece of jewellery), or act as a focal point for social gatherings (like a coffee machine). Psycho-pleasure is related to people’s cognitive and emotional reactions, and has to do with the cognitive demands of using products. A text processor that is easy to operate provides a higher level of psycho-pleasure than one that is cumbersome and illogical, because the former processor enables the user to complete the task more easily than the latter. Ideo-pleasure is related to people’s values (i.e. pleasures from ‘theoretical’ entities such as books). Ideo-pleasure experienced in response to products is related with the values that the products embody. A product made from biodegradable materials, for example, might be seen as embodying the value of environmental responsibility. This, then, would be a potential source of ideo-pleasure to those who are particularly concerned about environmental issues. In his book, Jordan (2000) comprehensively discusses and illustrates how products can bring about each of these types of pleasures, and how to link the pleasures to particular aspects of product design (and displeasures to inadequacies with respect to certain product properties). The role of emotion in the framework is not discussed explicitly. The framework focuses on the general bipolar pleasantness dimension that applies to all affect (see Figure 15.1), and therefore does not explain the differentiated nature of product emotions. In addition, it seems more likely that emotions can be associated with all types of pleasure instead of with only the psycho-pleasure. In that sense it is not an approach to product emotion, but to the more general concept of product affect. The main value of the framework is that it can be used as a tool in the design practice. Jordan stresses that it is not a theory that is intended to give an insight into why people experience pleasure, but a tool that can make it easier for those involved in the design process to consider the full spectrum of the sorts of pleasures that products can bring. Given this intention, an important contribution of the work is that it clearly illustrates the layered nature of product affect, and some of the important variables that are involved in the underlying process, such as the sensory quality of the product (physio-pleasure), the social context in which the product is used (socio-pleasure), task-related concerns of the user (psycho-pleasures), and values of the user (ideo-pleasures). The following sections will demonstrate how the appraisal approach offers a theoretical basis for explaining the role of these variables in the process that elicits product emotion.

3.2. Process-level approach to product emotion Donald Norman (2004) proposed a framework of product affect that distinguishes between three types of affect and three corresponding design focuses. He used the multi-level

388

Product Experience

analysis of information processing discussed by Ortony, Norman and Revelle (2005) as the theoretical fundament for his framework. In this analysis, affect is considered at each of three levels of information processing: The reactive; the routine; and the reflective level. The most elementary, reactive, level involves fixed action pattern responses, such as reflexes or simple fleeing behavior. These are biologically determined responses, and an example is the impulse to immediately reject a bitter substance by spitting it out. At this level, no emotions are experienced but only simple affect, that is, an unelaborated positive or negative value that is restricted to the here and now (and does not persist in the absence of the stimulus). The second level is called routine because it is concerned with the execution of well-learned routine behaviors and skills. Routine level processes are able to quickly engage in or inhibit actions to correct for simple deviations from expectations. An example is riding a bicycle, an activity that involves routine actions to control balance and speed. At this level, primitive emotions are experienced, that is, emotions that have not yet been interpreted and cognitively elaborated, such as basic happiness and fear. The third level, the reflective level, is the most sophisticated because it involves all higher-level cognitive processes. In general, this level comprises consciousness together with all advanced cognitive skills, such as the ability to form generalizations, to make plans, and to solve problems. This level is responsible for the rich emotional experience that we assume is unique to humans. These are cognitively elaborated emotions that can implicate representations of the present, the future, or the past, such as relief and disappointment. Norman (2004) discusses how each of the three levels of processing are involved in affective product experience. His main claim is that each level involves a distinct type of product affect and a corresponding design focus. The first type, i.e. visceral affect, is perceptually based and corresponds with ‘visceral design’ that is concerned with product appearance. The second type, i.e. behavioral emotion, is expectation-based and corresponds with ‘behavioral design’ that is concerned with the pleasure and effectiveness of use. The third type, i.e. reflective emotion, is intellectually based and corresponds with ‘reflective design’ that is concerned with self-image, personal satisfaction, and memories. Similar to the pleasure approach, the process-level approach involves several distinct, theoretically independent, sources of product affect. Whereas Jordan distinguishes responses on the basis of differentiated needs, Norman distinguishes responses on the basis of levels of processing in the brain. Although in that sense different, the types of affect seem to correspond at least to some extent: Visceral affect corresponds with physio-pleasure, behavioral emotion with psycho-pleasure, and reflective emotion with socio- and ideo-pleasure. Although the cursory translation to three types of emotional design (i.e. visceral, behavioral, and reflective design) overly reduces the complex nature of design and design activities, the distinction between three levels of processing with each associated affective phenomena (simple affect, primitive, and complex emotions) is an important contribution to the design and emotion discourse, because it clarifies and illustrates the role of cognition in the process of product emotion, and provides us with a basis for explaining why and how products elicit emotional responses. The process-level approach differs with the appraisal approach in what types of responses are considered to be elicited by appraisal processes. Whereas Norman argues that appraisal is involved only in the reflective level, many contemporary appraisal theorists propose that appraisal does not necessarily involve high-level cognitive processing, and therefore is not only involved in reflective emotions, but also in reactive and behavioral emotional responses (see e.g. Lazarus, 1991a). In following these models, the model of product emotions proposed next includes appraisals that generate responses at all three levels of processing.

389

Product emotion

3.3. Appraisal approach to product emotion In the tradition of the cognitive theories of emotion that are discussed by, for example, Lazarus (1991a), Frijda (1986), Ortony et al. (1988), Scherer (2001), and Smith and Ellsworth (1987), Desmet (2002) introduced a basic model of product emotions, shown in Figure 15.2. The model is ‘basic’ because it applies to all possible emotional responses elicited by (buying, using, or owning) products, and because it identifies three key variables in the process of emotion eliciting: (1) concern; (2) stimulus; and (3) appraisal. The proposition that the function of emotion is to motivate adaptive behavior in response to one’s circumstances implies that the eliciting conditions must involve some evaluation of the event’s significance for this person’s well-being. This evaluation is often referred to as ‘appraisal’ to signify that it is non-intellectual and automatic rather than conscious and deliberate (Arnold, 1960). A product appraisal has three basic possible outcomes: The product is (potentially) beneficial; harmful; or not relevant for personal well-being. These three general outcomes result in a pleasant emotion, an unpleasant emotion, or the absence of an emotion, respectively. The point of reference in the appraisal process is a concern, that is, a more or less stable preference for certain states of the world (Frijda, 1986). The third variable is the stimulus, which, in the case of product emotion, is not necessarily the product itself. The emotion can also be elicited by an event related to the product, such as a consequence of the product, the behavior of the product in interaction, or an associated object or person, like the manufacturer or the user. The basic model indicates that an emotion is not elicited by the product as such, but by an appraised concern match or mismatch, and that the study of product emotions requires an understanding of the concerns that underlie them. Each of the three variables is discussed in more detail below. The next section (4) provides examples of how these variables combine to elicit product emotions. Appraisal According to appraisal researchers, all emotions are preceded and elicited by an appraisal (Roseman, 1991). Appraisal is an evaluative process that serves to ‘diagnose’ whether a situation confronting an individual has adaptational relevance and, if it does, to identify the nature of that relevance and produce an appropriate emotional response to it (Lazarus, 1991b). One who is confronted with a fire alarm will most likely experience fear with a corresponding tendency to flee because the fire alarm signals a potentially Expression action tendency arousal feeling

Emotions

Appraisal

Latent/active Attitudes goals standards

Concern

Actual/associated Object

Product interaction consequence

FIGURE 15.2 Basic model of product emotions (adapted from Desmet, 2002).

390

Product Experience

harmful situation with particular behavioral consequences. This example illustrates that appraisals are inherently relational (e.g. Scherer, 1984). Rather than exclusively reflecting either the properties of the stimulus (e.g. a fire), the situation (e.g. the office), or the person (e.g. asthmatic condition), appraisal represents an evaluation of the properties of the stimulus and the situation as it relates to the properties of the individual (Smith and Lazarus, 1990). In short, appraisal is an evaluation of the significance of a stimulus for one’s personal well-being. It is this personal significance of a product, rather than the product itself, which causes the emotion. Because appraisal mediates between products and emotions, different individuals who appraise the same product in different ways will experience different emotions. One who is stressed may respond with irritation to the ring tone of his or her mobile phone because he or she appraises it as undesirable, whereas another person may appraise the same event as desirable. Concerns Emotions arise from encounters with events that are appraised as having beneficial or harmful consequences for the individual’s concerns, that is, his or her major goals, motives, well-being, or other sensitivities (Frijda, 1986; Lazarus, 1991a). Concerns are the dispositions that we bring into the emotion process, and stimuli are construed as emotionally relevant only in the context of one’s concerns (Lazarus, 1991a). In order to understand emotional responses to consumer products, one must understand the users’ concerns given the context in which the product is or will be used. The number and variety of human concerns is endless. Types of concerns reported in the research literature are, for example, drives, needs, instincts, motives, goals and values (for a discussion, see Scherer, 2001). Some concerns, such as the concern for safety and the concern for love, are universal, while others are context-dependent, such as the concern for being home before dark or the concern for securing a good seat for your friend at the cinema. Concerns can be latent (sleeping) or active (awake). Concerns are asleep as long as the circumstances pose no threat or possibility to their fulfillment. The concern of physical well-being, for example, will stay asleep until something threatens the physical well-being, like an upcoming headache, or a news item warning for the risk of eating unhealthy food. Sleeping concerns are latent in the sense that they only come to the foreground when the factual circumstances differ from satisfaction conditions. The concern of flexible communication can be awakened when one realizes that one’s mobile phone is missing. Our implicit expectations of how products should function are sleeping concerns that are awakened if the product does not meet that expectation (see Chapter 16). One expects, for example, that a door lock will not unlock spontaneously. A lock that meets this implicit expectation will not rouse an emotion. A lock that does not, however, will awaken the concern of security and elicit an unpleasant emotion for being appraised as insecure. Stimulus According to Frijda (1986) any perceived change, or event, has the potential to elicit an emotion. This can be someone approaching us, a car passing at high speed, the sound of a vase breaking, etc. In the case of products, the stimulus ‘event’ can be: Perceiving the product, using the product, and the consequences of (using) the product. Perceiving the product is the most straightforward stimulus event. Seeing, touching, hearing, and smelling an object can be a strong emotional stimulus. The second type of stimulus event is product usage. Using a product is an event in itself, which can be separated in many sub-events of action and reaction of both user and product. Each of these events can be an emotional stimulus. For example, the TV doesn’t respond to the remote control, the oven starts to produce a scent of freshly baked cookies, the alarm clock sets of, a drawer

Product emotion

391

runs unexpectedly smoothly, and so on. The third type of stimulus event is the consequences of (using the) product. The consequence of wearing a fashionable new suit can be positive remarks of colleagues; the consequence of using a laptop can be that the work is done more efficiently; the consequence of eating too much ice-cream can be a tummy ache. Each of these consequences can act as emotional stimuli. Note that the absence of an expected consequence can also elicit an emotional response. Those who expect a friend for dinner, will be disappointed when the friend does not show up, and those who buy an auto-bronzing lotion, will be dissatisfied when the product does not tan their skin. The model in Figure 15.2 distinguishes between actual and associated stimuli. This is an important distinction because not only events that actually happen can elicit emotions, but also events that are remembered, imagined, anticipated, dreamed, or fictional (Ekman, 1994a). Imagining what would happen if I arrive late for an important meeting is sufficient to arouse an emotion. Or, remembering a joyful event can be sufficient to experience joy all over again. This distinction between actual and associated stimuli also applies to products. Products can represent personal meaning (e.g. souvenirs and nostalgia), or cultural meaning (e.g. symbols and branding). One can, for example, be attracted to a record player because it represents a childhood memory, or feel contempt towards a pair of shoes because the brand is associated with hooligans. The same applies to product usage. One can anticipate usage events (‘I expect this handle to break when I push it too hard’), or fantasize about usage events (‘my computer thinks he knows what I want, but he does not have a clue’). One can also imagine, anticipate, or fantasize about possible consequences of usage. For instance, a person may feel desire towards a new abdominal work-out device because he or she anticipates that with this device the perfect body is within reach.

4. SOURCES OF PRODUCT EMOTION The key factor of an appraisal perspective on the elicitation and differentiation of emotion is the assumption that people constantly evaluate (actual or imagined) stimulus events for their personal significance. This significance is operationally defined by a number of evaluative issues or criteria that constitute the meaning structure in which the evaluation takes place (Scherer, 2001). In this view, the overall outcome of the appraisal process is the combined outcome of these evaluative issues. Each of these issues is addressed by a particular ‘appraisal check’. In the case of products, the appraisal checks relate to issues such as: Does this product help me to attain my ambitions? Can I afford it? Will my neighbors approve? Is it safe? Am I successful in operating it? etc. Many appraisal researchers have developed models that attempt to specify the evaluative issues that initiate specific emotions (see, e.g. Frijda, 1986; Lazarus, 1991a; Roseman, 1991; Scherer, 1988; Smith and Ellsworth, 1987). These distinct evaluative issues are in turn associated with distinct types of concerns. Emotion theorists often distinguish between three concern types: Attitudes; goals; and standards (see e.g. Ortony et al., 1988). This concern distinction has been found effective for understanding product emotion (see, Desmet, 2002). The different types of concerns and corresponding appraisal checks can be combined with different types of stimuli to represent various sources of product emotions. Figure 15.3 visualizes how the three concern types combine with three levels of product stimulus to nine sources of product emotion. These nine sources represent combinations of the variables in the basic model in Figure 15.2, and can be used as a framework for understanding product emotion. The columns represent three types of concerns with associated appraisal checks, and the

392

Product Experience

Attitudes

Goals

Standards

Product

Attracted to the sensuous shape of this product

Desire for owning a mobile phone

Admiring the designer for making an innovative design

Usage

Enjoying the gestures required for making an espresso

Frustrated at not being able to set the timer of a DVD recorder

Being angry with the product for breaking down

Consequence

Being inspired by an art gallery visit

Satisfied by increase in health resulting from using steam pan

Being proud of weight loss because of product

FIGURE 15.3 Nine sources of product emotion. The horizontal axis shows different types of stimuli, and the vertical axis shows different types of concerns.

rows represent three types of stimulus events that can evoke product emotions. Note that although many emotional responses can be explained with these nine sources, it is not claimed that these represent all possible sources of product emotions.

4.1. Product emotion related to affect dispositions The first source of product emotion is represented by an appraisal check of intrinsic pleasantness: How pleasant or unpleasant is this stimulus event? This appraisal is an evaluation of whether a stimulus is pleasurable or painful (or whether a stimulus is likely to result in pleasure or pain), and determines the fundamental pleasure response: Liking feelings, generally encouraging approach, versus disliking feelings, leading to withdrawal or avoidance. The concerns that serve as point of reference in this appraisal are our attitudes, which are the affect dispositions that were discussed in a previous section of this chapter. We have attitudes towards product types (‘I don’t like microwave ovens’) aspects or features of products (‘I like red cars’), towards style (‘I like Italian design’), towards usage (‘I like cars that have a firm drive’), and towards consequences of products (‘I like feeling relaxed after drinking a beer’). The intrinsic pleasantness check applies to all three stimulus levels. In the case of the product level, the stimulus is the (visual, tactile, olfactory, auditory, gustatory) manifestation of a product. Products are objects, and all objects (including their properties and features) are appraised as pleasant or unpleasant. As a result, one is attracted to a sensuous shape of a perfume bottle, feels aversion towards the off-colored leather suitcase, or enjoys the taste of sweet and cold ice-cream. The second stimulus level, that is, using the product, can also involve sensations that are appraised as pleasant or unpleasant. The gestures that are required to operate an espresso machine, the expressive movements of playing the violin, and the forces that are felt when driving a motor cycle are appraised as intrinsically pleasant or unpleasant. In those cases the act of using the product, rather than the product as such, generates sensations that are pleasing or displeasing. The third stimulus level represents those emotions that are elicited by the consequence of (using or owning) the product. These consequences can also be intrinsically pleasant or unpleasant. For example, the consequence of eating too much cake is an unpleasant feeling in the stomach, the consequence of looking at art is a pleasant feeling of inspiration, and the consequence of using a massage device is the pleasant feeling of relaxation.

Product emotion

393

Emotions evoked by an intrinsic pleasantness check have in common that they are elicited independent of the motivational state of the person (i.e. particular goals or motives), and are related only to attitudes. Some researchers refer to these emotions as ‘aesthetic emotions’ because they are elicited by sensations that are pleasing or displeasing to our senses (see Lazarus, 1991b). In the design research literature these experiences are sometimes not considered to be emotions (see e.g. Hekkert, 2006; Chapter 10 this volume), but rather a separate type of product experience. This corresponds with the process-level approach of Norman (2004) that identifies these experiences as simple affect (at the reactive level of information processing). Note however that this source of product emotion also represents emotions that are elicited by associated meanings of the product. Many people will be disgusted by the appearance of a swastika sign; not because the appearance of the sign is displeasing to the senses but because it represents values, memories, or ideas that are appraised as intrinsically unpleasant. In the same way, one may experience attraction towards a particular object (brand, company, color, etc.) because it symbolizes or represents something that is appraised as pleasant. These experiences involve higher-level cognitive processes that Norman associates with reflective emotions rather than with simple affect.

4.2. Product emotion related to goals The second source of product emotion is represented by an appraisal check of motive consistency: To what degree does this stimulus event help me to attain my goals? Goals are the things one wants to get done and the things one wants to see happen. The goals that people try to satisfy are often assumed to be structured in a hierarchy ranging between abstract goals or aspirations, like the goal to have a successful life, and goals as concrete and immediate as the goal to catch a train. Many goals are directly and indirectly activated in the human-product relationship. For example, we buy, own, and use products because we believe they can help us to achieve things (a digital agenda to make us more organized), or they fulfill a need (a bicycle fulfills the need for transportation). The more directly products facilitate goal attainment, and the closer they propel the person toward reaching a goal, the higher the appraised motive consistency (Ellsworth and Scherer, 2003). In other cases, products can be obstructive for goal attainment, by putting goal satisfaction out of reach, delaying its attainment, or requiring additional effort (see Srull and Wyer, 1986). In the case of the first stimulus level, i.e. the product level, the goal at stake involves the product as such, which can be, for example, the goal to own a particular product. Other examples of goals that involve products are the goal to share, personalize, restore, discard, or repair a product. If a person has the goal to own a particular pair of shoes, this person will be disappointed to hear that the model is out of stock. One who is interested in owning an original model T-Ford will be happily surprised when accidentally bumping into one in some old barn. The motive consistency appraisal check also applies to using the product. When using products, people are involved in goal-directed behavior sequences. If this sequence is blocked in the interaction, people will typically experience frustration. One can be frustrated by product packages that are impossible to open (e.g. pre-packed slices of cheese), satisfied with products that are easy to operate (e.g. the selfexplaining Apple i-Pod interface), and pleasantly surprised by the accurate response of a stereo set’s volume to the remote control. The third stimulus level, the consequence of (using or owning) the product, may be appraised as motive consistent (e.g. being satisfied by an alarm clock because one is not late for work), or motive inconsistent (e.g. being dissatisfied with the new mattress because it increases instead of decreases the backache). Emotions are not only experienced in response to actual goal achievement, but also in response to anticipated goal achievement. One can be attracted to a particular computer

394

Product Experience

because one anticipates that this computer will satisfy the goal of productivity. An additional appraisal check involved in emotions elicited by anticipated events is the probability check (Scherer, 1984). This check assesses the likelihood of possible effects. This is of particular importance in cases in which both the probability of the event occurring or its consequences are in doubt, as in the case of emotions like hope and fear.

4.3. Product emotion related to standards The third source of product emotion identified in Figure 15.3 is represented by an appraisal check of legitimacy: How praiseworthy is the stimulus event? The concerns involved in this appraisal check are our standards. Standards are our beliefs, social norms or conventions of how we think things should be. Whereas goals refer to the state of affairs we want to obtain, standards are the states of affairs we believe ought to be (Frijda, 1986; Ortony et al., 1988; Scherer, 1984). For example, many of us believe that we should respect our parents, or wear clean clothes at work. Most standards are socially learned and represent the beliefs in terms of which moral and other kinds of judgmental evaluations are made (Ortony et al., 1988). Whereas goals are relevant for our personal well-being, standards are relevant for the preservation of our social structures (and thus indirectly also for our personal well-being). Social organization depends on shared rules (norms) about what behavior is acceptable and what is unacceptable. Such norms are sustained by appropriate emotional reactions of group members to behavior that violates norms, as well as to conforming behavior. We approve of things that comply with standards and disapprove of things that conflict with them. We not only have standards regarding human (inter)action, but also regarding products. In the case of the first stimulus level, the legitimacy of the product itself is at stake. We have standards of how products should be, and how they should be designed and produced. One can admire a chair for being more eco-friendly than a conventional chair. Or one can be irritated by a new car model because automotive companies should not introduce new versions too often. In those cases the products are appraised as the outcome of the action of some person or institute, and that particular action is appraised as either legitimate or as improper. The second stimulus level involves standards of performance, that is, standards of how products should behave when they are used. For instance, one shouldn’t hear a rattling sound when driving a brand new car, a computer should not crash without a warning, and the brakes of a bicycle should be reliable. One can experience emotions such as anger or disappointment when a product does not meet the standards of performance. The third level is related to the consequence of owning or using the product. Discrepancy with standards of ownership can lead to emotions such as contempt, and exceeding the standards may produce admiration. One can admire one’s perfectly mown lawn, a consequence of using a high quality lawn mower, or one can feel contempt towards some person’s run down car. One can also appraise the legitimacy of one’s own behavior with reference to internal standards, one’s internalized moral code or self-concept. These standards represent the self-ideal and are central for the experience of the so called self-reflexive emotions, such as pride, guilt, and shame. One can be ashamed of owning sex toys or proud of owning a signed baseball.

4.4. Mixed emotions The distinction between various sources of product emotion enables us to explain why products sometimes elicit mixed emotions. First, mixed emotions may be elicited within a particular source. One can experience mixed emotions in response to events that are

Product emotion

395

consistent with one goal and obstructing another (Weigert, 1991). Buying a digital agenda may correspond with the goal to be more time efficient, but at the same time conflict with the goal to be independent of digital devices. The same applies to all other sources of emotion. One may appraise the color of a product as pleasant, and, at the same time, the tactile quality as unpleasant. One may appraise the innovativeness of a product as legitimate, and, at the same time, the high price as illegitimate. Secondly, mixed emotions may be generated by emotional responses elicited by different types of sources. For example, the intrinsic pleasantness check is independent of the motivational state of the person, whereas motivational state is the decisive element in the goal conduciveness check. We all know from experience that an inherently pleasant product can block goal achievement, since something pleasant (like chocolate cake) can obstruct us in reaching a goal (trying to lose weight). One can experience shame from using a rollator (standard of being independent), and at the same time, be satisfied with the increased mobility it provides (goal of being mobile). Note that our emotional responses are not mutually independent because our attitudes, goals, and standards are related to each other. One can have a favourable attitude towards the color red (appraise a red product as intrinsically pleasant), can have the goal to own a red car (appraise a red car as motive consistent), and may have the standard that cars of a particular brand should be available in the color red (appraise a brand that does not offer red cars as illegitimate). Our concerns are the dispositions that we bring into the emotion process. Although dispositional, these concerns do evolve over time, and differ in different contexts. Children have different goals, attitudes, and standards than teenagers and adults. Concerns are not independent of the context of usage. For example, people have other concerns with respect to computers in the context of work than in a family context. In addition, there are many factors that have a constant influence on our concerns, such as marketing, technological innovation, peer group behavior, and fashion. The standard of performance for a laptop computer today differs from the one we had two years ago. Attitudes towards floral prints on clothes change with the season, the attitude towards smoking cigarettes is influenced by anti-smoking campaigns, etc. The product emotion system is not static but dynamic and interactive. Similar to the pleasure framework of Jordan (2000), the main value of the framework is that it can be used as a tool that helps in taking a structured approach to design for emotion. It represents some of the important variables in the process of product emotion, and can be used as a means of structuring thoughts and discussion as regard to emotion, or as a means of formulating relevant questions about the user or context of usage. One can explore the variables with the aim to generate positive outcomes on as many appraisal checks as possible. Alternatively, one can also explore these variables in order to deliberately design products that elicit mixed emotions by, for example, confronting its user with his or her sometimes contradictory concerns. The palette of emotions that can be experienced in response to products displays many different varieties: Some mild, some intense, some cursory, and some long lasting. It includes immediate emotions that relate directly to the interaction or context of interaction, like anger, confusion, contentment, enjoyment, and aversion, perceptible at the surface of our emotional experience, expression, and behavior. And the palette also includes emotions that are more indirect; those that operate at a deeper level, invisible and less explosive, but no less oppressive and influential. These are emotions like trust, resignation, compassion, empathy, melancholy, hope, and consolation. The framework of product emotion introduced in this chapter does not explain all these possible emotional responses, and future research may reveal additional types of concerns, types of stimuli, or dimensions that enable us to explain a wider variety of emotional responses. Even so,

396

Product Experience

the basic framework of product emotion does illustrate that we can identify relationships between our highly personal and subjective emotional responses, and some objective and universal principles in the processes that elicit them.

REFERENCES Aaker, D. A., Stayman, D. M. and Hagerty, M. R. (1986). Warmth in advertising: Measurement, impact, and sequence effects. Journal of Consumer Research, 12, 365–381. Arnold, M. B. (1960). Emotion and personality, volume 1: Psychological aspects. New York: Colombia University Press. Bradley, M. M. and Lang, P. J. (1994). Measuring emotion – the self-assessment mannequin and the semantic differential. Journal of Behavior Therapy and Experimental Psychiatry, 25(1), 49–59. Clore, G. L. and Ortony, A. (1988). The semantics of the affective lexicon. In: V. Hamilton, G. H. Bower and N. H. Frijda (Eds.) Cognitive perspectives on emotion and motivation, pp. 367–397. Dordrecht: Kluwer Academic Publishers. Darwin, C. (1872). The expression of the emotions in man and animals. London: Murray. Davidson, R. J. (1994). On emotion, mood, and related affective constructs. In: P. Ekman and R. J. Davidson (Eds.) The nature of emotion, fundamental questions, pp. 51–56. Oxford: Oxford University Press. Demir, E., Desmet, P. M. A. and Hekkert, P. (2006). Experiential concepts in design research; a (not too) critical review. In: M. A. Karlsson, P. M. A. Desmet and J. van Erp (Eds.) Proceedings of the 5th conference on design and emotion, September 27–29. Gothenburg, Sweden: Chalmers University of Technology. Desmet, P. M. A. (2002). Designing emotions. Unpublished doctoral thesis, Delft University of Technology. Ekman, P. (1994a). Moods, emotions, and traits. In: P. Ekman and R. J. Davidson (Eds.) The nature of emotion, fundamental questions, pp. 56–58. Oxford: Oxford University Press. Ekman, P. (1994b). Strong evidence for universals in facial expressions – a reply to Russell’s mistaken critique. Psychological Bulletin, 115(2), 268–287. Ekman, P. and Friesen, W. V. (1978). Facial Action Coding System. Palo Alto: Consulting Psychologists Press. Ellsworth, P. C. and Scherer, K. R. (2003). Appraisal process in emotion. In: R. J. Davidson, K. R. Scherer and H. H. Goldsmith (Eds.) Handbook of affective sciences, pp. 575–595. Oxford: Oxford University Press. Etcoff, N. (1999). Survival of the prettiest: The science of beauty. New York: Doubleday. Frijda, N. H. (1986). The emotions. Cambridge: Cambridge University Press. Frijda, N. H. (1993). Appraisal and beyond. Cognition and Emotion, 7, 225–231. Frijda, N. H. (1994a). Emotions are functional, most of the time. In: P. Ekman and R. J. Davidson (Eds.) The nature of emotion, fundamental questions, pp. 112–122. Oxford: Oxford University Press. Frijda, N. H. (1994b). Emotions require cognitions, even simple ones. In: P. Ekman and R. J. Davidson (Eds.) The nature of emotion, fundamental questions, pp. 197–202. Oxford: Oxford University Press. Hekkert, P. (2006). Design aesthetics: Principles of pleasure in product design. Psychology Science, 48, 157–172. Jordan, P. W. (1999). Pleasure with products: human factors for body, mind and soul. In: W. S. Green and P. W. Jordan (Eds.) Human factors in product design: Current practice and future trends, pp. 206–217. London: Taylor and Francis. Jordan, P. W. (2000). Designing pleasurable products. London: Taylor and Francis. Laurans, G. and Desmet, P. M. A. (2006). Using self-confrontation to study user experience. In: M. A. Karlsson, P. M. A. Desmet and J. van Erp (Eds.) Proceedings of the 5th conference on design and emotion, September 27–29. Gothenburg, Sweden: Chalmers University of Technology. Lazarus, R. S. (1991a). Emotion and adaptation. Oxford: Oxford University Press. Lazarus, R. S. (1991b). Progress on a cognitive motivational relational theory of emotion. American Psychologist, 46, 819–834. Norman, D. A. (2004). Emotional design. New York: Basic Books. Ortony, A., Clore, G. L. and Collins, A. (1988). The cognitive structure of emotions. Cambridge: Cambridge University Press. Ortony, A., Norman, D. A. and Revelle, W. (2005). The role of affect and proto-affect in effective functioning. In: J. M. Fellous and M. A. Arbib (Eds.) Who needs emotions: The brain meets the machine. Oxford: Oxford University Press. Plutchik, R. (1980). Emotion: A psychobioevolutionary synthesis. New York: Harper and Row. Plutchik, R. (1984). Emotions: A general psychoevolutionary theory. In: K. R. Scherer and P. Ekman (Eds.) Approaches to emotion, pp. 197–219. Hillsdale, NJ: Lawrence Erlbaum.

Product emotion

397

Roseman, I. J. (1991). Appraisal determinants of discrete emotions. Cognition and Emotion, 5, 161–200. Rozin, P. and Fallon, A. (1987). A perspective on disgust. Psychological Review, 94, 23–41. Russell, J. A. (1980). A circumplex model of affect. Journal of Personality and Social Psychology, 39, 1161–1178. Russell, J. A. (1993). Force-choice response format in the study of facial expression. Motivation and Emotion, 17, 41–51. Russell, J. A. (2003). Core affect and the psychological construction of emotion. Psychological Review, 110, 145–172. Scherer, K. R. (1984). On the nature and function of emotion: a component process approach. In: K. R. Scherer and P. Ekman (Eds.) Approaches to emotion, pp. 293–318. Hillsdale, NJ: Lawrence Erlbaum. Scherer, K. R. (1988). Criteria for emotion-antecedent appraisal: a review. In: V. Hamilton, G. H. Bower and N. H. Frijda (Eds.) Cognitive perspectives on emotion and motivation, pp. 89–126. Dordrecht: Nijhoff. Scherer, K. R. (2001). Appraisal considered as a process of multi-level sequential checking. In: K. R. Scherer, A. Schorr and T. Johnstone (Eds.) Appraisal process in emotion: theory, methods, research, pp. 92–120. New York: Oxford University Press. Schubert, E. (1996). Measuring temporal emotional response to music using the two dimensional emotion space. Proceedings of the 4th International Conference for Music Perception and Cognition, pp. 263– 268. Montreal, Canada. Schwarz, N. and Clore, G. L. (1983). Mood, misattribution, and judgments of well-being: informative and directive functions of affective states. Journal of Personality and Social Psychology, 45, 513–523. Smith, C. A. and Ellsworth, P. C. (1987). Patterns of appraisal and emotion related to taking an exam. Journal of Personality and Social Psychology, 52, 475–488. Smith, C. A. and Kirby, L. D. (2001). Toward delivering the promise of appraisal theory. In: K. R. Scherer, A. Schorr and T. Johnson (Eds.) Appraisal processes in emotion, pp. 121–140. Oxford: Oxford University Press. Smith, C. A. and Lazarus, R. S. (1990). Emotion and adaptation. In: L. A. Pervin (Ed.) Handbook of personality: Theory and research, pp. 609–637. New York: Guilford. Srull, T. S. and Wyer, R. S., Jr. (1986). The role of chronic and temporary goals in social information processing. In: R. M. Sorrentino and E. T. Higgins (Eds.) Handbook of motivation and cognition, pp. 503–549. New York: Wiley. Tiger, L. (1992). The pursuit of pleasure. Boston: Little Brown. Weigert, A. J. (1991). Mixed emotions: Certain steps toward understanding ambivalence. Albany: State University of New York Press. Wensveen, S. A. G. (2005). A tangibility approach to affective interaction. Unpublished doctoral thesis, Delft University of Technology. Wundt, W. (1905). Fundamentals of psychology (7th Ed.) Leipzig: Engelman. Zajonc, R. B. (2000). Feeling and thinking: closing the debate over the independence of affect. In: J. P. Forgas (Ed.) Feeling and thinking: The role of affect in social cognition, pp. 31–58. New York: Cambridge University Press.

14

SEMANTICS: MEANINGS AND CONTEXTS OF ARTIFACTS1 KLAUS KRIPPENDORFF University of Pennsylvania, Philadelphia, PA

REINHART BUTTER Ohio State University, Columbus, OH

1. PRELIMINARIES English dictionaries trace the origin of the word ‘experience’ to knowledge of or skill in making experiments. Its etymology suggests an important conceptual truth: Experiences are not merely personal and subjective, but crucially related to interacting with something of interest, an artifact, an activity, or a situation involving other people. What we will explore here must therefore overcome the objective/subjective Cartesian dichotomy and be concerned instead with how humans experience the world by acting on it and creating it. The prefix ‘ex-’ also suggests that ex-periences require ex-ternalization, ex-pression, or ex-planations. We cannot know what others experience unless they let us know by whatever means are at their disposal. We cannot discuss or theorize experiences without using words. Thus, while the sharing of experiences is impossible, when we talk with each other of what we experience we do so con-sensually, that is, in reference to something jointly attended to – naturally including the discourse by which we coordinate our understanding and actions. It makes sense, therefore, to say that we shared a taxi ride or listened jointly to a concert, but not that we shared the experiences of these events. The latter may be different for each of us. Similarly, we can hear each other speaking, we may even talk about what we mean to say, but we cannot share meanings. Moreover, while 1 This chapter reflects a long collaboration between Klaus Krippendorff, who wrote its text, and Reinhart Butter, who provided the photographic illustrations for this text.

Product Experience Copyright © 2008 Elsevier Ltd.

353

354

Product Experience

our experiences are not only shaped and conceptualized by the categories provided to us by our use of language, we cannot help but talking about them in the expectation of being understood, which implicates the interests of sympathetic listeners or the community in which talking of certain experiences is valued. Inasmuch as the use of language is essentially social, what we know of each other’s experiences is, hence, fundamentally social as well, not entirely subjective. As Madison (1988, p. 165) suggests, Language is the way in which, as humans, we experience what we call reality … Expressed experience is experience that has settled down … Experience is not really meaningful until it has found a home in language, and without lived experiences to inhabit it, language is an empty, lifeless shell.

Our approach to design is human-centered, which contrasts with the technology-centered design of engineering and functionalism. Human-centeredness acknowledges the role of humans in actively constructing artifacts – conceptually, linguistically, and materially – being concerned with them, handling them, and putting them to work. It acknowledges the diversity of human conceptions that motivate how things are acquired, exchanged, rendered meaningful, and used. Consequently, when we talk of meanings, we must be clear about whose meanings we are talking of, and allow for the possibility that we may see things differently. A technology-centered approach, by contrast, seeks objective, generalizable, and non-experiential accounts of things. It stresses technical functionality and efficiency. We suggest four conceptual pillars that support our human-centered approach: Second-order understanding; meanings; networks of stakeholders; and interfaces.

1.1. Second-order understanding Technology-centered designers can work within a language that addresses their concerns without reference to the concerns of outsiders. Commitments to objectivity; belief in universalist theories of functionalism, economy, and aesthetics; the conviction that particular forms are responsible for particular uses, experiences, and feelings; and the privileging of one’s own views over those of less qualified people, makes design relatively easy. It implicates an authority which assures that designs are used as intended, on the one hand, and creates the distinction between knowledgeable designers and merely responsive users in need of instruction and guidance, on the other hand. In contrast, human-centered designers are committed to designing artifacts for use by others who may experience the same designs quite differently. It follows that human-centered designers cannot universalize their own conceptions of what they see and do. They have to understand how those that come in touch with their design understand it in the context of their own world. Understanding others’ understanding requires listening to what they say they experience, and acknowledging their understanding as legitimate, not inferior or mistaken, even when it deviates significantly from one’s own. Understanding others’ understanding is an understanding of understanding and this recursion is of a qualitatively different kind. We have called it second-order understanding and note that such an understanding is absent in technology-centered design. Machines do not understand, humans do, and the design of technology without user involvement can be accomplished by simple firstorder understanding.

1.2. Meanings It is a truism that we surround ourselves with objects that we are comfortable with and experience as meaningful. This is axiomatic for designers, as well as for those who have a stake in their designs. To design artifacts for use by others is to design them to be or

Semantics: Meanings and contexts of artifacts

355

to have the chance to become meaningful to these others – not merely in their designers’ terms, but according to these others’ own and often diverse conceptions. For these reasons, we prefer the term ‘meaning’ to ‘experience’ as it connects the design of artifacts – not to a psychological construct – but to how others see, feel, describe, and interact with in ways different to how we relate to them. We take semantics as the study of meanings in the broadest sense, not to be confused with how the word ‘semantics’ has been appropriated in the rigid structure of semiotics. For the above reasons, in conceptualizing ‘meaning’, we reject its ontologization, treating meaning as if it were an entity that could be attached to objects or contained in containers, for example, when saying: X has the meaning Y, or X contains the meaning Y,

both of which imply that meaning is the same for everyone. Saying: We experience the meaning Y

moreover locates meaning outside the human species, as the cause of experience. The latter conception is also at home in Latour’s (1996, 2005) actor-network-theory, which ascribes agency to artifacts, suggesting something like: Meaning Y is inscribed in X and acts on their users.

We also want to avoid the representationalism of semiotic discourse, for example in such constructions as: X represents Y,

or X is a symbol of (or sign for) Y,

X having to be on a logical level other than Y. In design, semiotic conceptions encourage artifacts that signify something unrelated to its use, for example, products that look more valuable than they are or take an alien form, like a telephone in the shape of Mickey Mouse or a radio in the shape of an owl (Figure 14.1).

FIGURE 14.1

356

Product Experience

We find the cognitivist approach to meanings as internal representations of an external world equally troubling. This conception is rooted in the Cartesian dualism, violates the biology of cognition (Varela, Thompson and Rosch, 1991), and ignores the social contingencies of experiences mentioned above. On the other extreme, we also consider the phenomenological conception of meanings as ‘inter-subjective’ phenomena unfortunate. We affirm that many meanings arise in social interactions with or about artifacts, using the medium of language, but this does not suggest that meanings reside between people. Finally, although we created the term ‘Product Semantics’ (Krippendorff and Butter, 1984), we do not wish to restrict our concern to industrial products. Talking of products implicates the perspective of their producers or manufacturers, and this terminology ties design to the industrial era. In an information society, designers address many more and more exciting artifacts, mental, computational, organizational, and cultural. To design them in order to mean something in their users’ world is the challenge. Key to our conception of meaning is the recognition that humans create their own worlds and distinguish among their artifacts not in physical terms, but according to what they mean to them, including how they enter the communications about them. Our concept of meaning involves a second-order understanding of how others come to understand and interact with our designs. Thus, for us, meanings cannot be separated from how people interact with the technologies that their culture creates and renders meaningful, with each other, and with how we – for example as designers or researchers – describe, conceptualize, and enact our conceptions of these meanings. In The semantic turn (Krippendorff, 2006), we develop this approach to meaning more fully. Among others, we develop four theories of meaning for the design of artifacts, each culminating in particular design objectives and methods. These theories are not seen as alternatives, but as concerning different aspects that human-centered designers cannot avoid. 1. A theory of meaning for artifacts in use accounts for how individual users come to understand their artifacts and interact with them in their own terms, and for their own reasons. It follows Ludwig Wittgenstein’s (1953) suggestion to locate the meaning of artifacts (for Wittgenstein: words) in their use, not as referring to extraneous things. It embraces James J. Gibson’s (1979) ecological theory of perception, but goes beyond it by focusing on human interfaces with artifacts, not only on what they essentially support. The theory provides numerous concepts – categories, visual metaphors, attractiveness, user conceptual models, constraints, affordances, metonyms, semantic layers, scenarios, and motivation. In The semantic turn, we develop ten kinds of clues, which we call ‘informatives’. They are means to indicate how to proceed. This theory conceives of meanings as enabling individual users of artifacts to get involved, revealing what can be done with them and how, and ultimately rendering them reliable (see Figure 14.2–14.3).

FIGURE 14.2

FIGURE 14.3

Semantics: Meanings and contexts of artifacts

357

2. A theory of meaning for artifacts in language recognizes that artifacts also occur in conversations among people, not only in user interactions. For example, narratives establish the ownership of artifacts, cause the attribution of personalities – from being stubborn to (user) friendly – determine technical performance characteristics, and decide their fate, successes or failures. This theory is essentially interpersonal or social, and the concepts it embraces concern what comes to be shared within a community of language users – categorization; attributions, including aesthetics; individual, group, and institutional identities; using linguistic metaphors and narratives 3. A theory of meaning in the genesis or life cycle of artifacts recognizes that any artifact undergoes numerous transformations – from its conception to its retirement – and in that process, must enroll stakeholders to form networks through which it can travel with ease and direction. We will write about stakeholder networks in the next section, 1.3. Here, we merely state the obvious that for artifacts to survive the process of their genesis, each of its transformations must be meaningful to all those capable of bringing the process to fruition. Meanings are conceived of as enrolling a succession of stakeholders into a design project and informing them about the possibilities that a design affords them – again in their own terms. The theory addresses the critical sizes of the communities needed to support a technology, explains the condition for technologies to spread through a population, and offers whole life cycle accounts as a way to evaluate how a design succeeds. It is far more comprehensive than the two theories of meaning mentioned before 4. A theory of meaning for ecologies of artifacts is concerned with how whole species of artifacts interact with one another, affect their population sizes, compete or cooperate, and form technological complexes. Unlike in ecological theories of living organisms, artifacts are not agents, however, and an ecology of artifacts is fuelled by how populations of stakeholders experience and interpret artifacts relative to each other, particularly putting them to use selectively. The theory describes the ecological properties of the meanings that communities of stakeholders bring to populations of artifacts. For example, any one species of (necessarily identical) artifacts is stable if its members are reproduced at the rate of their retirement. Artifacts of different species with synonymous meanings (interfaces) compete for the same ecological niches, while artifacts that have complementary meanings can work together, cooperate, and may develop larger technological cooperatives. Ecological meanings, thus conceived, enable one to predict parasitism – when one species of artifacts thrives on the back of another – cooperative dependencies – when one species enhances another without being able to exist on its own, etc. For designers, the lesson of this theory is that for any species of artifacts to succeed in the long run, one needs to render them mutually supportive, cooperative with other species, and more efficient than competing artifacts. Ordinary users may not be aware of the ecological properties of their artifacts, which are brought about by inserting them in contexts of their choice, but to designers, these properties are of central importance for a design to have a chance of surviving in the context of other species of artifacts. This chapter draws heavily on the first three theories above.

1.3. Networks of stakeholders We also discourage talking about THE user. It is a deceptive myth. Users are not only diverse in their interests, knowledgeable about the artifacts in their use, experts in their lives, but they also rarely are the only ones that count. In reality, designers mostly deal with clients who represent a business or corporation, including all of its decision makers. There are financiers who are concerned for their investment in a design. There are engineers who will have to solve its technical problems. There are marketing researchers who

358

Product Experience

have a say in whether a design is sellable. There are merchants who must see some benefit in bringing a design to the market. There are governmental agencies that enact standards that protect citizens from being exploited. There are buyers who may be motivated quite differently than those who end up using an artifact or consume a product. There are critics who put a spin on a design that can influence how an artifact is perceived. There are repairpersons, recyclers, ecological activists, and many more who variously experience a design and collectively affect its fate. These groups of people are knowledgeable and capable of asserting their stakes in a design. They do not act in unison, but form complex networks through which a design must proceed in order to be realized. We call this a stakeholder network. By comparison, the concept of ‘THE user,’ commonly invoked by designers who assume the role of THE user’s advocate, trivializes the network of stakeholders involved. THE user is nothing but a rhetorically convenient illusion that designers offer their clients in justifications of their design. This also includes references to so-called personas that designers conceptualize as endowed with particular social attributes. THE user, as well as personas, are designers’ constructions. They conform to designers’ expectations, have no voice of their own, and cannot object or contribute to a design in unexpected ways. Human-centered designers must acknowledge the critical role of stakeholders – supporters and opponents – welcome their active roles in bringing a design to fruition, and see themselves not as masterminding the process, but as active participants in such networks as well.

1.4. Interfaces We experience artifacts by interacting with them. Epistemologically, what artifacts essentially or materially are is not accessible to us. It is also of little interest to designers who, by designing for others, need to know what their designs mean to their stakeholders or the meanings that could emerge in use, language, genesis, or ecological interactions. Taking this premise seriously involves a radical shift from a concern for tangible artifacts, industrial products, for example, to a concern for how people interact with them, from what things objectively are to processes through which they are created and experienced, and from ontology to ontogenesis. Computer interfaces are most familiar to us. As computer users, we have no clue of how the masses of zeros and ones change within a computer. If visualized, we would not be able to comprehend that process. Moreover, changes within a computer take place at speeds at which we cannot possibly read. Despite these basic facts, computer users can handle their computers quite well. They experience their computers by interacting with them in human terms, at human speeds, according to what they want a computer to do, which includes that the interface unfolds meaningfully. Computer architecture is one thing, meaningful interfaces are quite another. We do not wish to limit the concept of an interface to computers. Handling a telephone is as much an interface as is driving a car or skiing downhill. To design artifacts for human use, one needs to go beyond their forms and decompose them into sequences of human actions and responses from the artifact, into sensory-motor coordinations that can be monitored, understood, and directed to desirable experiences. The form of artifacts is secondary to their temporally unfolding interfaces. To design artifacts as simple as eating utensils one needs to know how they support the cultural practices of their users. Eating with forks, knives, and spoons differs from eating with chopsticks, and with one’s hands, but they all can be conceived of as meaningful interfaces. Figures 14.4– 14.9 show a range of interfaces.

359

Semantics: Meanings and contexts of artifacts

FIGURE 14.4

FIGURE 14.6

FIGURE 14.8

FIGURE 14.5

FIGURE 14.7

FIGURE 14.9

So conceived, interfaces require dual or interactive descriptions, in terms of the conceptions that humans can and do successively enact and monitor, and how artifacts in turn support or deny what they mean to their users. Our conception of an interface brings us close to the work of James J. Gibson.

360

Product Experience

Gibson (1979) made the profound suggestion that we perceive not things, but what they afford us to do. In Gibson’s human-centered ecology of perception, it would be a categorical mistake to describe the physicality of things separate from how we perceive and act on them. He suggests that we cannot perceive what something IS, but what we can do with it or how it can affect us. Considering artifacts as bundles of affordances means describing them – not in terms of kilograms, but in terms of whether they are easy for us to lift or move; not in terms of heights, but in terms of whether they are reachable for us; not in terms of quantitative measurements, say, of the diameter of handles, but in terms of their ‘gripability’; not in terms of efficiency (measured in gain over effort), but in terms of whether they aid our sense of success; not in terms of whether something IS beautiful, but in terms of the fascination and excitement they trigger in us; not in terms of user-friendliness as an objective property, but in terms of how comfortable we are in handling them without fear of failures. The suffix ‘-able’ always refers to what actors can do, not to physical properties. Gibson was a psychologist with little interest in cross-cultural comparisons, and he studied enduring kinds of sensory motor coordinations, like walking on flat surfaces, handling small tools, including landing airplanes. This led him to characterize affordances as what we perceive unfailingly and direct. We must note that the sensory-motor coordinations he studied have very long histories of humans adapting their interactions to the relatively stable nature of our terrestrial environment. When we deal with technology, affordances cannot be presumed given, and perceiving them is learned. Following Heidegger’s notions (Dreyfus, 1992; Dourish, 2001), in The Semantic Turn (Krippendorff, 2006, p. 89 ff.) we differentiate three qualities of experiences with artifacts. Since they are typically experienced sequentially, we describe them here as stages. Accordingly, human interfaces with technology are always in one of three stages: 1. Recognition, more accurately, re-cognition or cognizing something again, refers to the stage in which we categorize artifacts according to what they could afford us to do or prevent us from experiencing. Recognizing what something is leads us to approaching, ignoring, or avoiding it. Without a clue to how an artifact could help or harm us, it is not likely to come into use. At the recognition stage, there is little feedback. Mistaken identities may surface only after our expectations fail. Figure 14.10 shows an artifact that may not be too obviously recognizable as a personal security device.

FIGURE 14.10

Semantics: Meanings and contexts of artifacts

361

2. Exploration follows recognition and the experience of failures. It describes the stage during which we search for ways to handle an artifact. Whereas recognition presupposes familiar forms, exploration involves expectations about the sequence of interactions that can bring about desirable contexts or results. Figure 14.11 shows the handling of the above mentioned security device. In the context of a hand, the device makes more sense. Recognition and exploration are two stages that we want to be transitional, not to be stuck in, in order to come to what really matters.

FIGURE 14.11

3. Reliance is the stage in which we have mastered the interface with an artifact and proceed naturally, seamlessly, and flawlessly. At this stage, an artifact recedes into the background of the interface, is taken for granted, and no longer noticed. Instead of attending to how it is handled, we focus on what we wish to accomplish with it. Alternative, but less differentiated, concepts of reliance are user-friendliness, usability, the naturalness of interfaces, and Gibson’s notion of directness. These three stages define a dynamic that usually starts with recognition, identifying an artifact of interest by its category. Once it is identified as such, one needs to position it in relation to one’s body, in order to engage in explorations of how to handle it and monitor the consequences of one’s actions. The aim of exploration is to reach reliance where one’s interface with the artifact flows without uncertainties and doubts. Reliance can be disrupted, however, which brings one back to the need for explorations of alternative ways. For example, once the correlation between the moving and clicking of a computer mouse and the pointing, selecting, grabbing, and dragging of icons on the screen is mastered, that mouse can be relied upon but is then no longer seen. When something interferes with that correlation, for example, when the mouse has reached the edge of the mouse pad, and the interface is disrupted, then one is thrown back from reliance to exploration – until the disruption disappears. Undoubtedly, what a computer mouse affords its user to do is learned. Its affordances may well be experienced directly – in Gibson’s sense – but only after having achieved reliance. One could say that the aim of all human-centered design is reliance, the stage where technology disappears from our attention, where we do not need to reflect on what something means to us, and where we can address what actually matters to us. Reliance is also the condition needed for intrinsic motivation to arise (Krippendorff, 2004b), a motivation

362

Product Experience

to engage interfaces for their own sake, skiing, for example, playing music, or losing oneself in a game. However, a viable design needs to succeed through all three stages easily, and with a minimum of disruptions. What these stages have in common is their guidance by the meanings that we bring to a design and are willing to enact.

2. ARTIFACTS AND THEIR VARIOUS CONTEXTS With these preliminaries, which are described more fully elsewhere (Krippendorff, 2006), we can now address the often-ignored, but for designers central, relationship between artifacts and their various contexts of use. The word ‘context,’ is of Latin origin and refers to the weaving together of words, their connections, and coherencies. Today, the use of the word ‘context’ is no longer limited to text. It denotes the surrounding conditions of something that sheds light on its meaning. Regarding texts, most words are ambiguous by themselves – note how many meanings a dictionary typically lists for a single word. In the context of a larger discourse, however, word meanings are usually singular and clear. Similarly, by themselves, artifacts may not mean much unless they are placed in a particular environment in which they play recognizable roles. It is important to note that relations generally, and the relations between artifacts and the contexts in which they may or do occur in particular, do not exist in nature. Artifacts are made, not found, and the distinction between them and their contexts is an intentional act, and so are considerations of how they are related. Attributing meaning to artifacts is a way of rendering the relationships between artifacts and their contexts sensible and coherent. While the natural sciences have no place for contexts, understandably, there are good reasons why the stakeholders in a technology, including of course the designers, must perceive its relations to its context of use differently. However, all ways to enter them into considerations of meanings are motivated by the apparent need to create individually satisfactory and socially acceptable explanations. We capture the forgoing with the help of Figure 14.12, which the following text will then elaborate. Meaning of the artifact

Perceived or constructed by someone

Enacted

Artifact

(2.1) Observing (2.2) Interfacing (2.3) Anticipating

Context

FIGURE 14.12 An artifact in its context of use.

When inquiring into meanings, we claim that there are fundamental methodological differences between merely (2.1) observing and describing how others, users or stakeholders, put their artifacts to various uses; (2.2) interfacing with them according to the

Semantics: Meanings and contexts of artifacts

363

meanings we bring into these interfaces and thus participating in their contexts; and (2.3) anticipating contexts of use from the narratives in which artifacts make sense to us. These distinctions chime with those developed by Poole and Folger (1988), who, concerned with validating social interaction data, have distinguished between three modes of researcher involvement. When observing, researchers are limited to describing their own observations of what others experience. Such researchers are then concerned with the experienced. When participating in a social process, researchers can be concerned with experiencing interactions with and in the presence of others. And when receiving verbal accounts of what particular individuals experience in such situations, researchers are informed about individual experiences. The distinction between the three kinds of data – about the experienced, the experiencing, and the experiences – loosely correlates with our distinction between observing the meaningfulness of artifacts, interfacing with artifacts based on their meanings, and anticipating contexts of use from narratives we accept about particular artifacts. These three ways of inquiring into meanings are shown in Figure 14.12 as accounting for the relationship between artifacts and their contexts, and will be addressed in the following three sections.

2.1. Observing the meaningfulness of artifacts in context of their use by others A conceptual prerequisite for speaking of meanings is that artifacts must be able to occur in more than one context, particularly in contexts other than the one presently at hand. Experientially, meanings require that something could be used otherwise, are variously interpretable, and different for different people, of different cultures, in different situations, or at different times. Without variability in the contexts of artifacts, meanings could not explain anything. The other extreme of the continuum in which meanings make sense is due to the requirement that the variability of contexts in which artifacts can occur must exhibit some constraints. If artifacts could occur everywhere, at any time, and for everyone alike, there would be nothing remarkable about the artifact–context relationship. Meaningfulness presupposes choices. In the position of an observer, for example, as a detached scientific observer, bystander, spectator, or tourist in an unfamiliar culture, one is of course quite free to interpret what one sees, free to create any meaning one pleases. This is because the meanings with which observed others approach their artifacts are inherently inaccessible. This is the unfortunate situation in which designers find themselves whose research consists of observing how existing artifacts are used, or watching videos of where their design will have to function – without the ability to ask questions of those involved. While the meanings that guide observed others’ interfaces are then not knowable, this does not prevent designers from studying the manifestations of observed others’ meanings, meaningfulness. We define: The meanings of an artifact are manifest in the set of contexts into which a community of its stakeholders places them – deliberately, i.e. to a degree better than chance.

It reaffirms the foregoing, namely, for artifacts to have meanings to somebody, they must be able to function in different contexts, and these contexts must be manifestly non-arbitrary. As suggested above, without evidence that choices have been made regarding appropriate contexts in which an artifact is allowed to occur, the concept of meaning would be meaningless. This definition also refers to a community of users – stakeholders of all kinds, including designers. People may have different reasons for using an artifact in a particular context and different interpretations for what it does there. However, if an artifact is observed to have been placed in a limited number of contexts, it is reasonable to assume

364

Product Experience

that these are the contexts in which that artifact makes sense to their stakeholders, that the observed artifact–context relationships are manifestly meaningful to them. In other words, without the ability to inquire into the reasons for the choice of contexts, all that designers can observe is how meanings manifests themselves, which is the set of contexts in which an artifact is presumed meaningful (see figures 14.13–14.16).

FIGURE 14.13

FIGURE 14.14

FIGURE 14.15

FIGURE 14.16

Often it is possible and desirable to differentiate among communities according to the set of contexts in which a particular artifact makes sense to them. We already mentioned the network of rather different stakeholders through which the various incarnations of an artifact needs to pass on its way to retirement. We know of cultural differences among users that result in different sets of acceptable contexts for an artifact. One may also acknowledge unequal competencies, interests, and willingness to acquire new meanings.

Semantics: Meanings and contexts of artifacts

365

In the absence of verbal communication, but also if one has reasons to distrust what people say about their uses, human-centered designers need to know the set of contexts in which existing artifacts occur. However, inasmuch as a design differs from what already exists, designers may have to extend this list and decide in which contexts their design should work and where it should not. For example, there are contexts in which artifacts could harm their users and safety measures become important, preventing hands geting into gears. There are contexts in which one community should have privileged access while another community should not, to medications in hospitals. And there are contexts that artifacts transform into other contexts, impacts on the environment. For a design to have the potential of being or becoming meaningful to their stakeholders calls on designers to study the set of contexts in which similar artifacts occurred. The fact that the experiences and meanings of others are observationally inaccessible has not prevented designers from interpreting the manifestations of others’ meanings in their own terms. They may justify this practice by claiming that they share the same cultural heritage with the user of their design, and believe they can speak for that fictional user. While this claim may succeed with similar minded clients, our overwhelming experience is that designers are different from those who have a stake in their design. When sending them into the field, they often return surprised, if not shocked, in disbelief that potential users experience their design in ways far from expected. Our experiences can serve as a warning against unwarranted projections of observers’ meanings to the meanings held by the observed. This is why the above definition refers to manifestations of meanings, not the meanings themselves, and encourages a concern for the contexts in which a design needs to make sense. For example, we can observe a knife on a dining room table, in a kitchen, and in the hand of someone arguing with someone else. These observations, however, merely suggest that it means something to those involved in these contexts. Whether the context in which the knife turns up in someone’s hand is a robbery, a negotiation of its sales price, a theatrical performance, or a criminal trial in a court of law, makes all the difference. To come to know what something means to others, much needs to be known about the stakeholders involved, the culture in which that something is contextualized, the rules governing its context of use, and, most importantly, what the people who are part of that context tell each other about what its use means to them. A good example is presented to us in the first 12 minutes of the 1984 movie titled The Gods Must Be Crazy. We see a small airplane crossing the Kalahari Desert. Bushmen live there and we are told they consider airplanes as evidence of gods in the sky. Its pilot finishes a Coke and throws the empty bottle out of the window. The bottle lands near a bushman named Xi, who, having never seen a glass bottle before, first carefully probes it with a primitive tool and then, having convinced himself of no apparent danger, takes it home to his tribe. There we see the bottle being collectively examined with curiosity, if not awe. Although the camera cannot show what the bushmen think, we are told that they consider it a gift from the gods. The Bushmen turn out to be ingenious in finding all kinds of uses for it. In a place without rocks, the hardness of the bottle encourages its use as what we would call a pestle for smashing roots. Its smoothness is seen to aid the flattening and stretching of snake skins. Its opening finds its use as a stamp for decorating a garment with circles. It turns out to make sounds when blowing over its opening, as well as when it turns, tied horizontally at the end of an untwisting rope. No one discovers its use as a container, perhaps because water is scarce in the desert. But because there is only one of its kind and it is popular for its many uses, the bottle also encourages competition, creates hostility, and starts to hurt somebody in ways, we are told, these Bushmen had not known before. The commentator explains that the gift from the gods becomes

366

Product Experience

an evil thing, which Xi, the leader of the tribe, then tries to get rid of, encountering all kinds of misadventures. Without sound, we have no trouble observing the many contexts in which what we call a Coke bottle seems to make sense to the Bushmen, but not what it means to them, hence the need of a commentator who claims to speak for the tribe. Whether the commentator’s interpretations are fair is something we cannot judge without asking the Bushmen themselves. The point of the story is that we may observe the many contexts in which an artifact is used. Assuming their use to be deliberate, we can conclude that they must be meaningful to those involved. As observers, we can create our own interpretations. However, understanding the meanings that others bring to a situation requires communicating with them, not more extensive observations.

2.2. Interfacing with artifacts according to what they mean, thus being part of their context Observing artifacts in use by others takes place from a position outside their artifact–context relationship. When actually interfacing with artifacts, experiences arise from inside that relationship and unfold in time. The difference between observing and being in such a relationship accounts for the difference between the ability to describe what it is that others experience, the experienced – of course only in observers’ terms – and experiencing one’s involvement in an interface first hand. When observing artifacts in use, one can at best speculate about their possible meanings. When interfacing with them, the set of possible meanings are reduced to those actually understood and also afforded, and when sharing these experiences with us, to those that can be articulated. For those actively involved: The meanings of an artifact are the recognizable actions and articulations it affords a community of its stakeholders.

The example from The Gods Must Be Crazy already demonstrated the limitations of observation. While the movie showed much talk among the Bushmen, it had to rely on a commentator to speak for them and bring their experiencing closer to us. An example closer to experiences in many US cities, is the illegitimate use of milk crates. Manufacturers ship milk to retail stores in crates containing smaller individually sold milk containers. These crates are made of sturdy plastic, and of a size that people can handle. Despite printed warnings that literally criminalize misuse – unintended from their manufacturers’ perspective – people invent numerous and widely popular applications for them. In the context of a home, they can become laundry baskets. Workers use them as stackable containers for small tools and supplies. Bicyclists tie them to the front of their handlebars to carry small personal items around. Upside down and stacked, they substitute for stepladders or, turned sidewise, they serve as open shelves for books. With their bottom cut out and mounted onto a pole, they enable city kids to practice basketball on sidewalks and in backyards, etc. In these contexts, they are entirely different things, and people are quite proud of ‘their creations’ and willing to explain them. When one is aware of their origin, it makes sense to say that these milk crates have acquired very many meanings to city folks. If a design is to enter these contexts, designers need to know the meanings with which it might be approached. Incidentally, to limit unintended uses, their producers enlarged the holes of these crates so that the milk cartons do not fall through, but smaller objects could no longer be carried in them. Just as different contexts can make the same artifact into different kinds, as in the above examples, so can the same context make different contexts into the same kind. This is the flipside of the artifact–context relationship that meanings do inform.

367

Semantics: Meanings and contexts of artifacts

One experiment, easily reproduced with variations, is to take one familiar context, for example the drawer of a cabinet, and vary what it contextualizes, what it features on its front. We know that drawers need to be pulled open and pushed shut. Given that context, it is amazing to see the diversity of objects that, attached to its front, acquire the meaning of a means to open it. Knobs, handles, and keys are conventional examples, often produced for this purpose. But a hole, a string, a ring, a tie, a nail, a push button, a dice, a nut and bolt, a Coke bottle, even the head of Barbie doll, all acquire nearly the same meaning. Affixing a numerical keyboard to its front adds the suggestion of privileged use. Using a heavy handle suggests the need to apply considerable force. Mounting small objects on the draw – a spoon, a wristband, a pen – may suggest what one finds inside the drawer. Even when painting a red circular surface on the front of a drawer, and nothing else, users are likely to think that they had to push there to open the drawer. A familiar context, like the drawer, can be quite determinative of the meanings of what they contextualize (see Figures 14.17–14.19).

FIGURE 14.17

FIGURE 14.18

FIGURE 14.19

Evidently, intended functions are secondary, if not irrelevant, to the meanings that artifacts acquire. Intentions describe what someone hopes to accomplish, functions, the role that an artifact is to play in a context. Neither is observable. Neither may be shared among the stakeholders of a design. Surely, Coke bottles are produced intentionally. In The Gods Must Be Crazy, the airplane pilot knew that he was throwing an empty Coke bottle overboard. For the Bushmen, however, what fell from the sky was something altogether different and it acquired all kinds of – for us unanticipated – meanings as they assimilated the ‘thing’ into their cultural practices. It would violate the idea of second-order understanding if one insisted that it was a Coke bottle that fell from the sky into the Bushmen’s world. It was a gift from the spirits living in the sky. The same is true for the milk crates in the city. Their manufacturers’ effort to control their use by threatening criminal actions against ‘misuses’ is quite meaningless when few if anyone cares. When it is tied to a pole on the street, it is no longer a milk crate, but a net for a basketball. The original meanings of the objects that could be mounted on the front of a drawer were erased by the context of the drawer. In a market driven information society, where a diversity of stakeholders of a design assert their interests and create their own meanings of what is available to them, one can no longer presume compliance with designers’ intentions. The enforcement of intended meanings requires coercive institutions, for example, the police regarding the use of traffic signs, or the military regarding the use of weapon systems. Eliciting people’s stories that parallel their interfacing with artifacts is a common way to gain access to how users are experiencing the process of their engagement. Such narratives chain kinesthetic senses of actions with perceptions of their consequences, and weave into them the emotions and motivations that go along with them. Protocol analysis, pioneered

368

Product Experience

by Newell and Simon (1972; Ericsson and Simon, 1993), concerns problem solving in interfaces. Interviews, focus groups, even surveys can elicit other kinds of meanings. Even experiments with how subjects interface with given artifacts are framed by verbal instructions about what the experimenter wants the subjects to do. What is observed here is informed by the use of language, and cannot be separated from the subjects’ ability to understand and enact the instructions given to them. The experiment with drawers as the single contexts for a variety of attached objects exemplifies this challenge. The point is that the process of experiencing interfaces with artifacts is not only influenced by the use of language, it also needs to be articulated in order for human-centered designers to take such meanings (other than their own) into account. We need to state what should be obvious from the above, that meanings escape measurements, say, of user-friendliness or efficiency in technical, objective, i.e. non-human terms. Meanings are always tied to the use of language. Inquiries into meanings in the context of ongoing interfaces have to acknowledge two constraints. First, they cannot elicit the meanings of interfaces in their most desirable stage, reliance, when users have mastered their interfaces to the point at which the participating artifact has retreated into the background of what users wish to accomplish with them. It is well known that when one asks experienced knitters, for example, to explain in detail how they knit, they can no longer do it with the regularity and speed at which they are comfortable. Piano players, bicyclists, typists, and experienced artisans do far better without articulating the meanings they expertly enact. This points to a fundamental paradox. While reliance is or should be the aim of human-centered design, information about experiencing reliable involvement is nearly unobtainable. What subjects can articulate and designers can learn from concerns the stage of interfacing that precedes the goal of human-centered design: Exploration. Secondly, such inquiries are limited to studying practices that already exist, or can be simulated by using models or prototypes experimentally. Articulations of meanings require attention to what is experienced, monitored, and acted upon. During such inquiries, subjects typically face relatively stable affordances, which exclude situations in which meanings are not tested, affordances are anticipated but without a history of successes, and contexts are novel or emerge unexpectedly. This situation is addressed in the following section.

2.3. Anticipating context of use from narratives involving particular artifacts Recall that Gibson theorized ‘direct perception’ of affordances. Directness is experienced in states of reliance, when enacting unproblematically afforded meanings, treading familiar grounds, and interfacing with artifacts without experiencing disruptions. In much of our artificial world, these certainties are continuously challenged by the design of new interfaces, new artifacts that may build upon familiar ones but work in new contexts and require new kinds of practices. Under these circumstances, affordances cannot be considered natural and fixed. Adoption of new technologies tends to start with information about the possibilities they offer (Rogers, 2003), often in the form of narratives, whether from neighbors, television, or the technical literature. Narratives shape meanings before they are enacted. They prepare people to act, and serve as hypotheses that have the potential of being enacted. Given suitable motivations and opportunities, narrative meanings may create new contexts of use, not merely derive from them – as addressed in the previous section, 2.2. The Gods Must Be Crazy presents us with a story, a sequence of images, narrated by a commentator. In it, we are told and have no reason to disbelieve, that the Bushmen had never seen what we call a Coke bottle, and that their elder, Xi, was therefore carefully

Semantics: Meanings and contexts of artifacts

369

experimenting with it before feeling confident it could be touched and taken. The movie also shows us how the community appropriated it into their culture. The process started with relating it to their bodies – learning very basic affordances – for example, putting a finger into its opening, trying out different way of handling it, including swinging it like a hammer, and utilizing its unusual properties. Importantly, these explorations were public. They took place with much talk among handlers and bystanders during which, we can only imagine, scenarios were suggested, uses were hypothesized, meanings negotiated, and settled, only some of which were actually enacted, tested for whether they would be afforded. Narratives guided the use of ‘the thing’ away from bodily affordances, outward to becoming a tool for all kinds of purposes. The process of appropriating the thing in the Bushmen culture proceeded from narrating conceivable uses to more abstract characterizations. For example, in use, there also emerged unexpected and unfortunate experiences, that, we are told, had no place in the Bushmen culture. Therefore, ‘the gift from the Gods’ became an ‘evil thing’, part of a narrative that determined the direction of the remainder of the movie: How to give it back to the Gods or get rid of it. Narratives are told to be understood, their contents are inherently imaginable, and the meanings of the artifacts occurring in them are necessarily conceivable – regardless of whether they turn out to be afforded in practice. Consideration of meanings in the absence of experiences, but in the expectation of being afforded, is also of central importance to professional designers. By definition, designers propose something new that would not come about naturally. Their proposals need to narrate desirable futures into being – artifacts, contexts of use, and practices – and such narratives must fuel the imagination of stakeholders who could realize a design. Whether designers weave sketches, drawings, models, prototypes, and experimental evidence into their arguments, effective proposals suggest meanings that must be conceivable and compelling. This leads to our third way to inquire into the meanings of artifacts: The meanings of an artifact are the narratives in which that artifact can occur, conceivable and realistic to a community of stakeholders.

Taking this definition seriously opens another line of inquiry. It would suggest asking people to tell us all the stories they know of the artifact in question or can imagine, provided they are considered realistic, not fiction. One should note that we mostly learn about new artifacts through narratives, can organize our knowledge of artifacts in narratives, and interfaces often follow scripts first conceptualized as narrative. Scenarios, a sequence of possible situations that a user can navigate, conforms to a narrative structure. Narratives told to novices may render complex interfaces manageable. One can extract a list of the conceivable contexts of an artifact from available narratives, extensive interviews, and focus group data by content analysis (Krippendorff, 2004a). Naturally, such lists can become extensive, depending on the creativity of those asked, but also on the interpretive flexibility of a technology (Pinch and Bijker, 1987) that prevails in a community. As almost anything can be rearticulated and obtain new meanings, such lists may not be finite. Yet, human-centered designers need to understand the diversity of meanings that their design can suggest and to whom, and which ones could be learned, under which circumstances, and which narratives can and need to be told and to whom. Designers also have to decide which meanings should be afforded by their design and which are to be discouraged. As a rule, designers have two ways to accomplish the latter: The semantic way, using forms unlikely to encourage undesirable uses, for example camouflaging ways for non-professionals to open an appliance; and a physical way, introducing mechanical constraints on what the artifact can do (forcing functions, according to Donald Norman, 1988), which makes undesirable contexts impossible.

370

Product Experience

Designers are not the only ones concerned with narrative meanings. Although ordinary people are not expected to have a second-order understanding, or to consider large numbers of contexts, as designers need to do, meanings that are narrated and create future contexts of use occupy much of everyday life. For example, people easily envision and can be observed to argue with their partner how the furniture they see in a department store might look in their living room, what it means to wear a particular outfit to a social occasion, or what one might experience when flying a hang glider for the first time. When the enactment of meanings is afforded, when the interfaces they guide are not disrupted, we may easily come to the mistaken conclusion that the reality we are facing is known to us. That meanings, even when afforded, do not represent what an artifact actually is or does is easily exemplified by the everyday use of computers. Obviously, the near incomprehensibility of how computers work internally does not prevent people from narrating how to use their computer, using metaphors, for example, from the familiar paper world, to conceptualize what goes on inside it. Dragging the icon of an unwanted file to an icon in the form of a wastepaper basket initiates much computational work, which we can hardly imagine. In addition, most options that contemporary computers do provide are never explored by any one user. Thus, successfully and reliably enacted narratives hardly bring us closer to reality, only closer to what we wish to do. Nor does the experience of failure, disruption, or breakdown in one’s interfaces provide us clues as to why they happen. Speaking metaphorically, one might say that material reality is not a helpful communicator. It cannot possibly know the narratives that prove not to be afforded. Artifacts can object to how they are treated, but they cannot reveal the reason of their objection. When facing a disruption, users need to modify or replace the unworking narrative by a more viable one, but the latter can be held on to only until another disruption occurs. Karl Popper’s insight, that empirical tests can disprove a theory but never prove its enduring correctness, applies to enacted narratives as well. We need to emphasize that narratives, meanings, perceptions, and affordances, are never guaranteed. Something may turn out not to be what it appears to be, whether by misreading or deception, pretentious semiotization, for example. Artifacts can break down when least expected. With an unintended click on an icon, computer users may find themselves in an unwanted world. Ignorance of the cultural context of a design may get people into trouble. For example, in the US flipping an electrical switch upwards turns the equipment on. In Britain, flipping it upwards turns it off. Smiling in Japan has different meanings than smiling in Europe. Confusing the colorful medication of their parents with candy can get children into serious difficulties. Incidences like these are what human-centered designers need to address. It is all too easy, but typical and convenient, for people to blame their artifacts for failing them, for example, telling stories about faults in material, artisanship, production, or design. However, an epistemologically more appropriate strategy would be to search for the causes of mishaps in one’s own conceptions, replacing inadequately attributed meanings by those that prevent future disruptions. This recommendation applies to designers as well. Designers often blame the stakeholders of their designs for misunderstanding their ideas, failing to use their design as intended, ignoring written instructions, misinterpreting crucial clues, pressing the wrong button, or being careless and ignorant. We are suggesting that human-centered designers can no longer play the authority on how their design is to function, but need to face the multiplicity of narratives that the stakeholders in their design could bring to the scene or are willing to learn and enact. There is one kind of narrative that designers need to address; these are narratives that can keep users trapped in untested conceptions. Such narratives underlie the phenomenon of technophobia or the fear of using certain technologies. The emotion of fear

Semantics: Meanings and contexts of artifacts

371

is based on narratives of the possibility of being harmed so terribly that one does not dare to go near that possibility. It is considered rational by those who share that fear, and irrational by those who do not. The pathology consists of being unable to escape from that conception by not risking to verify it. Technophobia typically emerges by generalizing the experience of failures in interfaces with a particular technology. Even in a less drastic form, the unwillingness of trying new things for fear that the comfort of traditional practices may be lost is a major bottleneck for the design of new artifacts. One of the more challenging aims of designers is to encourage narratives that reduce technophobia. Eliminating disruptions of an interface may not be possible entirely, but preventing human errors from having serious consequences and allowing them to be reversed without punishments are options that designers may consider. The ability to undo an action, going back to a computer screen that preceded the realization that a wrong path was taken, did much to overcome the resistance to adopting PCs. Other complex artifacts which are the target of technophobia, are genetically engineered food, nano-technology, the health system, the ecology, and government, which are treated with suspicion and reluctance to get involved. The latter still await redesign in human terms, including narratives that are not entrapping.

3. THREE CONCLUDING OBSERVATIONS 3.1. Meaninglessness One may be asking whether there are contexts that cannot provide meanings to artifacts. Although we humans can hardly escape making sense of our world, making up stories even if and especially when there seems to be no ground, in making meaningful connections between things and their surroundings, there are occasions where we: • Cannot or do not care to distinguish between something and its surroundings and have no reason to explain the difference, for example, when one faces a seamless continuum, when the two are causally determined – not the result of human actions – or simply of no interest. • Presume total arbitrariness of the relation between something and its surroundings, as when we cannot imagine any intentionality, even a mythical one, for example, when something is dropped accidentally, regretfully lost, or found where it does not belong. • Have no history of making sense of it, such as an inexplicable happening. Such incidences are rare and their validity may be questioned. However, once we draw a distinction, we imply that the results of the distinction are different, which is the condition for meaning to arise. Taking the context of an artifact as ‘accidental’ or ‘without a history of making sense’, is in fact a narrative with a minimal explanation. Inasmuch as humans live in narratives, with or without a physical basis, meanings are nearly inevitable, but they differ in their usefulness and ability to act on them. Backgrounds are always implicated in constructing meanings. They may not be meaningless, but merely not noticed – except in comparisons. Within any one culture, how artifacts relate to a culture’s grand narratives is typically backgrounded, unnoticed from within. In India, for example, it makes sense to relate artifacts to her national identity, including Hindu mythology. Mahatma Gandhi’s use of the spinning wheel, wearing of nonregionally marked linen clothes, and promoting a simple village life provided the mythological context of India’s unity in her struggle for independence (Balaram, 1989). In the

372

Product Experience

industrial West, technological progress, efficiency and accuracy, and the democracy of the market place provides the taken-for-granted context that yields meanings that are difficult to analyze from within. Even deliberate attempts to invent national identities, for example, Scandinavian design or Japanese design; if successful, they may become so pervasive that they are no longer distinguishable from within. However, looking from one community or culture to another sheds light on one’s own grand narratives. It would be a mistake, however, to conclude that what has meanings to us has the same or any meanings for others.

3.2. The size of a context One may also want to ask how large a context has to be for an artifact to acquire the needed meanings. This question is more difficult to answer. Natural scientists would argue that the notion of context defies formalization, as it has no boundary. It can be literally limitless in size. For a particular person, however this is rarely a problem. The size and features of a context consulted for an artifact to make sense depends on the depth of meaning with which one would be satisfied. To use our drawer example, one user may be content with focusing on the drawer that needs to be opened. A designer may look at how the manner of opening drawers relates to the cabinet as a whole – functionally, economically, and aesthetically. The writer of an article that discusses the cabinet may relate it to its contemporaries, to other works by its cabinetmaker, to the cultural period in which it was made or used – and so the context expands. However, for any one effort to suggest desired meanings there is a subjectively clear limit below which one has the feeling of insufficiency and above which one may be bothered by redundancy. Questions concerning the appropriate size of a context have a corollary: How resistant is an artifact to the imposition of meaning from its context? The answer to this question depends on the amount of detail it exhibits. A circle painted on the drawer’s surface does not provide much detail and is relatively ambiguous as such. In this context, it might simply suggest: ‘Push here and the drawer will open’. Geometrically simple shapes inform very little by themselves and are usually ready to assume various meanings in different contexts. A complex artifact is more resistant to the imposition of meanings from its surroundings. A bicycle, for example, is a bicycle regardless of whether it is displayed on the box it was delivered in, someone rides it on the street, a tarp covers it for protection from the elements, or it finds itself piled up in a junk yard together with other recyclables. In these diverse contexts, a bicycle is sure to acquire different attributes – new, fancy, lightweight, protected, or broken, but its bicycleness is rarely modified by these contexts. The archetype of a bicycle is only minimally affected by where it occurs (see Figures 14.20–14.22).

FIGURE 14.20

FIGURE 14.21

FIGURE 14.22

Semantics: Meanings and contexts of artifacts

373

As a rule, the difference in details exhibited by either side of the asymmetrical artifact–context relationship informs the meanings of the less complex side. If the artifact has a simple structure relative to that of its context, its meaning is likely to be determined by that context. As the artifact exhibits more details about its identity, it becomes increasingly resistant to contextual determinations. Designers need to balance the amount of details or information provided, depending on what the stakeholders of a design are familiar with and expect.

3.3. Metaphors revisited Simplified, a metaphor is seeing one thing in terms of another. Although this statement does not tell the whole story, it stresses changes in perception (seeing) as the defining consequence of metaphors. For example, the metaphorical statement ‘The village is covered with a blanket of snow’ invariably conjures images of coziness, warmth, and friendly village folks despite the wintry cold and without literally saying so. How does this come about so reliably? The study of linguistic metaphors has recently progressed beyond the mere identification of this rhetorical trope. Following Lakoff and Johnson’s (1980) seminal work, we identify four necessary features of linguistic metaphors: 1. Metaphors operate between two conceptual domains, a target domain of something presently attended to, here, a snowed-in village, and a typically more common source domain, here, the human use of blankets to cover themselves. 2. The vocabularies used in either domain enables one to construct a superficial structural correspondence between them, here, ‘being covered’. 3. This albeit tenuous correspondence serves as the bridge for entailments from the source domain to enter and inform the perception of the target domain, here, the warmth and comfort provided by being covered with a blanket. 4. The entailments of a metaphor organize their users’ target domain, here, one perceives a friendly place where one can feel comfortable and warm, and be with well intended people – unambiguously but without using such words literally. In powerful linguistic metaphors, the vocabulary imported from a source domain continues to grow in the target domain, taking it over, so to speak. Consider the familiar ‘war on drugs’ metaphor. Its target domain, the ‘drug scene’, is far from clear by itself, but ‘war’ certainly is. War entails urgency, to which a government must respond by allocating extraordinary resources to win. War also entails enemies that users of this metaphor promptly construct. War is not merely talked about, it must also be fought and won. In the US, the war on drugs is a mission of several Federal agencies, etc. The use of this metaphor reconstructs the use of drugs in terms originally reserved for war. While non-linguistic or visual metaphors may not proceed quite as straightforwardly, they too alter the perceptions and subsequent actions by those who recognize them. In the domain of artifacts, we contend that the shift of our attention to their contexts opens a new understanding of non-linguistic metaphors. Before detailing how non-linguistic metaphors work, we should warn against vulgarizing the concept. A telephone in the shape of a Mickey Mouse, or a radio with an MP3 player in the form of a car, as in Figure 14.23, currently sold by Sharper Image, do not qualify as metaphors, as their appearances have nothing to do with their use and may even impede it. Similarly, a truck in the shape of a huge hotdog may well serve to advertise a hotdog manufacturer, but sausages do not teach anything about driving a truck, much less about advertising. These are examples of what we have called ‘pretentious semiotizations’ (Krippendorff, 1988, 2006), ‘pretentious’ because such products pretend

374

Product Experience

to be something other than what they are, and ‘semiotization’ because their forms rely on a representational theory of meaning – defining meaning as what a sign (in the case of an artifact, an icon) represents (Mickey Mouse, a car, or a hotdog). These examples are neither ironical, as their appearance and use fail to produce tension between ignorance and understanding, required by irony, nor paradoxical, as neither denies the other. They are simply pretentious of something else. Now consider two examples of non-linguistic metaphors.

FIGURE 14.23

Just as with linguistic metaphors, non-linguistic metaphors bridge two empirical domains. The source domains of most metaphors are more familiar than their target domains, which are often difficult to understand. For example, the metaphors by which computers are being understood by their users is the world of paper handling, creating documents, reading them, organizing them into files, and moving them individually, in file folders, or packages of file folders. As already mentioned, what computers actually do, for example, when users think they open a file or discard it, is something altogether different. Trashcans are familiar to us from a context in which household trash is collected and removed. In the metaphorical paper world of a computer screen, dragging the icon of a document, which iconically resembles a document in everyday life, to the icon of a trashcan, which iconically resembles familiar trashcans, suggests disposing of that document. The computer, however, merely changes the index of the information thought to be contained in this file. This iconicity enables experiences from the context of the source of a non-linguistic metaphor to inform the perception of what can be done in the context of its target. Without such entailments, without the ability to draw on experiences in an absent but familiar context, pictorial resemblances mean little, if anything. What distinguishes metaphors from analogies, metonymies, ironies, and iconic representations is their capacity of making practices from the context of familiar but absent source domains available in a less familiar but present target domain. One example may suffice. In the early 1970s, high volume paper copiers had become complex machines, almost printing plants. Not only did they become increasingly difficult to handle, but frequent breakdowns required costly repairs by experts and caused disruptions and delays in office work. Xerox designers reconceptualized the machine in terms familiar to the office workers who used them. The overall shape of the new copier resembled a raised table on which office workers are accustomed to sort their paper work, now while standing. Its horizontal plane was interrupted by an indentation, resembling a familiar paper tray in which office workers tend to keep piles of documents, face up to work from, here, to be copied. The copies exited at the other end, suggesting that the paper flowed

Semantics: Meanings and contexts of artifacts

375

through the machine in one direction, moreover making it difficult to insert documents where copies were to exit. The front of the machine featured drawers, just like in familiar chests. In the context of copying, it was natural to keep blank paper (of different sizes) in these drawers. Similarities or structural correspondences between features of the new copier and of artifacts that users know from a familiar context served as the conduit of experiences and expectations from familiar contexts to the context in which the new copier was to work. The Xerox designers also introduced what we call semantic layers. While copying documents, office workers operate in one layer of coherent metaphors. When something goes wrong, they can open a door and find themselves in another layer, a world with well-marked paper flows and colored handles to undo paper jams. This second layer shields the user from the complexities of a third semantic layer, which is accessible only by trained repair personnel. Today, the metaphors employed in the Xerox copier have become so standard that their origin is nearly forgotten. This is the fate of most metaphorical meanings. Their novelty wears out over time and their meanings end up being indistinguishable from literal ones. Thus, metaphors are not entities that could be photographed or recorded. They organize perceptions and render understandable situations that users have difficulties with comprehending. They are rooted in what poets have always known and human-centered designers may utilize as well. Whereas linguistic metaphors transfer meanings by adopting vocabularies from a source domain in a target domain, visual metaphors transfer meanings from the context of a source domain to a target domain, including the artifact’s context of use. Our elaboration of how meanings connect to the contexts in which artifacts are used brings non-linguistic, particularly visual, metaphors into a clearer focus. Whereas the entailments of linguistic metaphors reorganize present perceptions, the entailments of non-linguistic metaphors restructure the present contexts of artifacts. The Xerox copier was designed to metaphorically invoke common office experiences – without having to understand how the copier actually works, without fear of being caught in its mechanisms, and without requiring expensive repair services for undoing minor paper jams. Without the use of metaphors in the human interfaces with computers, computer use would have been confined to a few specialists.

4. CONCLUSION In the foregoing, and with reference to Figure 14.12, we have demonstrated how the meanings of artifacts derive from and subsequently direct stakeholders’ conceptions and interactively transform an artifact’s contextualizations. It is not enough to merely talk about their meanings – agree or disagree on what they are, usually at the expense of the diversity of contexts that support them – meanings are best seen as guiding human actions. In human interfaces with artifacts, these meanings are enacted, tested for their affordances, and modified to fit what one wants to accomplish. Metaphors alter an artifact’s context as well, but perhaps more thoroughly, more effectively, and perhaps least noticed. Finding practices to rely on, or ways to create the conditions for desirable interfaces with artifacts to emerge, is important to human-centered designers. Knowing how such practices are transferred from familiar to new artifacts enables designers to intervene successfully into how their designs may be used. Linguistically, ‘success’ comes from ‘succeeding’. In this sense, a design succeeds by traveling through an always-emerging

376

Product Experience

network of its stakeholders – directed by the meanings that its stakeholders attribute to it and that designers may wish to track and utilize.

REFERENCES Balaram, S. (1989). Product symbolism of Gandhi and its connection with Indian mythology. Design Issues 5, 68–85. Dourish, P. (2001). Where the action is; The foundations of embodied interaction. Cambridge, MA: MIT Press. Dreyfus, H. L. (1992). Being-in-the-world; A commentary on Heidegger’s being and time, division 1. Cambridge, MA: MIT Press. Ericsson, K. A. and Simon, H. A. (1993). Protocol analysis: Verbal reports as data (Rev. Ed.) Cambridge, MA: MIT Press. Gibson, J. J. (1979). The ecological approach to visual perception. Boston, MA: Houghton Mifflin. Krippendorff, K. (2006). The semantic turn; A new foundation for design. Boca Raton: Taylor and Francis CRC Press. Krippendorff, K. (2004a). Content analysis; An introduction to its methodology (2nd Ed.) Thousand Oaks, CA: Sage. Krippendorff, K. (2004b). Intrinsic motivation and human-centered design. Theoretical Issues in Ergonomics Science 5(1), 43–72. Krippendorff, K. (1988). Design muss Sinn machen; Zu einer neuen Design Theorie. Paper presented at the International Design Forum, Ulm, Germany, September 2–4, 1988. Proceedings published 1989. Krippendorff, K. and Butter, R. (1984). Exploring the symbolic qualities of form, Innovation, 3(2), 4–9. Lakoff, G. and Johnson, M. (1980). Metaphors we live by. Chicago, IL: University of Chicago Press. Latour, B. (1996). Aramis or the love of technology. C. Porter (Tr.) Cambridge, MA: Harvard University Press. Latour, B. (2005). Reassembling the social: An introduction to actor-network-theory. New York: Oxford University Press. Madison (1988). reference lost. Newell, A. and Simon, H. A. (1972). Human problem solving. Englewood Cliffs, NJ: Prentice-Hall. Norman, D. A. (1988). The psychology of everyday things. New York: Basic Books. Pinch, T. J. and Bijker, W. E. (1987). The social construction of facts and artifacts, In: W. E. Bijker, T. P. Hughes and T. J. Pinch (Eds.) The social construction of technological systems, pp. 17–50. Cambridge, MA: MIT Press. Poole, M. S. and Folger, J. P. (1981). Modes of observation and the validation of interaction analysis schemes. Small Group Behavior, 12(4), 477–493. Rogers, E. M. (2003). Diffusion of Innovation (5th Ed.) New York: Free Press. Varela, F. J., Thompson, E. and Rosch, E. (1991). The embodied mind: Cognitive science and human experience. Cambridge, MA: MIT Press. Wittgenstein, L. (1953). Philosophical investigations. Oxford: Blackwell.

16

CONSUMPTION EMOTIONS MARSHA L. RICHINS University of Missouri, Columbia, MO

1. INTRODUCTION Consumption emotions add color and texture to our daily lives. A young woman beams with pride and delight as she shows her new engagement ring to friends. A homeowner angrily throws an electric can opener in the trash when it jams, yet again, while opening a can of tomato sauce. A stressed-out college student looks at his baseball mitt and feels a little better knowing that he can play ball when he’s finished his math assignment. All these are examples of how products elicit consumption emotions, and they can be multiplied many times over as consumers interact with products throughout each day. Consumption emotions are clearly important to consumers, who purchase products to meet their needs and achieve goals. These emotions constitute a signaling system that tells consumers whether they’ve achieved their goals. In addition, certain kinds of emotions, particularly positive ones such as joy and excitement, are inherently satisfying, and the experience of such emotions is itself a goal for much consumption behavior. Because consumption emotions are so powerful in regulating consumer behavior, they are also important to marketers. Positive consumption emotions generate brand loyalty and commitment, and both positive and negative emotions influence consumers’ word of mouth. Much of daily discourse concerns products as a central or peripheral element, and the feelings consumers experience as they use their products influence what they say about them (Westbrook, 1987). The buzz or word-of-mouth surrounding a new product, especially, is often critical to the product’s success or failure. In these marketer-relevant examples, consumption emotions are important because of their association with customer satisfaction, but knowledge of consumption emotions is important in other marketing arenas as well. For instance, knowledge of consumption emotions can be useful to advertisers as an input to the creative process of developing Product Experience Copyright © 2008 Elsevier Ltd.

399

400

Product Experience

communications that resonate with consumers and increase their interest in purchasing the advertised product.

1.1. Scope of the chapter In this chapter, we explore the emotions elicited by products as consumers own and use them. The extant literature includes a considerable body of research designed to help marketers understand and elicit the kinds of emotions that will encourage product purchase (e.g. McDonagh et al., 2004; O’Shaughnessy and O’Shaughnessy, 2003; Pullman and Gross, 2004). However, much less attention has been paid to the emotions consumers experience after purchase as they possess and consume the product. These postacquisition emotions are the focus of this chapter. In keeping with the goals of this book, and to keep the discussion to a manageable size, we deal primarily with tangible products and for the most part exclude emotions elicited by the shopping experience or the consumption of services. This chapter begins with a brief discussion of the conceptualization of consumption emotions, which is followed by an examination of how the nature of products themselves influence their potential to elicit affective reactions. The chapter also discusses how prepurchase processes set the stage for consumption emotions, identifies the conditions that elicit consumption emotions, and reviews the methods that have been used to assess them.

1.2. Consumption emotions defined The nature of emotion has been described in detail in the chapter by Pieter Desmet, so it is necessary to provide only a brief description here. For our purposes, consumption emotions are defined along the lines proposed by Ortony, Clore and Collins (1988), who consider emotion to be a valenced affective reaction to perceptions of situations. This definition excludes mood, which is a background affective state that frequently arises independent of situations, and nonvalenced cognitive states such as interest and surprise. A key factor in this definition is the qualification that emotion arises in response to perceptions or appraisals of situations that are relevant to the perceiver (Lazarus, 1991; Smith, 1989). These appraisals assess the extent to which a situation appears to have positive or negative consequences for a person’s goals and well-being. For this chapter, the set of situations of interest to us is consumption situations; that is, occasions upon which people are consuming, contemplating, or otherwise using a product in their possession. A consumption emotion, then, is an emotional reaction that one has in response to a product in a consumption situation. It is immediately evident that the vast majority of consumption situations involve no emotional reactions at all. We are constantly and perpetually involved in consumption. Yet for much of the time, we don’t think at all about the products we consume. I may not even notice the soap I use to wash my hands, the carpet on my living room floor, or my desk lamp. The clothes I wear from day to day are unlikely to elicit an emotional reaction, unless I should happen to drip spaghetti sauce on them (dismay), be complimented about them by a coworker (pleasure), or have difficulty with a zipper (frustration or embarrassment, depending on the situation). Yet some products or consumption situations can provoke intense emotions, including panic (as when the car won’t start when your wife is experiencing labor pains), joy (the delight of new golf clubs, for instance), frustration, or despair. In the following sections we explore the factors that influence whether consumption emotions will be elicited

Consumption emotions

401

and the intensity of these emotions when they do arise. We begin by examining how the nature of the product itself may influence the experience of consumption emotions.

1.3. Product categories and consumption emotions At the outset, it is necessary to make a distinction among three categories of products that differ in their likelihood of eliciting consumption emotions. It should be noted that the boundaries delineating these three categories are somewhat blurred and that the categories themselves are somewhat fluid, depending on characteristics of the consumer using the product and the situation in which the product is intended to be used. Rather than providing an immutable taxonomic scheme, these categories are intended primarily to simplify the discussion of consumption emotion processes, and particularly to demonstrate how such processes may vary according to the nature of the product, the consumer, and the usage situation. The first category consists of mundane products – canned peaches, toothpaste, motor oil, and paperclips, for instance – that have a low probability of inducing consumption emotions. Such products create feelings only in unusual circumstances, such as when they fail or when an unexpected product benefit is revealed (Wow! I didn’t know I could use baking soda to remove the blood stain on my shirt!), or when an unusual or especially elegant product design is encountered that temporarily lifts the product from the mundane to the extraordinary. Although these products are relevant to our well-being, we use them so routinely and their performance is so predictable that they slip below the emotion radar unless something unusual happens. Some products, on the other hand, are likely to elicit consumption emotions in nearly all consumers, at least for the immediate time period following purchase. Most consumers get an emotional buzz when they drive a newly purchased car. Moving into a new house may generate strong feelings of excitement, anxiety, frustration, or other emotions. For purposes of this chapter, such products will be referred to as extraordinary products. Products that tend to fall in this category are those that are expensive, those with strong hedonic or experiential qualities (Holbrook and Hirshman, 1982), those with strong symbolic meaning, such as self-expressive products (including fashion goods), and status products. Extraordinary products are either rarely purchased (a car, for example) or highly differentiated from one another (fashion goods or a music CD). The unusualness associated with these products almost demands that they be appraised in terms of whether they promote or thwart our goals and whether they advance or threaten our well-being. In addition, many extraordinary goods seem quite significant in our quest for well-being (a house, for instance, seems much more significant than a paperclip), and this significance can amplify the intensity of the emotions associated with extraordinary products. Between these two extremes is our third category – a large group of products that may shift in status from mundane to extraordinary, depending on the consumer or the consumption situation. Such products do not universally generate consumption emotions, but may do so in some people or on some occasions. A T-shirt, for example, can be mundane if you just need something to wear, or it can be quite important if you’re wearing it on your first date after a divorce (does it match? is it too loose or too tight? does it look too nice? not nice enough?). For some people (i.e. the highly fashion conscious), T-shirts may be important all the time and constantly have the potential to generate consumption emotions. This category of goods are called conditional products because, more than products in the other two categories, the elicitation of appraisals and associated consumption emotions depends on the situations in which they are consumed or the nature of the person consuming them.

402

Product Experience

While this third category of products is the most dependent of the three on person and situation, this happens to some extent for all three product categories. The following sections describe how the nature of the products in each of the categories, the circumstances in which they are consumed, and the personal qualities of the consumer interact to generate consumption emotions.

2. SETTING THE STAGE FOR CONSUMPTION EMOTIONS Although consumption emotions are generally restricted to feelings experienced after acquisition or possession of a product, to understand them we must begin by examining the consumers’ cognitive and affective processes before acquisition. What goes on before acquisition will have an important influence on whether consumption emotions are experienced at all, and on the nature and intensity of those emotions that are experienced.

2.1. Consumption hypotheses For most products, consumers have some beliefs about product performance prior to purchase. When buying a laundry detergent, for instance, one expects it to do a reasonably good job of removing odors and cleaning clothes. Food products are expected to be sanitary, and an automobile might be expected to achieve a particular level of fuel economy. These beliefs have been termed ‘expectations’ in the satisfaction literature (Oliver, 1997, see Chapter 3). Many expectations are unconsciously held and activated only when something unusual happens, when considering a brand switch, or when one has or hears about an unsatisfactory consumption experience involving the product. For many more important products, however, consumers’ prepurchase cognitive processes go much further (Fournier and Guiry, 1993; Phillips, Olson and Baumgartner, 1995). Consider a recent college graduate contemplating the purchase of his first new car. He is likely to have the usual expectations about functional characteristics of the car (fuel economy, handling, repair frequency, quality of the sound system, etc.). But as he thinks about his upcoming car purchase, searches the internet for car information, and test drives cars in his consideration set, he also begins to engage in a series of creative and imaginative thoughts. As he slides into the car and touches the leather upholstery, he imagines what it would be like to sit in this car every morning on the way to work. He grips the steering wheel and stares over the hood, noting the sporty lines of the body and the perfect finish of the paint. What will other drivers think, he wonders, when he pulls up next to them at the stop light, his window down, the sound system turned up just loud enough for others to hear? Especially for purchases with a long prepurchase period, there’s plenty of time for such contemplation. Our young driver might imagine himself showing his new car to his friends and try to anticipate their reactions. He tries to imagine what an attractive woman will think of him as he drives her somewhere for the first time. He visualizes how easy or difficult it will be to park the car, to maneuver in traffic, and whether getting a bright red one will increase his chances of getting a speeding ticket. Many of these contemplations include thoughts about how he will feel should he purchase (and consume) this particular automobile (Bagozzi et al., 2000). In short, this driver is developing hypotheses about the ownership experience. Although these consumption hypotheses have some elements in common with expectations as they are described in the satisfaction literature, they differ in that they may

Consumption emotions

403

involve considerable cognitive effort, mental focus, and active imaginative processes (Campbell, 1987). Consumption hypothesis development is not limited to young people or the purchase of new cars. It’s not uncommon for women to try to imagine how they will look and feel in a new article of clothing. When planning a vacation, consumers try to envision particular vacation experiences and develop hypotheses about what a cruise vacation would be like for their family, for instance, as compared with a trip to the mountains. Before purchasing a house, one mentally tries out the daily commute, the yard maintenance, the parties that could be hosted, the play and sleeping arrangements for the children, and how difficult it would be to clean the bathroom. Such consumption hypotheses are influenced by a variety of factors, including advertising and other manufacturer representations, one’s prior experiences with similar products and the experiences of others, media representations of product use (such as in movies or television programs), and even by consumer folklore. For extraordinary product purchases especially, these consumption hypotheses are instrumental in setting the stage for the emotional experiences that occur later during the consumption process (Klaaren, Hodges and Wilson, 1994; Wilson et al., 1989).

2.2. Prepurchase affective states While consumers are busy imagining how they will feel should they acquire a particularly important product, they are also busy experiencing a variety of prepurchase emotions. Like consumption hypotheses, these prepurchase emotions can have significant influences on the experience of product-related emotions after purchase. While the gamut of prepurchase emotions is quite large and is limited only by a consumer’s imagination, we focus here on two prepurchase emotions likely to have significant impact on the later consumption emotions experienced by consumers. Hope When a product is particularly important and desirable, its purchase contemplation may be associated with a great deal of hope and yearning (MacInnis and de Mello, 2005). Parents hope that the purchase of specialized educational software will help their child with a learning disability do better in school. A couple hopes that installing a hot tub in their back yard will increase the time they spend together, improve the communication between them, and save their marriage. A young woman with a persistent weight problem who contemplates purchasing a weight-loss supplement for the first time may be filled with hope. As she reads testimonials, gazes at before and after photos, and looks at models in magazines wearing swimsuits, her hopes intensify. In each of these situations, hope and imagination may interact and reinforce one another to create a strong desire for the product and a persistent longing for the desired outcome (Belk, Ger and Askegaard, 2003). Prepurchase hope can have two seemingly-paradoxical effects on consumption emotions. On the one hand, very high hope and overly-optimistic consumption hypotheses can be hard to sustain in the face of reality, and when one’s weight is not reduced, one’s marriage continues to founder despite the hot tub, and your child continues to struggle in school, disillusionment and dissatisfaction seem to be the inevitable result. On the other hand, however, de Mello and MacInnis (2005) argue for the opposite. They suggest that prepurchase hope tends to bias post-purchase product evaluation in a positive direction. That is, hope can create an optimistic bias that encourages consumers to overlook disconfirming evidence and instead focus on (and search out) shreds of evidence that suggest their hopes have in fact been fulfilled. This biased processing and

404

Product Experience

resulting misperception of the consumption experience allow consumers, in a sense, to ‘create’ the consumption emotions they hope for, a kind of ‘reality negotiation’, as it has been described by Snyder (1989). It is unclear when hope will lead to illusion and when it will result in disillusion. Social judgment theory (Sherif and Hovland, 1965) and adaptation level theory (Appley, 1971; Helson, 1964) would suggest that when the consumption experience comes close to fulfilling hopes, consumers can maintain the illusion, but when the consumption experience falls far short, disillusionment is likely to occur and the associated negative emotions may be stronger than if hope had been less intense and compelling in the first place. Anxiety Hope is a predominantly positive emotion, but strong negative emotions might also be experienced when the upcoming purchase is highly important for any of the reasons described above or by Bloch and Richins (1983) in their analysis of product importance. Specifically, consumers may experience anxiety, uncertainty, and indecision before making a major purchase decision (Luce, 1998; Otnes, Lowrey and Shrum, 1997). There have been only a few investigations of the effects of prepurchase anxiety and uncertainty on post-purchase consumption emotions. In a cross-sectional study, Chaudhuri (1997) found that consumers who experienced negative emotions when using a particular product had higher levels of perceived risk for that product. From these results, Chauduri inferred that these negative emotions led to higher risk perceptions (a form of product-specific anxiety), but the study design makes it impossible to determine the causal direction of this relationship. A small longitudinal study by Roster and Richins (2006) suggests that the causal flow may operate in the opposite direction, and that prepurchase anxiety may instead influence consumption emotions. In this study, consumers were surveyed before making an important purchase replacement decision, and again one year later. Consumers with high levels of prepurchase decision ambivalence (a cognitive and affective state characterized by indecision and anxiety) were more likely to report negative consumption emotions after purchase than were consumers low in ambivalence. In addition to having their own independent effects on consumption emotions, the pre-purchase emotions of hope and anxiety may fuel the hypothesis development process. These emotions may lead consumers to formulate elaborate and idealized consumption hypotheses (in the case of hope) or to develop a set of competing and contradictory hypotheses (both positive and negative) which may be simultaneously imagined and played against one another (in the case of anxiety). The consumption hypotheses and prepurchase emotions described above set the stage for consumption emotions that will follow after purchase. It is likely that the more elaborate and thoughtful the consumption hypotheses a consumer develops and the stronger the prepurchase emotions of hope and/or anxiety felt by the consumer, the stronger the consumption emotions will be after purchase. We now turn, in the next section, to the purchase and consumption phases of product ownership to examine the factors at those stages that influence consumption emotions.

3. ELICITING CONDITIONS FOR CONSUMPTION EMOTIONS Although consumers engage in continual and automatic evaluation of their experiences (Bargh and Chartrand, 1999), including product experiences, most daily consumption activities elicit no emotions on the part of the user because the consumption experience is unremarkable. This is particularly true when the consumer has developed no elaborate

Consumption emotions

405

or conscious consumption hypotheses concerning the product, and has not experienced strong product-related emotions in the prepurchase phase. But in some cases consumption emotions, ranging from mild to intense, do occur. A variety of factors at the purchase and consumption stages can cause emotions to be experienced, and these factors are reviewed in this section.

3.1. Purchase One of the most common circumstances that elicits consumption emotion is purchase. As noted above, purchase or other forms of product acquisition are most likely to generate consumption emotions when consumption hypotheses for the product are welldeveloped, when the prepurchase emotions of hope and anxiety are especially high, or when the product is important in other ways enumerated by Bloch and Richins (1983). These conditions are most likely to occur with respect to extraordinary products (as defined above). For instance, Mano and Oliver (1993) found that student respondents reported both stronger positive and stronger negative consumption emotions when they described their feelings concerning an important and complex purchase than when they reported on a mundane purchase. Consumption emotions are least likely to occur as the result of the purchase of mundane products such as canned foods, toothpaste, or other relatively unimportant and frequently purchased items. For these products, consumption emotions may occur only if there is something unusual about the purchase situation, such as a product failure or when the consumer tries a new brand of a frequently purchased product. In most cases, however, and consistent with appraisal theory, the purchase and consumption of mundane products will not result in consumption emotions because these products generally have limited implications for the consumer’s well-being. Purchase of conditional products, the third category of products described earlier, may elicit consumption emotions if it’s a first-time purchase in the product category or the product is expected to be used in an important usage situation. In some cases, experiences at the time of purchase may influence consumption emotions later on. Gardner and Rook (1988) have described how impulse purchases often lead to a feeling of pleasure and a sense of excitement in the period immediately following purchase, and these emotions may be re-experienced later when the product is used or when the purchaser recounts the purchase occasion to others. It is also likely that memories and residual affect from any highly emotion-laden shopping experience (either positive or negative) will cause those emotions to be re-experienced to some degree when products purchased during that experience are consumed or are used for the first few times. Smart-shopper purchase occasions are also likely to result in consumption emotions later on. These are occasions in which the consumer has received a particularly good deal on a purchase and are most likely to influence emotions when consumers attribute the successful price deal to their own efforts and shopping strategies. Getting a good deal increases consumers’ positive feelings about the shopping experience (Mano and Elliott, 1997) and is expected to induce feelings of pride and other positive affects (Schindler, 1998). Although the studies cited here have examined emotions at the point of purchase, it is possible that these positive emotion states will be reactivated when the purchased item is consumed or displayed to others.

3.2. Influences of consumption situation As the foregoing discussion illustrates, consumption emotions are most predictably experienced after purchase, particularly when the product involved is an extraordinary one.

406

Product Experience

However, elements of the consumption situation or the intended use of a product can generate consumption emotions, even for mundane or conditional products. Several relevant consumption situations are discussed here. An important consumption situation can turn a mundane product into a particularly important one, at least for a short period of time, and during such situations the product might be evaluated in much the same way an extraordinary product is. For example, a suit that just ‘feels right’ and boosts the wearer’s self-confidence may elicit especially positive emotions when it’s worn for a job interview. A cut of beef served to the family is relatively mundane, but when grilling a steak for a dinner party, the importance of the situation tends to magnify any product-related feelings that might arise during consumption. If the steak served to guests turns out to be tough or tasteless, feelings of dissatisfaction may be accompanied by emotions of anger and embarrassment. If the steak is particularly excellent, on the other hand, pride and other pleasurable emotions may be experienced. Two things are happening here to cause this result. First, the emotions that might ordinarily occur in response to satisfaction or dissatisfaction are magnified by the importance of the situation. Hence, the annoyance generated by an inferior cut of meat becomes anger when the situation is important, and the pleasure of a good steak becomes delight. Secondly, the important situation may have elements absent in the more ordinary situation that involve different threats and appraisals, which in turn result in the experience of different kinds of emotions. For example, the cook will probably not experience embarrassment when serving a tough steak to his family, but is quite likely to suffer that emotion if he should inadvertently serve a tough steak to guests. A novel consumption situation may also elicit consumption emotions. New situations have potential rewards and threats not previously encountered, and are thus especially likely to generate appraisals and associated emotions. When driving in the snow for the first time, for instance, discovering that your car has sufficient traction to get you home from work can lead to a feeling of relief. Finding out that your old bicycle works just fine in pulling a baby trailer for your child may result in feelings of pride and pleasure. Consumption emotions are also likely to occur in situations in which social comparison occurs. As Festinger (1954) noted years ago, people tend to compare themselves with others to determine the correctness of their opinions and the status of their abilities, and tend to do so on a somewhat continuous basis. When comparing themselves with others who have superior talents, people tend to feel such emotions as envy, jealousy, and anger (Salovey and Rodin, 1991). When comparing with inferior others, people feel positive emotions such as pride. Ackerman, MacInnis and Folkes (2000) demonstrated that social comparison of possessions can lead to similar emotional responses. Specifically, they found that seeing others who own a product superior to their own may lead to feelings of embarrassment, envy, and anger. Interestingly, however, they found that such social comparisons don’t always result in these negative emotions, but, depending on how the consumer appraises the situation, comparisons with a superior other may on occasion lead to positive emotions. This is particularly true if the consumer is already highly satisfied with his/her possession and does not feel the inequality in possessions is unfair. Ackerman et al. did not test the extent to which positive emotions result from downward comparisons, but anecdotal evidence suggests that negative emotions are a more common response to upward comparison of possessions than are positive emotions. Consumption situations in which the consumers’ product expectations are greatly exceeded are also likely to result in consumption emotions, in this case, positive ones. This is especially true for extraordinary or aesthetic products (St. James and Taylor, 2004), but can also happen for relatively mundane products. An unexpectedly delicious soup at an otherwise ordinary cafe may prove more delightful than an equally tasty soup

Consumption emotions

407

at an expensive restaurant. A neglected flashlight that’s been sitting in your glove compartment for two years, but still works when you need to change a tire after dark may elicit emotions of both delight and relief. It is the combined characteristics of surprisingness and positive product performance that work together to create the consumption emotion of delight (Rust and Oliver, 2000). Product failure or other events that create dissatisfaction are another cause of consumption emotions for products in all three of the categories mentioned above. Because this situation has some unique properties and has been studied more extensively that other situational variables with respect to consumption emotion, we discuss product dissatisfaction in greater detail in the following section.

3.3. Dissatisfaction Dissatisfaction is generally viewed as a cognitive judgment that a product has failed to meet its user’s expectations concerning product performance (e.g. Day 1984; Westbrook and Oliver, 1991). Although dissatisfaction is usually conceptualized as having an affective component, it is generally viewed as discriminant from consumption emotions (Mano and Oliver, 1993). Negative consumption emotions are likely to occur when a consumer is dissatisfied with a product, immediately after purchase or later on. The dissatisfaction can result from some functional failure of the product, such as a lawn mower that is balky and difficult to start, or from a failure of the product to confirm the consumer’s consumption hypotheses. A number of studies have examined the consumption emotions associated with product satisfaction and dissatisfaction (e.g. Mano and Oliver, 1993; Mooradian and Olver, 1997; Westbrook, 1987; see also Oliver, 1997, Chapter 11). Typical emotions experienced in such situations include anger, disgust, and sadness. Unfortunately, research on consumer dissatisfaction has not adequately distinguished between the two different sources of dissatisfaction (functional failure versus failure to confirm consumption hypotheses), so it is unknown whether the kinds of emotions experienced as a result of dissatisfaction arising from these two sources differ. It is plausible to expect, however, that feelings of anger will be more prominent in the first instance, while feelings of dismay, disappointment, and sadness may be more strongly associated with the latter case. Unfortunately, the duration of consumption emotions in either of the above situations has not been studied. It is likely that the duration of these emotions is related to their intensity, the length of time until the dissatisfaction is resolved (e.g. by acquiring a replacement or finding substitutes to meet one’s needs), and the presence of reminders of the unhappy consumption episode. In cases of intense dissatisfaction, it is not unusual for consumers to experience remnants of the original unpleasant emotion associated with a particularly disastrous consumption experience years later upon retelling of the consumption incident. However, even in cases of intense dissatisfaction, the acute emotional reaction does wane with time and is only occasionally reactivated by random reminders or events (Levine and Bluck, 2004).

3.4. Persistent consumption emotions Consumption emotion frequency, intensity, and duration often fade with time after a product purchase has been made (Richins and Bloch, 1986), when the important

408

Product Experience

consumption situation has passed, or when a product dissatisfaction has been resolved. However, there are some exceptions to this general rule. Some products are more likely than others to generate continuing emotion after product purchase. Sheller (2004) particularly singles out automobiles as a source of diverse, enduring, and encompassing emotional response. Hedonic or experiential goods such as a canoe, music CD, or scented candles may evoke positive emotions every time they are used. Fashion and other status goods may also continue to produce consumption emotions long after they were purchased. Because the meaning and value of fashion goods can change rapidly due to cultural and mass media shifts, they are likely to be subjected to frequent appraisal, with the associated consumption emotions undergoing a gradual shift over time. The pride, joy, and excitement experienced shortly after purchase may gradually shift to dissatisfaction and possibly shame if a highly visible fashion good cannot be replaced with something more current, or if a status good has been superceded by a new model or ‘superior’ brand. Other products and possessions continue to evoke emotions long after acquisition for nostalgic (Holak and Havlena, 1992) or sentimental reasons (Schultz, Kleine and Kernan, 1989). Using or viewing objects associated with our youth or with a deceased friend evokes memories of a happier or different time and the associated emotions of happiness, sadness, or wistfulness. Finally, for some consumers a high level of enduring involvement with a product class may fuel strong and on-going consumption emotions, such as that evidenced by car enthusiasts, technophiles, hobbyists, and collectors (Bloch, 1986).

4. INDIVIDUAL DIFFERENCES IN CONSUMPTION EMOTION EXPERIENCE There have been few investigations of the influence of individual differences on the experience of consumption emotions. A study by Derbaix and Pham (1991) looked at gender differences in the expression of consumption emotion by asking 120 undergraduate students at a western European business school to describe 17 different purchase and consumption situations. The authors’-content analyzed the students’ responses for affective reactions and found no differences in number of positive affective reactions reported by males and females, but did find that females were slightly more likely to report negative affective reactions. I was unable to find any other reports of associations between demographic variables and consumption emotions. However, in my own research (an unpublished mail survey of more than 600 adult American consumers), I found no significant correlations between consumption emotions (as measured by the CES, described below) and the demographic characteristics of gender, marital status, education, and income. However, age was negatively correlated with the experience of many consumption emotions, particularly excitement, eagerness, envy, fear, and concern. Personality characteristics appear to have a remarkably consistent effect on emotions generally. Specifically, extroversion has repeatedly been found to be associated with positive affective states, and neuroticism (a trait variable defined as the tendency to experience distress), not surprisingly, has been associated with negative affective states (Costa and McCrae, 1980; Watson, Clark and Tellegen, 1988). Mooradian and Olver (1997) extended these findings by examining the relationship between these two traits and the experience of consumption emotions. Consistent with the research in psychology concerning more general affective states, they found a positive relationship between extroversion and positive consumption emotions (r ⫽ ␾.23), while neuroticism was associated with negative consumption emotions (r ⫽ ␾.44).

Consumption emotions

409

Other research has gone beyond personality to examine the relationship between temperament and the experience of emotion. Larsen (1984) identified emotionality as one of the four fundamental dimensions of temperament, and subsequent research has found that individuals can be reliably identified in terms of affect intensity – the strength of the emotions with which a person typically responds to affect laden stimuli or situations (Larsen and Diener, 1987). Although emotionality and affect intensity have not been studied in the context of consumption emotions, a study by Moore and Homer (2000) found that affect intensity was associated with the strength of emotional response to advertising that contained emotional content. It is plausible that consumers with temperaments characterized by affect intensity would also have more intense consumption emotions in those consumption situations that arouse an affective response. Finally, two studies have attempted to link personal values to consumption emotions. Laverie, Kleine and Kleine (1993) speculated that personal values would influence the kind of consumption experiences consumers seek, and that this biased selection of consumption experiences would influence the kinds of consumption emotions experienced. A small study by Richins, McKeage and Najjar (1992) found that materialism, a personal value, was weakly related to consumption emotions, such that those high in materialism felt negative consumption emotions more strongly than those low in materialism. There was no difference in the experience of positive emotion intensity between those low and high in materialism.

5. RESEARCH ON CONSUMPTION EMOTIONS So far, this chapter has examined the influences on consumption emotions. This section describes in more detail the manner in which consumption emotions have been studied by researchers. Studies of consumption emotions fall into two categories: Those that examine a single emotion in depth, and those that attempt to assess more broadly the range of emotions experienced in particular consumption situations.

5.1. Single emotion studies Studies that examine a particular emotion in considerable depth are common in the psychology literature, including studies of love (Fehr and Russell, 1991), anger (Averill 1983), shame and pride (Tracy and Robins, 2004), and disappointment (Zeelenberg et al., 1998). However, only a few of the single emotion studies focus on emotions in the consumption context. Perhaps the most extensively studied consumption emotion is regret, which has been defined as ‘the painful sensation of recognizing that “what is” compares unfavorably with “what might have been”’ (Sugden, 1985, p. 77). Consumer decision scholars have been particularly interested in understanding the conditions that give rise to regret (e.g. Inman and Zeelenberg, 2002), the strategies consumers use to avoid regret (e.g. Cooke, Meyvis and Schwartz, 2001), and the consequences of regret (e.g. Tsiros and Mittal, 2000). The topic of guilt has also received some attention in the consumer behavior literature. Most of this research has been conducted in the context of guilt-inducing persuasion attempts. However, a thoughtful and extensive analysis of guilt in the consumption context was carried out by Dahl, Honea and Manchanda (2003), who employed a critical incidents methodology to determine the nature of guilt and the actions consumers subsequently undertook to cope with these feelings.

410

Product Experience

Finally, a study by Yi and Baumgartner (2004) looks specifically at a small set of negative consumption emotions (anger, disappointment, regret, and worry) and examines the strategies that consumers use to cope with these emotions.

5.2. Broad range studies Studies that assess consumption emotions more broadly have appeared more frequently in the consumer literature; many of these studies are shown in Table 16.1 and are discussed throughout this chapter. These more broadly focused studies generally find that consumers report more positive than negative consumption emotions. Beyond this general observation, however, it is difficult to draw more definitive conclusions about consumption emotions from this research, because the nature of emotions experienced varies widely, depending on the consumption situation and the nuances of the appraisals made by individual consumers. Another difficulty when trying to draw generalizations from this literature is that the kinds of emotions found to be associated with consumption is necessarily dependent on the measurement tool used to assess these emotions. Qualitative measures rely on consumers to recall and report the emotions they experienced in a particular consumption situation. Examples of studies using this approach are Derbaix and Pham’s (1991) study of the relationship between situation and consumption emotion and Schultz, Kleine and Kernan’s (1989) examination of consumption emotions and possession attachment (see also Table 16.1). While qualitative and open-ended measures have the advantage of allowing respondents to report their emotions and experiences in their own words and in some depth, these measures have some limitations. Responses in these studies are dependent on the participants’ motivation to provide detailed responses and their ability to articulate their feelings. In free recall situations like this, respondents are also prone to forget details or nuances of their consumption reactions. In my own research, I have found that consumers have difficulty in articulating the emotions they experience. In short interviews, people found it difficult to label their feelings beyond ‘felt great’, ‘made me feel good’, or ‘I felt bad’. In more extensive depth interviews, it was possible to distinguish more nuanced emotions, and consumers were more likely to use terms like ‘disappointed’, ‘relieved’, ‘mad’, ‘guilty’, ‘excited’, or ‘sad’ to describe their feelings associated with consumption. Qualitative studies are especially useful when investigating emotion responses in considerable depth and attempting to understand the complexities of specific emotions and the conditions that give rise to them. Because they are resource-intensive, they are less practical for studying large groups of consumers or simultaneously investigating multiple consumption emotions. To obtain a more detailed picture of consumption emotions, researchers frequently rely on emotion rating scales. The advantage of such scales is that they make the emotions vocabulary more accessible to respondents and can serve as an aid to recall, enabling consumers to remember and report their feelings in more detail. The disadvantage is that data obtained using this technique are sensitive to and dependent upon the specific emotions included in the measurement instrument. Instruments that omit a particular emotion that a consumer experienced will not be able to identify that the consumption emotion occurred. On the other hand, it isn’t feasible to include the entire list of emotions a consumer might feel in a single instrument, so studies using rating scales are forced to be selective in the emotions they assess. A number of scholars have addressed the problem of how to characterize and measure emotions. Those efforts that are particularly relevant to the measurement of consumption emotions are described in the following section.

Consumption emotions

TABLE 16.1

Studies measuring consumption emotions* Role of consumption emotions in the study

Product class studied

Types of affective reactions prompted by specific consumption situations

Havlena and Holbrook (1986); Havlena, Holbrook and Lehmann (1989)

Authors

Respondents

Measure used

Various

120 college students in western Europe

Free response written questionnaire

Evaluated the relative appropriateness of two different schemes for representing consumption emotions

Various

Convenience sample of 20 adults in US

Free response written questionnaire

Richins, McKeage and Najjar (1992), Study 1

Relationship between consumption emotions and materialism

Pleasurable consumption experiences

Convenience sample of 48 adults in US

Free response written questionnaire

Schultz, Kleine and Kernan (1989)

Emotions associated with weakly and strongly attached possessions

Various

95 college students in US

Free response written questionnaire

Ackerman, MacInnis and Folkes (2000)

Emotions generated by social comparison of possessions

Various

42 college students in US

Ad hoc measure of 11 emotions based on Burke and Edell (1989) and CES (Richins 1997)

Buck and Georgson (1997)

Examined the multidimensional space of consumption emotions

Various

94 college students in US

Ad hoc measure of 53 emotions based on Shaver et al. (1987)

Chaudhuri (1997)

Emotions associated with perceived risk

Various

4380 adults in US

Ad hoc measure of 5 basic emotion clusters identified by Shaver et al. (1987)

Gardner and Rook (1988)

Affective states associated with impulse purchases

Various

155 adults in US

Ad hoc measure of 13 affective states

Holbrook et al. (1984)

Determinants of emotional responses to games

Video games

60 MBA students in US

PAD (Mehrabian and Russell 1974)

Holbrook and Gardner (1993)

Effect of emotions on duration of listening to jazz recordings

Music

58 subjects in US

Affect Grid (Russell et al. 1989)

Qualitative studies Derbaix and Pham (1991)

Rating scale studies

(Continued)

411

412

TABLE 16.1 (Continued) Product class studied

Respondents

Measure used

Laros and Steenkamp (2005)

Developed hierarchical model of consumption emotions

Food items

645 Dutch consumers

Ad hoc measure of 4 negative and 2 positive basic emotions

Laverie, Kleine and Kleine (1993)

Relationships between personal values and consumption emotions

Various

131 college students in US

DES-II (Izard 1977)

Mano and Elliott (1997)

Emotions experienced as a result of being a smart shopper

Various

228 college students in US

Ad hoc measure based on PANAS (Watson et al. 1988) and Mano (1991)

Mano and Oliver (1993)

Explored dimensionality of consumption emotions and their relationship to satisfaction

Various

118 college students in US

Ad hoc measure based on PANAS (Watson et al. 1988) and Mano (1991)

Mehrabian and Wixen (1996)

Related emotional response to product preference

Video games

58 college students in US

PAD dimensions (Mehrabian 1978)

Mooradian and Olver (1997)

Relationship between extroversion and neuroticism and consumption emotions

Automobiles

193 adults in US

PANAS (Watson et al. 1988), adapted to measure consumption emotions

Nyer (1997)

Relationship between appraisals and consumption emotions

Computer system 164 college students in US

Ad hoc measure of 4 emotion types

Oliver (1993)

Relationship between consumption emotions and satisfaction

Automobiles, college course

125 adults in US, 178 college students in US

DES (Izard 1977)

Phillips and Baumgartner (2002)

Consumption emotions associated with satisfaction

Orange juice

55 college students in US in each of two studies

23-item measure based on Burke and Edell (1989)

Richins (1997)

Developed CES to measure consumption emotions

Various

6 studies with sample sizes ranging from 97 to 448, college student and adult consumers in US

CES, DES-II, Havlena and Holbrook’s (1986) measure, PAD (Mehrabian and Russell 1974), advertising emotion scales (Batra and Holbrook 1990; Edell and Burke 1987)

Richins, McKeage and Najjar (1992), Study 2

Relationship between consumption emotions and materialism

Various

107 adults in US

Ad hoc list of 24 emotion descriptors

Product Experience

Role of consumption emotions in the study

Authors

Relationship between appraisals and consumption emotions

Gifts received

198 college students and staff in US, 122 adults in US 200 automobile owners and 154 cable television subscribers in US

Ad hoc set of 10 emotions expected to relate to gift receipt

Westbrook (1987)

Relationship between consumption emotions and satisfaction, complaint behavior, and word-of-mouth

Automobiles, cable television

Westbrook and Oliver (1991)

Relationship between consumption emotions and satisfaction

Automobiles

125 adults in US

DES-II (Izard 1977)

Wood and Moreau (2006)

How consumption emotions influence evaluation and use of innovative products

Personal digital assistant; computer software

175 college students in US; 106 college students in US

Modified DES-II (Izard 1977)

Yi and Baumgartner (2004)

Relationship between negative consumption emotions and coping strategies

Various

124 college students in US, 78 female college staff in US

Ad hoc measure of 4 negative emotion categories

DES-II (Izard 1977)

Consumption emotions

Ruth, Brunel and Otnes (2002)

*Restricted to studies of emotions experienced during the consumption of goods; consumption of services is outside the scope of this chapter.

413

414

Product Experience

6. IDENTIFYING AND MEASURING CONSUMPTION EMOTIONS Most languages contain hundreds of emotion descriptors in their lexicons (e.g. Averill, 1975). We all readily recognize feelings like fear, resentment, guilt, contentment, pride, and the like. A number of scholars have attempted to create some order among the hundreds of emotion terms by identifying the basic categories of emotion (e.g. Ekman and Friesen, 1975; Izard, 1977; Plutchik, 1980; see Ortony and Turner, 1990, for a review) or the dimensions along which emotions can be represented (e.g. Daly, Lancee and Polivy, 1983; Russell, 1980). Others have examined the relationships among the hundreds of emotion terms by examining their hierarchical structure. Two relatively comprehensive studies that examined knowledge and semantic structures among emotion terms, using different methods, both identified a three level hierarchical structure for emotions (Shaver et al., 1987; Storm and Storm, 1987). In these studies, the superordinate level divides emotion terms into two groups, representing positive and negative emotions. At the intermediate level are categories (such as fear, anger, joy, etc.) that represent clusters of related emotion terms, and the specific terms themselves constitute the subordinate level of the hierarchy. Figure 16.1 shows the hierarchical arrangement derived by Shaver et al. (1987). While several researchers have found evidence for a three-level hierarchical structure of emotions and tend to agree that the positive and negative division of emotions constitute the superordinate level, there is less agreement about the identity of the emotion clusters at the intermediate level. Most recently, Laros and Steenkamp (2005) applied a hierarchical approach to the study of consumption emotions generated by food products and placed happiness, contentment, shame, sadness, fear, and anger at the intermediate level of their hierarchy (they also proposed that love and pride are intermediate level emotions, but excluded them from the empirical portion of their study because of their low relevance to the product category).

Positive emotions

Love

tenderness love passion etc.

Negative emotions

Joy

Anger

happiness excitement contentment hope etc.

frustration anger disgust envy etc.

Sadness

unhappiness disappointment shame loneliness etc.

Fear

worry fear panic etc.

FIGURE 16.1 Hierarchical semantic structure of emotions. Adapted from Shaver et al. (1987).

Consumption emotions

415

In the measurement of consumption emotions a central problem is to identify which of the hundreds of emotion descriptors in a language represent emotions that are elicited by consumption, and to determine the appropriate way to measure these consumption emotions. In the face of considerable ambiguity concerning this question, some researchers have chosen to assume that the exact nature of the emotion experienced is not as important as the valence of the emotion. These researchers have conducted their analyses on two aggregate consumption emotion measures: One reflecting the sum (or average) of ratings on positive emotion descriptors and the other reflecting aggregate ratings on negative emotion descriptors (e.g. Chaudhuri, 1997; Mooradian and Olver, 1997; Wood and Moreau, 2006). The disadvantage of this approach, however, is that it misses important distinctions among types of negative emotion and among types of positive emotion, each of which may have different implications for consumer behavior (Lerner and Keltner, 2000). Other researchers have dealt with this ambiguity by developing their own ad hoc measures to assess the emotions they consider to be most relevant to the consumption situation under investigation (see Table 16.1). However, many researchers have chosen to measure consumption emotions with established scales that have been developed to measure emotions in broader contexts than consumption. The relative merits of these scales and their application in the marketing literature have been reviewed by Laros and Steenkamp (2005) and by Richins (1997). Three bear special mention in this chapter on consumption emotions.

6.1. Differential Emotions Scale Among established measures, the Differential Emotions Scale, or DES (Izard, 1977), has had the most widespread use, particularly in early studies of consumption emotions (e.g. Oliver, 1993; Westbrook, 1987). Based in part on the identification of emotions that are universally associated with and recognizable in distinctive facial expressions, Izard (1977) proposed 10 fundamental emotions: Interest, enjoyment, surprise, distress (sadness), anger, disgust, contempt, fear, shame/shyness, and guilt. The DES-II contains 30 adjectives to measure these emotions. A concern with the DES-II scale is the predominance of negative emotions in Izard’s framework, and the need to provide a broader sampling of emotions that may occur in the consumption environment.

6.2. Pleasure-arousal-dominance dimensional representation Another approach used in consumption emotion research is based on the work of Mehrabian and Russell (1974), who proposed that emotional responses to the environment can be mapped onto three dimensions: Pleasure; arousal; and dominance. The PAD measure developed by these authors consists of 18 semantic differential items that measure these three dimensions. The Affect Grid developed by Russell, Weiss and Mendelsohn (1989) is another approach to measuring pleasure, arousal, and dominance. Unlike other emotion measures, the PAD and Affect Grid do not purport to measure emotions per se, but instead measure the extent to which the three dimensions are activated in a particular situation. The existence of specific emotion states cannot be unequivocally inferred from PAD scores.

6.3. Consumption Emotions Set In response to limitations of existing measures, Richins (1997) developed the Consumption Emotions Set (CES). In a series of six studies, this research identified the range of emotions

416

Product Experience

TABLE 16.2 The consumption emotions set Anger • frustrated • angry • irritated Discontent • unfulfilled • discontented Worry • nervous • worried • tense

• passionate Love • loving • sentimental • warm-hearted Peacefulness • calm • peaceful Contentment • contented • fulfilled

Sadness • depressed • sad • miserable

Optimism • optimistic • encouraged • hopeful

Fear • scared • afraid • panicky

Joy • happy • pleased • joyful

Shame • embarrassed • ashamed • humiliated

Excitement • excited • thrilled • enthusiastic

Envy • envious • jealous

Surprise • surprised • amazed • astonished

Loneliness • lonely • homesick Romantic love • sexy • romantic

Other items • guilty • proud • eager • relieved

typically encountered in consumption contexts and the language consumers use to identify their emotional states. The resulting Consumption Emotions Set consists of the 47 emotion descriptors shown in Table 16.2. Rather than being a definitive measure of consumption emotions, this set is intended to provide a pool of items for researchers to draw upon when measuring consumption emotions. Researchers can be confident that the items in the set represent terminology familiar to consumers. And while the set, collectively, spans the entire multidimensional space of emotions (see Figure 16.2), researchers can choose to use items that tap those areas of the space most relevant to their purposes in a research study without needing to include the entire measure. Like other emotion measures, the CES can be used to measure pre-purchase emotions and shopping emotions, as well as consumption emotions.

7. FUTURE DIRECTIONS This chapter has reviewed the nature of consumption emotions and described many of the factors that influence them at both the pre-purchase and post-purchase stages.

417

Consumption emotions

2.5 2 1.5 t

u

Receptive/Active

1 r

p

0.5

s

n

k

0

E F

j

l

G H K J

i h

g

⫺0.5

f

⫺1

C

v

q o

B

A

d

e c b

Z Y a

X

D

I

LM O N P Q SR T UV W

⫺1.5 ⫺2 ⫺2.5 ⫺2.5

⫺2

⫺1.5

⫺1 ⫺0.5 0 0.5 1 Positive/Negative valence

1.5

2

2.5

FIGURE 16.2 Multidimensional space of consumption emotions, as measured by the Consumption Emotion Set. Note: Starting at the upper middle portion of the figure and continuing in a clockwise direction, letters are as follows: A, impatient; B, frustrated; C, irritated; D, angry; E, unfulfilled; F, discontented; G, worried; H, tense; I, disgusted; J, furious; K, grouchy; L, depressed; M, miserable; N, sad; O, panicky; P, threatened; Q, afraid; R, ashamed; S, embarrassed; T, guilty; U, envious; V, jealous; W, lonely; X, homesick; Y, tender; Z, sexy; a, romantic; b, loving; c, sentimental; d, warmhearted; e, calm; f, peaceful; g, comforted; h, relieved; i, hopeful; j, optimistic; k, contented; l, fulfilled; m, proud; n, joyful; o, glad; p, pleased; q, enthusiastic; r, excited; s, eager; t, amazed; u, surprised; v, overwhelmed (from Richins, 1997).

Although considerable research has investigated these emotions, there are many unknowns, and additional research remains to be done to understand this important determinant and consequence of consumer behavior. Some suggestions for further investigation are provided below.

7.1. Theory building research Consumption emotions are interesting to scholars because they are fundamental to understanding purchase motivations and consumption behaviors. However, our understanding of these emotions and their effects is quite fragmented, and research along the following lines could do much to advance knowledge. A very large gap exists in our understanding of the variables that influence consumption emotion processes and that influence the nature of the consumption emotions experienced. Although the role of appraisals in emotion formation are reasonably wellunderstood (Roseman, Antoniou and Jose, 1996; Silvia, 2005), research has not adequately investigated the conditions antecedent to appraisals. One topic that has received almost no research is that concerning consumption hypotheses. It would be useful to know in more detail the factors that influence the creation of these hypotheses, their content, the strength with which they are held, and the influences of these hypotheses on post-acquisition emotions.

418

Product Experience

Another large unknown is the role of personal and situation variables on consumption emotions. In the realm of personal variables, both enduring characteristics such as personality traits and more temporary conditions such as mood or energy level could be studied. Situational variables such as the presence of others, the complexity of the sensory environment in which the consumption takes place, and the consumer’s past experiences with similar situations also bear further investigation. The effects of consumption emotions on various outcome variables also is not well understood. While it has been shown that consumption emotions influence word of mouth (Westbrook, 1987), there has been little investigation of other possible outcomes such as repeat purchase behavior, product attachment, and product disposition decisions. Another promising area for research concerns the duration of consumption emotions, both positive and negative. How long does it take for negative consumption emotions to dissipate? How can positive consumption emotions be sustained? Some products are purchased with the hope that positive emotions will be generated each time the product is used (a new car, for instance, or a pair of skis). However, these positive consumption emotions usually aren’t sustained indefinitely. Marketers may be interested in studying how consumers can be induced to make product replacement purchases to recapture the emotional buzz associated with acquisition. Environmentalists may be more interested in helping consumers learn how to sustain positive consumption emotions or how to achieve positive emotional states in venues other than purchase and excessive consumption.

7.2. Managerially oriented research In addition to some of the issues described in the preceding section, managers may have some very pragmatic interests in better understanding consumption emotions. For instance, effective advertising for some products may involve convincing consumers that a particular product will produce desired emotional states more effectively than alternatives. Thus, it would seem very useful to identify the consumption emotion profiles for particular products, with an eye toward examining both the experienced emotions and the desired emotions. Richins (1997) has already demonstrated that the CES can be used to determine the emotional profile of a product class by showing that automobiles, recreation goods, and sentimental items could be distinguished by the consumption emotions they elicit. Developing such emotion profiles at the product specific level and for different market segments would be even more useful. Finally, from a managerial perspective, more knowledge about the relationship between consumption emotions and satisfaction may be helpful in building brand equity and brand loyalty. Although the relationship between consumption emotions and satisfaction has been addressed at a general level in a number of studies (see Table 16.1), it may be necessary for managers to examine these relationships more specifically by product class or even by brand to enhance their ability to generate especially high levels of satisfaction and loyalty.

REFERENCES Ackerman, D., MacInnis, D. and Folkes, V. (2000). Social comparisons of possessions: When it feels good and when it feels bad. In: S. J. Hoch and R. J. Meyer (Eds.) Advances in consumer research, Vol. 27, pp. 173–178. Provo, UT: Association for Consumer Research.

Consumption emotions

419

Appley, M. H. (Ed.) (1971). Adaptation-level theory: A symposium. New York: Academic Press. Averill, J. R. (1975). A semantic atlas of emotional concepts. JSAS: Catalog of Selected Documents in Psychology, 5, 330. (Ms. No. 421). Averill, J. R. (1983). Studies on anger and aggression: Implications for theories of emotion. American Psychologist, 38, 1145–1160. Bagozzi, R. P., Baumgartner, H., Pieters, R. and Zeelenberg, M. (2000). The role of emotions in goal-directed behavior. In: S. Ratneshwar, D. G. Mick and C. Huffman (Eds.) The why of consumption: Contemporary perspectives on consumer motives, goals, and desires, pp. 36–58. London: Routledge. Bargh, J. A. and Chartrand, T. L. (1999). The unbearable automaticity of being. American Psychologist, 54, 462–479. Batra, R. and Holbrook, M. B. (1990). Developing a typology of affective responses to advertising. Psychology and Marketing, 7, 11–25. Belk, R. W., Ger, G. and Askegaard, S. (2003). The fire of desire: A multi-sited inquiry into consumer passion. Journal of Consumer Research, 30, 326–351. Bloch, P. H. (1986). Product enthusiasm: Many questions, a few answers. In: R. J. Lutz (Ed.) Advances in consumer research, Vol. 13, pp. 539–543. Provo, UT: Association for Consumer Research. Bloch, P. H. and Richins, M. L. (1983). A theoretical model for the study of product importance perceptions. Journal of Marketing, 47, 69–81. Buck, R. and Georgson, M. (1997). Consumer emotion space: An investigation of semantic space and context effects in self-reported emotion elicitation. In: M. Brucks and D. J. MacInnis (Eds.) Advances in consumer research, Vol. 24, pp. 431–437. Provo, UT: Association for Consumer Research. Burke, M. C. and Edell, J. A. (1989). The impact of feelings on ad-based affect and cognition. Journal of Marketing Research, 26, 69–83. Campbell, C. (1987). The romantic ethic and the spirit of modern consumerism. New York: Basil Blackwell. Chaudhuri, A. (1997). Consumption emotion and perceived risk: A macro-analytic approach. Journal of Business Research, 39, 81–92. Cooke, A. D. J., Meyvis, T. and Schwartz, A. (2001). Avoiding future regret in purchase-timing decisions. Journal of Consumer Research, 27, 447–459. Costa, P. T., Jr. and McCrae, R. R. (1992). Influence of extraversion and neuroticism on subjective well-being: Happy and unhappy people. Journal of Personality and Social Psychology, 38, 668–678. Dahl, D. W., Honea, H. and Manchanda, R. V. (2003). The nature of self-reported guilt in consumption contexts. Marketing Letters, 14, 159–171. Daly, E. M., Lancee, W. J. and Polivy, J. (1983). A conical model for the taxonomy of emotional experience. Journal of Personality and Social Psychology, 45, 443–457. Day, R. L. (1984). Modeling choices among alternative responses to dissatisfaction. In: T. C. Kinnear (Ed.) Advances in consumer research, Vol. 11, pp. 496–499. Provo, UT: Association for Consumer Research. de Mello, G. E. and MacInnis, D. J. (2005). Why and how consumers hope: Motivated reasoning and the marketplace. In: S. Ratneshwar and D. G. Mick (Eds.) Inside consumption: Consumer motives, goals, and desires, pp. 44–66. London: Routledge. Derbaix, C. and Pham, M. T. (1991). Affective reactions to consumption situations: A pilot investigation. Journal of Economic Psychology, 12, 325–355. Ekman, P. and Friesen, W. V. (1975). Unmasking the face. Englewood Cliffs, NJ: Prentice-Hall. Fehr, B. and Russell, J. M. (1991). The concept of love viewed from a prototype perspective. Journal of Personality and Social Psychology, 60, 425–438. Festinger, L. (1954). A theory of social comparison processes. Human Relations, 7, 117–140. Fournier, S. and Guiry, M. (1993). ‘An emerald green Jaguar, a house on Nantucket, and an African safari’: Wish lists and consumption dreams in materialist society. In: L. MacAlister and M. L. Rothschild (Eds.) Advances in consumer research, Vol. 20, pp. 352–358. Provo, UT: Association for Consumer Research. Gardner, M. P. and Rook, D. W. (1988). Effects of impulse purchases on consumers’ affective states. In: M. J. Houston (Ed.) Advances in consumer research, Vol. 15, pp. 127–130. Provo, UT: Association for Consumer Research. Havlena, W. J. and Holbrook, M. B. (1986). The varieties of consumption experience: Comparing two typologies of emotion in consumer behavior. Journal of Consumer Research, 13, 394–404. Havlena, W. J., Holbrook, M. B. and Lehmann, D. R. (1989). Assessing the validity of emotional typologies. Psychology and Marketing, 6, 97–112. Helson, H. (1964). Current trends and issues in adaptation-level theory. American Psychologist, 19, 26–38. Holak, S. L. and Havlena, W. J. (1992). Nostalgia: An exploratory study of themes and emotions in the nostalgic experience. In: J. F. Sherry, Jr. and B. Sternthal (Eds.) Advances in consumer research, Vol. 19, pp. 380–387. Provo, UT: Association for Consumer Research.

420

Product Experience

Holbrook, M. B., Chestnut, R. W., Oliva, T. A. and Greenleaf, E. A. (1984). Play as a consumption experience: The roles of emotions, performance, and personality in the enjoyment of games. Journal of Consumer Research, 11, 728–739. Holbrook, M. B. and Gardner, M. P. (1993). An approach to investigating the emotional determinants of consumption durations: Why do people consume what they consume for as long as they consume it? Journal of Consumer Psychology, 2, 123–142. Holbrook, M. B. and Hirschman, E. C. (1982). The experiential aspects of consumption: Consumer fantasies, feelings, and fun. Journal of Consumer Research, 9, 132–140. Inman, J. J. and Zeelenberg, M. (2002). Regret in repeat purchase versus switching decisions: The attenuating role of decision justifiability. Journal of Consumer Research, 29, 116–128. Izard, C. E. (1977). Human emotions. New York: Plenum Press. Klaaren, K. J., Hodges, S. D. and Wilson, T. D. (1994). The role of affective expectations in subjective experience and decision making. Social Cognition, 12, 77–101. Laros, F. J. M. and Steenkamp, J.-B. E. M. (2005). Emotions in consumer behavior: A hierarchical approach. Journal of Business Research, 58, 1437–1445. Larsen, R. J. (1984). Theory and measurement of affect intensity as an individual difference characteristic. Dissertation Abstracts International, 85, 2297B. (University Microfilms No. 84-22112). Larsen, R. J. and Diener, E. (1987). Affect intensity as an individual difference characteristic: A review. Journal of Research in Personality, 21, 1–39. Laverie, D. A., Kleine, R. F. III and Kleine, S. S. (1993). Linking emotions and values in consumption experiences: An exploratory study. In: L. McAlister and M. L. Rothschild (Eds.) Advances in consumer research, Vol. 20, pp. 70–75. Provo, UT: Association for Consumer Research. Lazarus, R. S. (1991). Emotions and adaptation. Oxford: Oxford University Press. Lerner, J. S. and Keltner, D. (2000). Beyond valence: Toward a model of emotion-specific influences on judgment and choice. Cognition and Emotion, 14, 473–493. Levine, L. J. and Bluck, S. (2004). How emotions fade: Valence, appraisals, and the emotional impact of remembered events. In: S. P. Shohov (Ed.) Advances in Psychology Research, Vol. 30, pp. 3–20. Hauppauge, NY: Nova Science Publishers. Luce, M. F. (1998). Choosing to avoid: Coping with negatively emotion-laden consumer decisions. Journal of Consumer Research, 24, 409–433. MacInnis, D. J. and de Mello, G. E. (2005). The concept of hope and its relevance to product evaluation and choice. Journal of Marketing, 69, 1–14. Mano, H. (1991). The structure and intensity of emotional experiences: Method and context convergence. Multivariate Behavioral Research, 26, 389–411. Mano, H. and Elliott, M. T. (1997). Smart shopping: The origins and consequences of price savings. In: M. Brucks and D. J. MacInnis (Eds.) Advances in consumer research, Vol. 24, pp. 504–510. Provo: UT: Association for Consumer Research. Mano, H. and Oliver, R. L. (1993). Assessing the dimensionality and structure of the consumption experience: Evaluation, feeling, and satisfaction. Journal of Consumer Research, 20, 451–466. McDonagh, D., Hekkert, P., van Erp, J. and Gyi, D. (Eds.) (2004). Design and emotion: The experience of everyday things. London: Taylor and Francis. Mehrabian, A. (1978). Measures of individual differences in temperament. Educational and Psychological Measurement, 38, 1105–1117. Mehrabian, A. and Russell, J. A. (1974). An approach to environmental psychology. Cambridge, MA: MIT Press. Mehrabian, A. and Wixen, W. J. (1986). Preferences for individual video games as a function of their emotional effects on players. Journal of Applied Social Psychology, 16, 3–15. Mooradian, T. A. and Olver, J. M. (1997). ‘I can’t get no satisfaction’: The impact of personality and emotion on postpurchase processes. Psychology and Marketing, 14, 379–393. Moore, D. J. and Homer, P. M. (2000). Dimensions of temperament: Affect intensity and consumer lifestyles. Journal of Consumer Psychology, 9(4), 231–242. Nyer, P. U. (1997). A study of the relationships between cognitive appraisals and consumption emotions. Journal of the Academy of Marketing Sciences, 25, 296–304. Oliver, R. L. (1993). Cognitive, affective, and attribute bases of the satisfaction response. Journal of Consumer Research, 20, 418–430. Oliver, R. L. (1997). Satisfaction: A behavioral perspective on the consumer. New York: McGraw-Hill. Ortony, A., Clore, G. L. and Collins, A. (1988). The cognitive structure of emotions. Cambridge: Cambridge University Press. Ortony, A. and Turner, T. J. (1990). What’s basic about basic emotions? Psychological Review, 97, 315–331.

Consumption emotions

421

O’Shaughnessy, J. and O’Shaughnessy, N. J. (2003). The marketing power of emotion. Oxford: Oxford University Press. Otnes, C., Lowrey, T. M. and Shrum, L. J. (1997). Toward an understanding of consumer ambivalence. Journal of Consumer Research, 24, 80–93. Phillips, D. M. and Baumgartner, H. (2002). The role of consumptions emotions in the satisfaction response. Journal of Consumer Psychology, 12, 243–252. Phillips, D. M., Olson, J. C. and Baumgartner, H. (1995). Consumption visions in consumer decision making. In: F. R. Kardes and M. Sujan (Eds.) Advances in consumer research, Vol. 22, pp. 280–284. Provo, UT: Association for Consumer Research. Plutchik, R. (1980). Emotion: A psychoevolutionary synthesis. New York: Harper and Row. Pullman, M. E. and Gross, M. A. (2004). Ability of experience design elements to elicit emotions and loyalty behaviors. Decision Sciences, 35, 551–578. Richins, M. L. (1997). Measuring emotions in the consumption experience. Journal of Consumer Research, 24, 127–146. Richins, M. L. and Bloch, P. H. (1986). After the new wears off: The temporal context of product involvement. Journal of Consumer Research, 13, 280–285. Richins, M. L., McKeage, K. K. R. and Najjar, D. (1992). An exploration of materialism and consumption-related affect. In: J. F. Sherry, Jr. and B. Sternthal (Eds.) Advances in consumer research, Vol. 19, pp. 229–236. Provo, UT: Association for Consumer Research. Roseman, I. J., Antoniou, A. A. and Jose, P. E. (1996). Appraisal determinants of emotions: Constructing a more accurate and comprehensive theory. Cognition and Emotion, 10, 241–277. Roster, C. A. and Richins, M. L. (2006). Ambivalence and attitudes in consumer replacement decisions. Working paper. Russell, J. A. (1980). A circumplex model of affect. Journal of Personality and Social Psychology, 39, 1161–1178. Russell, J. A., Weiss, A. and Mendelsohn, G. A. (1989). Affect grid: A single-item scale of pleasure and arousal. Journal of Personality and Social Psychology, 57, 493–502. Rust, R. T. and Oliver, R. L. (2000). Should we delight the customer? Journal of the Academy of Marketing Science, 28, 86–94. Ruth, J. A., Brunel, F. F. and Otnes, C. C. (2002). Linking thoughts to feelings: Investigating cognitive appraisals and consumption emotion in a mixed-emotions context. Journal of the Academy of Marketing Sciences, 30, 44–58. Salovey, P. and Rodin, J. (1991). Provoking jealousy and envy: Domain relevance and self-esteem threat. Journal of Social and Clinical Psychology, 10, 395–413. Schindler, R. M. (1998). Consequences of perceiving oneself as responsible for obtaining a discount: Evidence for smart shopper feelings. Journal of Consumer Psychology, 7, 371–392. Schultz, S. E., Robert E. Kleine, I. and Kernan, J. B. (1989). ‘These are a few of my favorite things’: Toward an explication of attachment as a consumer behavior construct. In: T. K. Srull (Ed.) Advances in consumer research, Vol. 16, pp. 359–366. Provo, UT: Association for Consumer Research. Shaver, P., Schwartz, J., Kirson, D. and O’Connor, C. (1987). Emotion knowledge: Further exploration of a prototype approach. Journal of Personality and Social Psychology, 52, 1061–1086. Sheller, M. (2004). Automotive emotions – feeling the car. Theory, Culture and Society, 21, 221–241. Sherif, M. and Hovland, C. I. (1965). Social judgment: Assimilation and contrast effects in communication and attitude change. New Haven, CT: Yale University Press. Silvia, P. J. (2005). Emotional responses to art: From collation and arousal to cognition and emotion. Review of General Psychology, 9, 342–357. Smith, C. A. (1989). Dimensions of appraisal and physiological response in emotion. Journal of Personality and Social Psychology, 56, 339–353. Snyder, C. R. (1989). Reality negotiation – from excuses to hope and beyond. Journal of Social and Clinical Psychology, 8, 130–157. St. James, Y. and Taylor, S. (2004). Delight-as-magic: Refining the conceptual domain of customer delight. In: B. E. Kahn and M. F. Luce (Eds.) Advances in consumer research, Vol. 31, pp. 753–758. Valdosta, GA: Association for Consumer Research. Storm, C. and Storm, T. (1987). A taxonomic study of the vocabulary of emotions. Journal of Personality and Social Psychology, 53, 805–816. Sugden, R. (1985). Regret, recrimination and rationality. Theory and Decision, 19, 77–99. Tracy, J. L. and Robins, R. W. (2004). Putting the self into self-conscious emotions: A theoretical model. Psychological Inquiry, 15, 103–125. Tsiros, M. and Mittal, V. (2000). Regret: A model of its antecedents and consequences in consumer decision making. Journal of Consumer Research, 26, 401–417.

422

Product Experience

Watson, D., Clark, L. A. and Tellegen, A. (1988). Development and validation of brief measures of positive and negative affect: The PANAS scales. Journal of Personality and Social Psychology, 54, 1063–1070. Westbook, R. A. (1987). Product/consumption-based affective responses and postpurchase processes. Journal of Marketing Research, 24, 258–270. Westbrook, R. A. and Oliver, R. L. (1991). The dimensionality of consumption emotion patterns and consumer satisfaction. Journal of Consumer Research, 18, 84–91. Wilson, T. D., Lisle, D. J., Kraft, D. and Wetzel, C. G. (1989). Preferences as expectation-driven inferences: Effects of affective expectations on affective experience. Journal of Personality and Social Psychology, 56, 519–530. Wood, S. L. and Moreau, P. (2006). From fear to loathing? How emotion influences the evaluation and early use of innovations. Journal of Marketing, 70, 44–57. Yi, S. and Baumgartner, H. (2004). Coping with negative emotions in purchase-related situations. Journal of Consumer Psychology, 14, 303–317. Zeelenberg, M., van Dijk, W. W., Manstead, A. S. R. and van der Pligt, J. (1998). The experience of regret and disappointment. Cognition and Emotion, 12, 221–230.

17

PRODUCT ATTACHMENT: DESIGN STRATEGIES TO STIMULATE THE EMOTIONAL BONDING TO PRODUCTS RUTH MUGGE Delft University of Technology, Delft, The Netherlands

JAN P.L. SCHOORMANS Delft University of Technology, Delft, The Netherlands

HENDRIK N.J. SCHIFFERSTEIN Delft University of Technology, Delft, The Netherlands

Three years ago, the first author bought a brand new convertible, a Citroën C3 Pluriel (see Figure 17.1). Since she owns it, the car has shown several defects: The roof leaked several times, the brakes creaked, the window was dislodged, and she had problems with the battery and the gears. Also, it is not a very user-friendly car, as she repeatedly has had difficulties in removing the roof of the car. Due to these issues, she paid many visits to the garage. But, does she regret that she purchased her Citroën C3 Pluriel? No, on the contrary, she loves her car! She loves it for its beautiful, extraordinary design and its eye-catching, green color. She loves it for the fun and relaxation it provides her when she drives it with the top down during summer. She loves it, for the fact that only few people own an identical car. And last but not least, she loves it, because it makes her smile when she sees it standing on the parking space after a long working day. As a result, her car has gained a special meaning to her and she feels attached to it despite the utilitarian issues. This chapter is about why people develop strong relationships to certain products and how designers may influence the degree of attachment through product design.

1. DEFINING PRODUCT ATTACHMENT In the literature on interpersonal relationships, it is proposed that an attachment is an emotion-laden target-specific bond between two persons (Bowlby, 1979). Correspondingly, product attachment is defined as the strength of the emotional bond a Product Experience Copyright © 2008 Elsevier Ltd.

425

426

Product Experience

FIGURE 17.1 The first author’s Citroën C3 Pluriel.

consumer experiences with a specific product1 (Schifferstein, Mugge and Hekkert, 2004). First of all, the definition of product attachment suggests that when experiencing attachment to a product, a strong relationship or tie exists between the individual on the one hand and the object on the other. Secondly, this definition implies that experiencing attachment to products is a matter of degree (Kleine and Baker, 2004; Schultz, Kleine and Kernan, 1989). People may experience relatively strong emotional bonds to their most favorite or special possessions, whereas other products are less significant to them. Thirdly, the definition implies that the object to which a person experiences attachment triggers one’s emotions. Schultz et al. (1989) investigated which emotions are elicited by products to which people are attached. In this study, a total of 83 different emotions were reported. Some of the most reported emotions were happiness, love, warmth, nostalgia, sadness, pride, security, comfort, excitement, and joy. Although a great deal of variety is present in the experienced emotions, people thus most often experience positive emotions to their objects of attachment. In contrast, products to which people do not experience attachment often do not elicit any emotions at all (Schultz, Kleine and Kernan, 1989). Also, negative emotions (e.g. boredom, frustration, and disgust) were mainly reported for objects to which people did not feel attached. An exception was the emotion sadness. Sadness may be elicited by products that are cherished for the memories associated with them. For example, a brooch that reminds someone of one’s deceased mother can simultaneously elicit both love and sadness. Although people usually experience positive emotions toward the product to which they feel attached, several arguments can be given why research on (positive) emotions is inadequate to understand the experience of attachment to a product. The occurrence of positive emotions is not sufficient to conclude that a person is attached to a product. 1 In all our studies, product attachment was measured using four items on seven-point Likert scales: (1) ‘I am very attached to this product’; (2) ‘I have a bond with this product’; (3) ‘This product is very dear to me’; and (4) ‘This product has no special meaning to me’ (reversed item).

Product attachment: Design strategies to stimulate the emotional bonding to products

427

Many products can instantaneously elicit strong positive emotions even without any direct contact with a product (Desmet, 2002). For example, a person can immediately feel excitement or joy based on a picture of a water tap, which can encourage him/her to purchase it for his/her new bathroom. However, these emotions may change radically into frustration and disappointment when the use of the water tap appears to be extremely complex and counterintuitive. Only if a product continues to elicit positive emotions over time, may the owner become attached to the product. The experience of attachment tends to develop bit by bit over the course of time, as a result of multiple, recurring interactions between an individual and the attachment object (Baldwin et al., 1996; Kleine and Baker, 2004; Thomson, MacInnis and Park, 2005). Typically, these recurring interactions occur during ownership of the product. For example, through possession rituals, such as using, displaying, cleaning, discussing, personalizing, and storing, a product may gradually accumulate personal meaning (McCracken, 1986). As a consequence, products to which one feels attached are generally considered to be special and significant to the owner. Another consequence of attachment is that it results in specific protective behaviors, because people cherish their relationship with the object and want to preserve the object (Mugge, Schoormans and Schifferstein, 2005). When a person feels attached to a product, he/she is more likely to handle the product with care, to repair it when it breaks down, and to postpone its replacement. Experiencing positive emotions in response to a product does not necessarily bring about these protective behaviors. Past research suggested that people become attached to a product for the personal and special meaning it conveys (Csikszentmihalyi and Rochberg-Halton, 1981; Wallendorf and Arnould, 1988). To obtain a personal and special meaning, a product should provide the owner with something exceptional over and above its utilitarian meaning (Mugge, Schoormans and Schifferstein, 2005). A product can have a utilitarian meaning, because it enables a person to fulfill a certain need. For example, a watch can show a person the correct time and a lamp can shine light. Most products within the same product category can provide this meaning. Accordingly, the product just functions according to expectations and does not provide anything special. In that case, a replacement decision is made relatively easily. Products to which people become attached provide a special meaning and, therefore, exceed their merely utilitarian meaning to the owner. For example, a watch may serve as a reminder of one’s father and a lamp may express a person’s identity. In these cases, the replacement of the product is much more difficult, because other products may not provide this special meaning to the owner. The product has ceased to be an ordinary object and has become extraordinary (Kleine and Baker, 2004). The former does not necessarily imply that a product needs to be expensive or rare to become an object of attachment. Ordinary objects may just as well elicit feelings of attachment, for instance, when the product is associated with an important memory. With respect to the construct of product attachment, we can distinguish between the experience of attachment to certain product variants or to specific product specimens (Schifferstein and Pelgrim, 2004). Being attached to a product variant implies that this specific type of product has a special meaning to the owner. In that case, the attachment will not only hold for this specific object, but also for other products of the same type that are physically identical. For example, a person may be attached to a Citroën C3 Pluriel (see Figure 17.1), because the car’s innovative and eye-catching design supports one’s identity. This special meaning is present in all physically identical Citroën C3 Pluriels, because they all have the same design. An identical-looking Citroën C3 Pluriel can thus also elicit feelings of attachment for this person. This does not mean that other variants are truly identical to the one that is owned. In time, most products show signs of use (e.g. stains or scratches). However, for the attachment to a product variant, it is

428

Product Experience

the overall design that all these specific product variants have in common that induces the special meaning, and not these personal signs of use. Being attached to a product specimen implies that the attachment concerns one particular object. Another physically identical product cannot completely replace such a product, because the context in which the object was obtained or used is inimitable. Accordingly, the special meaning cannot be present in other products, and the product is irreplaceable. ‘An irreplaceable possession is one that a consumer resists replacing, even with an exact replica, because the consumer feels that the replica cannot sustain the same meaning as the original’ (Grayson and Shulman, 2000, p. 17). For a product to become irreplaceable, the product’s meaning should have a factual connection with the object itself (Grayson and Shulman, 2000; Verbeek and Kockelkoren, 1997). The special meaning should be deeply anchored in that specific object, and the product and its meaning have to become inseparable. Because other products cannot provide this special meaning, a person will feel that replacing such a product results in a loss of the special meaning. For example, a person may be attached to one’s Citroën C3 Pluriel, because the car reminds him/her of all the pleasant trips made. This meaning is only present in this particular Citroën C3 Pluriel, because the trips were made in this product specimen. For the attachment to a product specimen, the signs of use on the product (e.g. stains or scratches) may be important for the product’s special meaning, because they may serve as proof for certain events.

2. RELEVANCE OF PRODUCT ATTACHMENT FOR DESIGNERS For designers the construct of product attachment is valuable from two perspectives. First, strengthening the emotional bond can help designers to create emotional experiences with products during ownership. Secondly, product attachment can serve as an eco-design strategy to create long-lasting person–product relationships.

2.1. Creating emotional experiences In today’s markets, most consumer durables are comparable with respect to their features, quality, and user-friendliness (Veryzer, 1995). This makes it difficult for companies to differentiate their products from competitors. To gain a competitive advantage, companies and designers are focusing more and more on the ‘emotional responses and experiences’ that products can bring about, rather than on their functional benefits. Figure 17.2 shows several advertisements of companies that suggest that people are attached to the advertised product. For example, the watches and jewellery company Breil uses the pay-off ‘Don’t touch my Breil’ in all their communications, suggesting a special caring for the object. The growing interest of scientific research, as well as design practice in the emotional impact of products, is also illustrated by the conferences and events that were organized on this topic over the past few years (e.g. Design and Emotion conference, Eternally Yours conference, Designing Pleasurable Products and Interfaces (DPPI) conference). Emotional responses to products can be a decisive factor in purchase decisions (e.g. Desmet, 2002; Jordan, 2000; Norman, 2004). Nevertheless, Desmet (2002) argued that studying emotional responses for a purchase situation may not be sufficient: ‘In the long run, it may be more fruitful to establish a long-term emotional relationship with the consumer’ (p. 187). Emotions enrich a person’s life and can increase one’s general experience of well-being (Diener and Lucas, 2000). Because part of a person’s day-to-day emotions are elicited by the products this person owns, designers need an understanding of the

Product attachment: Design strategies to stimulate the emotional bonding to products

429

FIGURE 17.2 Advertisements of Breil, Patek Philippe, Toyota Yaris, and Swatch that suggest the experience of product attachment in their pay-offs.

emotional impact of their designs over time. The construct of product attachment can be instrumental in achieving this goal.

2.2. Stimulating sustainable consumption From a sustainability perspective, the replacement of consumer durables is often undesirable. Many of the replaced durables eventually end up in the waste stream, which creates an environmental burden. In addition, replacing products requires the production of new consumer durables. Because scarce resources are used up during production, replacement also has an indirect detrimental effect on the environment. To reduce the negative environmental effects of consumers’ product replacement, scholars have proposed a strategy toward product longevity (Cooper, 1994; Von Weizsacker, Lovins and Lovins, 1997). Nowadays, product lifetime seems to be determined not only by technical constraints, for many products are replaced while they are still functioning properly (Van Nes, 2003). As a result, it is particularly interesting to lengthen the product lifetime by focusing on the product’s psychological lifetime: The time during which the user perceives the product to be valuable. A possible eco-design strategy to address the psychological lifetime of products is to strengthen the person–product relationship (Van Hemel and Brezet, 1997; Van Hinte, 1997; Van Nes, 2003). During the replacement process, a person abandons his/her relationship with a product in possession to be able to develop a new relationship with the replacement product (Roster, 2001). On the one hand, a person is attracted to a new product (e.g. for its new features or styling), which pushes him/her away from the currently owned one. On the other hand, the product in possession exerts a pull on the person (e.g. because it is familiar or has a special meaning). If a person is attached to a product, detaching from and ultimately abandoning this product is undesirable. People feel that losing the product implies that the special meaning conveyed by the product is lost as well. Therefore, people strive to maintain products to which they are attached and exhibit protective behaviors toward these products (Ball and Tasaki, 1992; Belk, 1988; Schultz, Kleine

430

Product Experience

and Kernan, 1989). As long as the product’s special meaning is sufficiently important to the individual and cannot be substituted by a replacement product, a person will be reluctant to replace and dispose of these objects (Mugge, Schifferstein and Schoormans, 2006c). More knowledge on product attachment can help designers to design products with a prolonged psychological lifetime.

3. DETERMINANTS OF PRODUCT ATTACHMENT People develop feelings of attachment to consumer durables, irrespective of the utilitarian meaning of these products. Why do people become attached to certain objects? In the literature on product attachment, several reasons for becoming attached to products have been proposed (Ball and Tasaki, 1992; Kleine, Kleine and Allen, 1995; Schifferstein, Mugge and Hekkert, 2004; Schultz, Kleine and Kernan, 1989; Wallendorf and Arnould, 1988). In addition, several consumer behavior researchers have explored the meanings of products that are considered to be special, treasured, important, or favorite (Csikszentmihalyi and Rochberg-Halton, 1981; Dittmar, 1991; Dyl and Wapner, 1996; Kamptner, 1995; Richins, 1994). Describing an object as special or favorite may imply the presence of an emotional bond. Consequently, the product meanings distinguished in these studies may be possible determinants of product attachment. Based on the findings of these studies, we propose the following four determinants of product attachment: 1. 2. 3. 4.

Pleasure: the product provides pleasure. Self-expression: the product expresses one’s unique identity. Group affiliation: the product expresses one’s belonging to a group. Memories: the product is a reminder of the past.

Below, each determinant is explained in more detail. Specifically, we propose several design strategies influence the determinants, and thereby strengthen the emotional bond between a person and his/her product.

3.1. Pleasure An example of pleasure as a determinant was given at the start of this chapter, when the first author elaborated on the feelings of attachment to her car: The car’s extraordinary and attractive design evokes pleasure, due to which an emotional bond has developed (see Figure 17.1). Several scholars advocated that the experience of pleasure during product usage is related to attachment (Davis, 2002; Norman, 2004; Savas, 2004). Schifferstein et al. (2004) indeed found empirical evidence for the effect of pleasure on product attachment. Feelings of pleasure for a product can come about in two ways. First of all, pleasure may result from the product’s primary function in cases where these products provide entertainment or relaxation, such as televisions, stereos, or ski equipment. However, it is unlikely that the pleasure resulting from the product’s primary function will bring about the experience of product attachment, because this meaning is delivered by all products in the category. For example, a stereo may provide a person with pleasure, because it provides him/her the benefit of listening to music. In this case, we cannot speak of product attachment, because the attachment concerns the product category stereos in general, rather than one particular object. On the other hand, a product’s superior utility (e.g. extra features, greater usability, or higher quality) can be a source of pleasure as well (Jordan, 1998). In this case, it

431

Product attachment: Design strategies to stimulate the emotional bonding to products

is not the primary function that evokes the pleasure, but the extras that are not delivered by other products in the category. In addition, the product’s appearance may evoke aesthetic pleasure (Creusen and Snelders, 2002; Jordan, 1998). Mugge, Schifferstein and Schoormans (2008) found that, due to a product’s superior utility and/or superior appearance, the product may evoke pleasure that similar products do not, which affects the experience of product attachment. An example is a person who enjoys his high-quality stereo, because it provides a great sound or because it has a beautiful design. To stimulate product attachment through the determinant pleasure, designers should design products that perform better and/or are more beautiful than comparable products. These particular products may gain a special meaning to the owner and an emotional bond may develop. We acknowledge that this is no easy task for designers, because many companies strive for a superior utility or appearance in their product designs. Another difficulty of this design strategy is that technological improvements in new products may quickly reduce the pleasure for the product in possession. Manufacturers often deliberately accelerate product lifecycles by introducing new products with new features and by stimulating fashion changes. This planned obsolescence negatively affects the experience of pleasure for the currently owned product and, therefore, the special meaning and the experience of product attachment will only be short-lived. These arguments are corroborated by the findings from Schifferstein et al. (2004). They found that the experienced degrees of attachment and pleasure are relatively high for products that are owned for less than a year (see Figure 17.3). However, after the first year of ownership, the experienced attachment and pleasure has already decreased. This suggests that attachment to a product that provides a person with pleasure is often short-lived. To evoke long-lived pleasure, designers should try to incorporate pleasure eliciting attributes that are more or less exclusive for a particular product variant. Then it is less likely that other products can take over the special meaning of pleasure. Accordingly, people have a stronger tendency to continue such a relationship, and the experience of attachment to the product will persist for a longer period of time. An opportunity to achieve this is to create products that surprise the consumer. Past research concluded that surprising products are more enjoyable (Vanhamme and Snelders, 2003). Although it is unlikely that the product continues to strongly surprise the consumer over time, implementing a

5

Mean rating

4

3

Attachment Memories Enjoyment

2

1

0

<1

1 to 3

4 to 8

9 to 20

>20

Length of ownership (years)

FIGURE 17.3 Attachment, memories and enjoyment as a function of length of ownership, reprinted from Schifferstein et al. (2004).

432

Product Experience

surprise in the product design can still have a long-term effect on pleasure and on product attachment. Being surprised brings about physiological (e.g. changes in heart rate) and behavioral (e.g. special facial expression) changes, which encourages the surprised person to focus their attention on the product. As a result of this heightened awareness, a surprise is better stored in one’s memory (Derbaix and Vanhamme, 2003; Lindgreen and Vanhamme, 2003). Consumers may think back to the pleasantly surprising event while using the product and may, therefore, continue to experience pleasure from the product over an extended period of time. Ludden, Schifferstein and Hekkert (2007) presented several design strategies to create surprising products. These strategies are based on a combination of new and familiar elements in the product design.

3.2. Self-expression The determinant self-expression stems from a person’s desire to differentiate oneself from others and to express his/her personal identity. People are motivated to establish and communicate a personal identity, distinct from that of others. By acquiring, displaying, and using products, an individual can symbolically display one’s individuality to oneself and to others (Solomon, 1983). For example, a person’s clothing expresses who he/she is as an individual. If a product is used to define and maintain one’s personal identity, this product gains a special meaning to the owner. Past research on product attachment concluded that people tend to develop stronger attachment to products that are used to express and maintain a personal and unique identity (Ball and Tasaki, 1992; Kleine, Kleine and Allen, 1995; Schultz, Kleine and Kernan, 1989; Wallendorf and Arnould, 1988). Product personality A possible design strategy to stimulate product attachment through the determinant self-expression, is to implement product personality in the product design. Govers (2004) has defined product personality as ‘the profile of personality characteristics that people use to describe a specific product variant and to discriminate it from others’ (p. 15). For example, a product variant can be experienced as cute, extrovert, or practical (see Figure 17.4). Product personality is believed to be a meaningful tool for designers of consumer durables to communicate symbolic meaning (Govers, 2004; Janlert and Stolterman, 1997; Jordan, 2002). The product’s shape, material, texture, and color affect the personality that consumers recognize in a product. Designers are able to translate personality characteristics into the product appearance in a way consumers understand (Govers, Hekkert

FIGURE 17.4 An extrovert and a practical toaster.

Product attachment: Design strategies to stimulate the emotional bonding to products

433

and Schoormans, 2002). Moreover, Govers and Mugge (2004) found evidence that people become more attached to products with a personality that is similar to their own personality, than to products with a dissimilar personality. Products with personality associations similar to the owner’s personality allow him/her to express himself/herself. As a result, the product gains a special meaning and an emotional bond may develop. Although personality is a stable construct that does not change much over time (Costa and McCrae, 1988), a second study on product personality showed that personality congruity does not necessarily imply that the person–product relationship will be longlasting (Mugge, Schifferstein and Schoormans, 2006c). For most consumer durables, the experience of product attachment is dynamic (Ball and Tasaki, 1992; Kleine and Baker, 2004; Mugge, Schifferstein and Schoormans, 2006b; Schultz, Kleine and Kernan, 1989). The strength of the person–product relationship can change over time as a result of changes in the product (e.g. performance deficiencies), changes concerning the owner (e.g. role transitions), or changes in the situational context (e.g. fashion changes). The findings of our study suggested that fashion may serve as a moderator for the relationship between product attachment and product lifetime. A necessary condition for an extension of the product lifetime appears to be that the product’s design remains in general fashion acceptance. Otherwise, evaluation of the product as being old-fashioned will reduce the product’s value for maintaining a positive view of the self, resulting in early detachment. The finding that the degree of attachment to a product with a congruent personality depends on fashion changes and on other competitive products can be explained by the fact that these self-congruent products are considered to be replaceable (Mugge, Schoormans and Schifferstein, 2005). If the product and its meaning can be separated, the product can be replaced by a new product that conveys the same meaning to the owner. For example, an extrovert watch can serve as a sign to express one’s extrovert personality. However, this symbolic meaning is not exclusively related to this particular object. Other watches or consumer durables can have similar personalities and can serve as similar signs. Accordingly, the watch is replaceable and the attachment to the product is likely to be only short-term. As long as the product is replaceable, the strength of the person– product relationship strongly depends on the characteristics of competitive products. To stimulate long-term attachment, designers should encourage a product’s irreplaceability by designing products that are inextricably bound up with their special meaning. Product personalization From this perspective, an interesting design strategy to influence the determinant self-expression is to personalize the product. Based on the definition of Blom (2000), product personalization is defined as a process that defines or changes the appearance or functionality of a product to increase its personal relevance to an individual. Mugge, Schifferstein and Schoormans (2004) found that people become more attached to products they have personalized themselves. During the personalization process, a person is actively involved in the design of his/her personal product. The outcome of the personalization process is that the consumer obtains a more personal and unique product (Schreier, 2006). The personalized product often also represents a personal accomplishment to the owner (Franke and Piller, 2004). As a result, the product can fulfill the need for self-expression (Blom and Monk, 2003). We acknowledge that this may partly be triggered by a person’s biased perception of the personalized product. As a result of the active participation, people may simply perceive the product as providing a better fit to their preferences and, therefore, as more self-expressive (Simonson, 2005). Self-expression, in turn, has a positive effect on the degree of attachment to this product. Because this specific object resulted from the person’s active participation in the design process, it

434

Product Experience

FIGURE 17.5 Nike mass customization site (http://nikeid.nike.com).

is likely that the product obtains a special meaning that has a factual connection to the product. Consequently, this product may become irreplaceable to the owner. Various personalization options differ in the degree of design authority, that is the degree of creative involvement offered to the consumer (Mugge, Schifferstein and Schoormans, 2006a). Accordingly, not all personalization options are equally relevant to stimulate product attachment. The more a person is involved in the design process and can act as a co-designer of his/her own product, the more effort he/she will invest in the product, and the more personal and self-expressive the product is likely to become. A popular way for manufacturers to personalize durables is by offering mass customization services. A mass customization service allows consumers to create a personalized product by selecting components, accessories, and colors from a predefined set of options. An example is the mass customization website of Nike (http://nikeid.nike.com) that enables consumers to ‘design’ their own shoes by allowing consumers to specify different colors for the shoe (see Figure 17.5). Norman (2004) argued that these products are better in satisfying our needs, but they do not guarantee emotional attachment. We suggest that the degree of design authority offered to the consumer is relatively low in most mass customization services. Consumers are often only allowed to make choices among predetermined alternatives. Because other consumers can easily create an identical product, mass customized products are probably not entirely unique. As a result, these ‘personalized’ products are necessarily limited in providing the symbolic meaning of self-expression and in stimulating product attachment. To stimulate product attachment, designers should implement those types of product personalization that demand a sufficient level of design authority by offering consumers the possibility to be truly creative. An example of the latter is the NOKIA 3220 mobile phone that allows consumers to personally design their own cut-out cover using their creativity, instead of merely choosing one among several predetermined covers (see Figure 17.6). This provides consumers with the opportunity to create a more personal and unique product.

Product attachment: Design strategies to stimulate the emotional bonding to products

435

FIGURE 17.6 Nokia design tool.

A potential drawback of offering consumers a great deal of design authority is that they may not fully understand what solutions correspond to their needs and desires. Furthermore, they may spoil the product, because they are not sufficiently skilled to design their own products. Consumers may also become confused with the great number of options available (Huffman and Kahn, 1998). It is the designer’s task to create a context in which a balance is found between creating design opportunities and guaranteeing adequate product quality. Product personalization can only be a success if designers are able to design the personalization process in such a manner that the consumer can handle the consequences. Accordingly, designers could create a toolkit to support the consumers in their choice, while they may still take credit for the product design (Crabbe, 2001; Von Hippel, 2001). For example, toolkits can offer consumers module libraries with a number of standards for several product parts. Consumers can creatively use these standards as a starting point to create one’s own unique product while restricting the required risk. Group affiliation Group affiliation is concerned with the relational side of the self. This determinant of product attachment stems from people’s need to be connected, joined, associated, and involved with others. Products that support group affiliation define to what groups an individual belongs. They symbolize a person’s desirable connections to family members, friends, or social groups. An example is an emotional bond to a wedding ring, because this particular ring symbolizes the connection to one’s spouse. People can also use products to enact one of their social identities (Kleine, Kleine and Kernan, 1993). For example, a sweater can show a student’s connection to a fraternity. People become more attached to products that symbolize an important person or social group, because these products enhance that part of the self that needs to feel connected (Ball and Tasaki, 1992; Kleine, Kleine and Allen, 1995; Schultz, Kleine and Kernan, 1989; Wallendorf and Arnould, 1988).

436

Product Experience

Although the determinant group affiliation seems to oppose the determinant selfexpression, they can actually co-exist. Kleine et al. (1995, p. 328) commented on this issue that ‘People are motivated universally to establish and maintain a personal and unique identity, distinct from that of others (i.e. autonomy seeking), while at the same time are motivated to maintain interpersonal connections that also define the self (i.e. affiliation seeking)’. An example of a product that is used for both self-expression and group affiliation is a Harley-Davidson motorcycle. The motorcycle represents one’s belonging to a particular group of motorcyclists. Because most Harley-Davidsons are customized by the owner, they simultaneously express the unique identity of the owner. A design strategy to stimulate the determinant group affiliation is to encourage social contact with and through products by designing products that are shared with others or used in a group setting. As a result of the shared use, the owner may associate the product more and more with certain people or events. In time, the associations and the product become inseparable for the owner, and the product may become irreplaceable. For example, a guitar may become irreplaceable, because it symbolizes a person’s belonging to a rock band. Memories A product can remind a person of people, events, or places that are important to that particular individual. It can help him/her to maintain a sense of the past, which is essential to define and maintain one’s identity. Part of who we are today is the result of who we were in the past. For example, a person can be attached to a souvenir, because it reminds him of a pleasant holiday. In the same way, an heirloom can serve as a reminder of one’s family. Due to the physical association between the product and a special person or place in the past, these products have gained symbolic meaning for the owner (Belk, 1988, 1990). Past research observed a relatively strong relationship between the memories associated with the product and the experience of attachment (Kleine, Kleine and Allen, 1995; Schultz, Kleine and Kernan, 1989; Wallendorf and Arnould, 1988). Schifferstein et al. (2004) found that products that were owned for longer than 20 years were mostly associated with memories (see Figure 17.3). The fact that people have kept these products for more than 20 years suggests that they have developed a strong attachment to these objects. Memories might thus be an important reason for people to develop a long-lasting relationship to the product. Based on these studies, we suggest that designers can influence the emotional bond between consumers and their durables by encouraging the memories associated with a product. However, product-related memories usually develop independently from the product design and are difficult to influence by the designer. Nevertheless, we would like to propose two exploratory design strategies that stimulate the formation of productrelated memories (Mugge, Schoormans and Schifferstein, 2005). The first strategy is to implement odors in products. Odors may elicit associations that are resistant to change (Aggleton and Waskett, 1999) and that are more effective in arousing consumers’ mood or feelings than other sources of sensory stimulation (Herz, 1998). In cases where an odor is likely to evoke a similar, pleasant association among a large group of people, implementing this odor in a product may be used effectively to make consumers experience this pleasant feeling again. For example, Alessi’s Mary Biscuit is a biscuit box that releases a vanilla biscuit smell upon opening, which is likely to elicit a feeling of nostalgia (Holbrook, 1993) by reminding a person of the past (e.g. my grandmother’s cookies). Another opportunity to stimulate the experience of attachment through the determinant memories is by designing products that ‘age with dignity’. Such products are made of materials that form and wear gracefully in time. An example is a leather jacket

Product attachment: Design strategies to stimulate the emotional bonding to products

437

that starts showing wear and tear. During use, a leather jacket can shape according to the owner’s body and can show bare spots. The result is a unique jacket with a personal touch. Accordingly, the product symbolizes the shared history of the person with the object. When implementing this design strategy, designers should bear in mind that ‘aging with dignity’ can only be a success if the signs of wear are interpreted positively by the owner.

4. CONCLUSION Product attachment is a valuable concept for designers who are interested in the emotional impact of their product designs during ownership. To stimulate the degree of attachment to products, the following four determinants are distinguished: Pleasure; selfexpression; group affiliation; and memories. If a product conveys one of these meanings, it may be judged special in comparison to other similar products and, consequently, an emotional bond may develop between the owner and the product. The four determinants are discussed as separate elements that can stimulate product attachment. However, this does not imply that these determinants are completely independent. Products can simultaneously convey multiple meanings and these meanings can also become intertwined. For example, gifts can remind a person of the specific event when the gift was received (memories), but can also have a relational meaning (group affiliation), because the gift connects the recipient to the giver (Sherry, 1983). Based on an understanding of these four determinants of product attachment, several design strategies are proposed for stimulating the experience of attachment to products. These design strategies can help designers to design a product in such a manner that a special meaning is more likely to be associated with it. Particularly interesting are those design strategies that encourage the product’s irreplaceability. In that case, the special meaning only holds for a particular object, and the experience of attachment to the product is likely to last over time. Nevertheless, it is questionable whether designing products that are likely to convey a special meaning to the owner is sufficient to stimulate long-term product attachment. Based on their research findings, Mugge et al. (2006b) concluded that designers should first of all strive for continued product usage; for the people who stopped using their product, the product’s special meaning lost its impact on the experience of attachment. Stopping the usage of a product is a divestment ritual that people employ to empty products of their meanings (McCracken, 1986). Due to these divestment rituals (e.g. cleaning, continued storage without use), meaning-loss will not take place at the moment when the product is disposed of. Usage thus seems essential to prolong the impact of a product’s special meaning on product attachment and for sustaining the consumer–product relationship. A possible reason for people to stop using a product is that the product does not perform according to the owner’s expectations anymore. In other words, to continue product usage the owner should feel satisfied with the product’s performance. Although the presented design strategies can be justified theoretically, it remains difficult to determine the actual effect of implementing these strategies in the product design. At present, these design strategies have only been used sporadically for consumer durables. Moreover, some design strategies are relatively specific and probably only feasible in certain situations. More research is necessary to investigate the generalizability of our findings and to explore what preconditions or limitations the design strategies may depend upon. We do not claim that the presented design strategies are the only ways in which designers can stimulate product attachment. Other opportunities may exist to design

438

Product Experience

products in such a way that they are more likely to bring about one of the determinants of product attachment. Finally, designers should take into consideration that although they may be able to encourage a particular product meaning in a product, it is eventually the individual consumer who gives a product its meaning. People may give the same product very different meanings as a result of cultural, social, and personal influences. Therefore, the exact meaning a product will obtain remains hard to predict.

REFERENCES Aggleton, J. P. and Waskett, L. (1999). The ability of odors to serve as state-dependent cues for real-world memories: Can Viking smells aid the recall of Viking experiences? British Journal of Psychology, 90(1), 1–7. Baldwin, M. W., Keelan, J. P. R., Fehr, B., Enns, V. and Koh-Rangarajoo, E. (1996). Social-cognitive conceptualization of attachment working models: Availability and accessibility effects. Journal of Personality and Social Psychology, 71(1), 94–109. Ball, A. D. and Tasaki, L. H. (1992). The role and measurement of attachment in consumer behavior. Journal of Consumer Psychology, 1(2), 155–172. Belk, R. W. (1988). Possessions and the extended self. Journal of Consumer Research, 15(September), 139–168. Belk, R. W. (1990). The role of possessions in constructing and maintaining a sense of past. In: M. J. Houston (Ed.) Advances in Consumer Research, Vol. 17, pp. 669–676. Provo, UT: Association for Consumer Research. Blom, J. O. (2000). Personalization – A taxonomy. In: CHI 2000 Conference on Human Factors and Computing Systems, pp. 313–314. New York. Blom, J. O. and Monk, A. F. (2003). Theory of personalization of appearance: Why users personalize their PCs and mobile phones. Human–Computer Interaction, 18, 193–228. Bowlby, J. (1979). The making and breaking of affectional bonds. London: Tavistock. Cooper, T. (1994). The durability of consumer durables. Business Strategy and the Environment, 3(1), 23–30. Costa, P. T. and McCrae, R. R. (1988). Personality in adulthood: A six-year longitudinal study of self-reports and spouse ratings on the NEO personality inventory. Journal of Personality and Social Psychology, 54(5), 853–863. Crabbe, A. (2001). Expert systems and the emergence of teledesign. Design Studies, 22(6), 543–555. Creusen, M. and Snelders, D. (2002). Product appearance and consumer pleasure. In: W. S. Green and P. W. Jordan (Eds.) Pleasure with products: Beyond usability, pp. 69–75. London: Taylor and Francis. Csikszentmihalyi, M. and Rochberg-Halton, E. (1981). The meaning of things: Domestic symbols and the self. Cambridge: Cambridge University Press. Davis, G. (2002). Prolonging the pleasure. In: W. S. Green and P. W. Jordan (Eds.) Pleasure with products: Beyond usability, pp. 315–319. London: Taylor and Francis. Derbaix, C. and Vanhamme, J. (2003). Inducing word-of-mouth by eliciting surprise – a pilot investigation. Journal of Economic Psychology, 24(1), 99–116. Desmet, P. M. A. (2002). Designing emotions. Delft: Delft University of Technology. Diener, E. and Lucas, R. E. (2000). Subjective emotional well-being. In: M. Lewis and J. M. Haviland-Jones (Eds.) Handbook of emotions (2nd Ed.), pp. 325–337. New York: The Guilford Press. Dittmar, H. (1991). Meanings of material possessions as reflections of identity: Gender and social-material position in society. Journal of Social Behavior and Personality, 6(6), 165–186. Dyl, J. and Wapner, S. (1996). Age and gender differences in the nature, meaning, and function of cherished possessions for children and adolescents. Journal of Experimental Child Psychology, 62, 340–377. Franke, N. and Piller, F. T. (2004). Value creation by toolkits for user innovation and design: The case of the watch market. Journal of Product Innovation Management, 21(6), 401–415. Govers, P. C. M. (2004). Product personality. Delft: Delft University of Technology. Govers, P. C. M., Hekkert, P. and Schoormans, J. P. L. (2002). Happy, cute and tough: Can designers create a product personality that consumers understand? In: D. McDonagh, P. Hekkert, J. Van Erp and D. Gyi (Eds.) Design and Emotion: The Experience of Everyday Things, pp. 345–349. London: Taylor and Francis. Govers, P. C. M. and Mugge, R. (2004). ‘I love my Jeep, because its tough like me’: The effect of product– personality congruence on product attachment. In: A. Kurtgözü (Ed.) Proceedings of the Fourth International Conference on Design and Emotion. Ankara, Turkey. Grayson, K. and Shulman, D. (2000). Indexicality and the verification function of irreplaceable possessions: a semiotic analysis. Journal of Consumer Research, 27(June), 17–30.

Product attachment: Design strategies to stimulate the emotional bonding to products

439

Herz, R. S. (1998). Are odors the best cues to memory? A cross-modal comparison of associative memory stimuli. Annals of the New York Academy of Sciences, 855, 670–674. Holbrook, M. B. (1993). Nostalgia and consumption preferences: Some emerging patterns of consumer tastes. Journal of Consumer Research, 20(September), 245–256. Huffman, C. and Kahn, B. E. (1998). Variety for sale: Mass customization or mass confusion. Journal of Retailing, 74(4), 491–513. Janlert, L. E. and Stolterman, E. (1997). The character of things. Design Studies, 18(3), 297–314. Jordan, P. W. (1998). Human factors for pleasure in product use. Applied Ergonomics, 29(1), 25–33. Jordan, P. W. (2000). Designing pleasurable products: an introduction to the new human factors. London: Taylor and Francis. Jordan, P. W. (2002). The personalities of products. In: W. S. Green and P. W. Jordan (Eds.) Pleasure with products: Beyond usability, pp. 19–47. London: Taylor and Francis. Kamptner, N. L. (1995). Treasured possessions and their meanings in adolescent males and females. Adolescence, 30(118), 301–318. Kleine, R. E., Kleine, S. S. and Kernan, J. B. (1993). Mundane consumption and the self: A social-identity perspective. Journal of Consumer Psychology, 2(3), 209–235. Kleine, S. S. and Baker, S. M. (2004). An integrative review of material possession attachment. Academy of Marketing Science Review, 1. Kleine, S. S., Kleine, R. E. and Allen, C. T. (1995). How is a possession ‘me’ or ‘not me’? Characterizing types and an antecedent of material possession attachment. Journal of Consumer Research, 22(December), 327–343. Lindgreen, A. and Vanhamme, J. (2003). To surprise or not to surprise your customers: The use of surprise as a marketing tool. Journal of Customer Behavior, 2(2), 219–242. Ludden, G. D. S., Schifferstein, H. N. J. and Hekkert, P. (2007). Surprise as a design strategy. Design Issues (in press). McCracken, G. (1986). Culture and consumption: a theoretical account of the structure and movement of the cultural meaning of consumer goods. Journal of Consumer Research, 13(June), 71–84. Mugge, R., Schifferstein, H. N. J. and Schoormans, J. P. L. (2004). Personalizing product appearance: The effect on product attachment. In: A. Kurtgözü (Ed.) Proceedings of the Fourth International Conference on Design and Emotion. Ankara, Turkey. Mugge, R., Schifferstein, H. N. J. and Schoormans, J. P. L. (2006a). The different ways to personalize products: A categorization study. Working paper, Delft University of Technology, the Netherlands. Mugge, R., Schifferstein, H. N. J. and Schoormans, J. P. L. (2006b). A longitudinal study on product attachment and its determinants. In: K. M. Ekström and H. Brembeck (Eds.) European Advances in Consumer Research, Vol. 7, pp. 641–647. Duluth, MN: Association for Consumer Research. Mugge, R., Schifferstein, H. N. J. and Schoormans, J. P. L. (2006c). Product attachment and product lifetime: The role of personality congruity and fashion. In: K. M. Ekström and H. Brembeck (Eds.) European Advances in Consumer Research, Vol. 7, pp. 460–466. Duluth, MN: Association for Consumer Research. Mugge, R., Schifferstein, H. N. J. and Schoormans, J. P. L. (2008). Product attachment and satisfaction: The effects of pleasure and memories. In: S. Borghini, M. McGrath and C. C. Otnes (Eds.) European Advances in Consumer Research, Vol. 8 (in press). Duluth, MN: Association for Consumer Research. Mugge, R., Schoormans, J. P. L. and Schifferstein, H. N. J. (2005). Design strategies to postpone consumers’ product replacement: The value of a strong person–product relationship. The Design Journal, 8(2), 38–48. Norman, D. A. (2004). Emotional design: Why we love (or hate) everyday things. New York: Basic Books. Richins, M. L. (1994). Valuing things: The public and private meanings of possessions. Journal of Consumer Research, 21(December), 504–521. Roster, C. A. (2001). Letting go: the process and meaning of dispossession in the lives of consumers. In: M. C. Gilly and J. Myers-Levy (Eds.) Advances in Consumer Research, Vol. 28, pp. 425–430. Provo, UT: Association for Consumer Research. Savas, O. (2004). A perspective on the person–product relationship: Attachment and detachment. In: D. McDonagh, P. Hekkert, J. Van Erp and D. Gyi (Eds.) Design and emotion: The experience of everday things, pp. 317–321. London: Taylor and Francis. Schifferstein, H. N. J., Mugge, R. and Hekkert, P. (2004). Designing consumer–product attachment. In: D. McDonagh, P. Hekkert, J. Van Erp and D. Gyi (Eds.) Design and emotion: The experience of everyday things, pp. 327–331. London: Taylor and Francis. Schifferstein, H. N. J. and Pelgrim, E. (2004). Consumer–product attachment: The construct and its measurement. Unpublished manuscript, Delft University of Technology, the Netherlands. Schreier, M. (2006). The value increment of mass-customized products: An empirical assessment and conceptual analysis of its explanation. Journal of Consumer Behavior, 5(July–August), 317–327.

440

Product Experience

Schultz, S. E., Kleine, R. E. and Kernan, J. B. (1989). ‘These are a few of my favorite things.’ Toward an explication of attachment as a consumer behavior construct. In: T. Scrull (Ed.) Advances in Consumer Research, Vol. 16, pp. 359–366. Provo: UT: Association for Consumer Research. Sherry, J. F. (1983). Gift giving in anthropological perspective. Journal of Consumer Research, 10(September), 157–168. Simonson, I. (2005). Determinants of customers’ responses to customized offers: Conceptual framework and research propositions. Journal of Marketing, 69(January), 32–45. Solomon, M. R. (1983). The role of products as social stimuli: a symbolic interactionism perspective. Journal of Consumer Research, 10(December), 319–329. Thomson, M., MacInnis, D. J. and Park, C. W. (2005). The ties that bind: Measuring the strength of consumers’ emotional attachments to brands. Journal of Consumer Psychology, 15(1), 77–91. Van Hemel, C. G. and Brezet, H. (1997). Ecodesign; A promising approach to sustainable production and consumption. Paris: United Nations Environmental Programme. Van Hinte, E. (1997). Eternally yours: Visions on product endurance. Rotterdam: 010 Publishers. Van Nes, N. (2003). Replacement of durables: Influencing product life time through product design. Rotterdam: Erasmus University. Vanhamme, J. and Snelders, D. (2003). What if you surprise your customers... will they be more satisfied? In: P. A. Keller and D. W. Rook (Eds.) Advances in Consumer Research, Vol. 30, pp. 48–53. Valdacosta: Association for Consumer Research. Verbeek, P. P. and Kockelkoren, P. (1997). Matter matters. In: E. van Hinte (Ed.) Eternally yours: Visions on product endurance, pp. 100–115. Rotterdam: 010 Publishers. Veryzer, R. W. (1995). The place of product design and aesthetics in consumer research. In: F. R. Kardes and M. Sujan (Eds.) Advances in Consumer Research, Vol. 22, pp. 641–645. Provo, UT: Association for Consumer Research. Von Hippel, E. (2001). PERSPECTIVE: User toolkits for innovation. Journal of Product Innovation Management, 18(4), 247–257. Von Weizsacker, E. U., Lovins, A. B. and Lovins, L. H. (1997). Factor four: Doubling wealth, halving resource use. London: Earthscan. Wallendorf, M. and Arnould, E. J. (1988). ‘My favorite things’: A cross-cultural inquiry into object attachment, possessiveness, and social linkage. Journal of Consumer Research, 14(March), 531–547.

18

CRUCIAL ELEMENTS OF DESIGNING FOR COMFORT PETER VINK TU Delft, Delft, The Netherlands

MICHIEL P. DE LOOZE TNO, The Netherlands

1. ATTENTION FOR COMFORT IN DESIGN Comfort is increasingly important in sales of cars, hand tools, seats, earth-moving machines, and airplane tickets. Discomfort is a predictor of musculoskeletal injuries and should be reduced in situations that consume a significant part of our time. Therefore, manufacturers of products such as seats, cars, beds, hand tools, and production lines need knowledge on comfort and discomfort. Despite the frequent use of the term comfort, there is no such thing as a general notion of comfort or discomfort. Webster’s dictionary defines comfort as a state or feeling of having relief, encouragement and enjoyment. Slater (1985) defines comfort as a pleasant state of physiological, psychological, and physical harmony between a human being and its environment. Richards (1980) stresses that comfort is a state of a person involving a sense of subjective well-being, in reaction to an environment or situation. According to de Looze, Kuijt-Evers and van Dieën (2003) the following issues are generally accepted in the literature: (1) comfort is a construct of a subjectively defined personal nature; (2) comfort is affected by factors of various nature (physical, physiological, psychological); and (3) comfort is a reaction to the environment. Therefore, in this chapter comfort is seen as a pleasant state or feeling as a reaction to the physical environment. Within this domain, this chapter first concerns theoretical considerations on comfort and discomfort. The theoretical aspects will be illustrated in some specific cases.

2. TWO ENTITIES: COMFORT AND DISCOMFORT In the scientific literature the definition of comfort is under debate. Many authors make a distinction between comfort and discomfort. These authors state that if we consider comfort Product Experience Copyright © 2008 Elsevier Ltd.

441

442

Product Experience

and discomfort during sitting, absence of discomfort does not automatically result in comfort. Comfort will be felt when more is experienced than expected. This vision is supported by research of Zhang, Helander and Drury (1996) and by Helander and Zhang (1997). Based on questionnaires, they found that discomfort is more related to pain, ache, and heavy legs (see Table 18.1). When there is absence of discomfort, nothing is experienced. To notice comfort, more should be experienced. Comfort is related to, for instance, a sense of well-being and to the plushness of the chair. If these factors are clustered, discomfort is more related to physical aspects and comfort more to emotional experiences. De Looze et al. (2003) developed a scheme to explain the effect of the different factors (see Figure 18.1). The left part of this scheme concerns discomfort. According to Zhang et al. (1996), physical processes underlie discomfort. As with previous models on the causes of work-related physical complaints (Winkel and Westgaard, 1992; Armstrong et al., 1993), the authors consider ‘exposure’, ‘dose’, ‘response’, and ‘capacity’ as the main issues. According to Armstrong et al. (1993), exposure refers to the external factors producing a disturbance of the internal state (dose) of an individual. The dose may evoke a cascade of mechanical, biochemical or physiological responses. The extent to which external exposure leads to an internal dose and responses depends on the physical TABLE 18.1 Factors influencing comfort or discomfort during sitting, according to Zhang et al. (1996) Discomfort

Comfort

Fatigue Pain Posture Stiffness Heavy legs

Luxury Safe Refreshment Well-being Relaxation

COMFORT MODEL

Human

Physical capacity

SITTING DISCOMFORT (⫽feeling of pain, soreness, numbness, stiffness)

Product

Physical features

Context

Physical processes

SITTING COMFORT (⫽feeling of relaxation, well-being)

expectations

emotions

Physical features aesthetic design Physical environment task

Physical environment task psychosocial factors

FIGURE 18.1 Reasoning scheme of comfort and discomfort and its underlying factors at human, seat and context level according to de Looze et al. (2003).

Crucial elements of designing for comfort

443

capacity of the individual. With regard to sitting, it could be said that the physical characteristics of the office seat (e.g. form, softness), the environment (e.g. table height) and the task (e.g. the performance of VDU activities) expose a seated person to loading factors, which may concern forces and pressure from the seat on the body and joint angles. These external loads may yield an internal dose in terms of muscle activation, internal force, intra-discal pressure, nerve and circulation inclusion, and skin and body temperature rise, provoking further chemical, physiological and biomechanical responses. By exterocepsis (stimuli from skin sensors), propriocepsis (stimuli from sensors in the muscle spindle, tendons and joints), interocepsis (stimuli from internal organ systems), and nocicepsis (stimuli from pain sensors), the perception of discomfort might be established. The right part of the scheme concerns comfort, i.e. feelings of relaxation and well-being. Again, the influential factors are presented on a human, product, and context level. At a context level, not only the physical features are assumed to play a role, but also psycho-social factors such as job satisfaction and social support. At a seat level, the aesthetic design of a seat in addition to physical features may affect the feelings of comfort. At the human level the influential factors are assumed to be individual expectations, and other individual feelings or emotions. The most important underlying factors of discomfort, as suggested by Helander and Zhang (1997), are shown in Figure 18.1 by the vertical arrow pointing upwards. From this model it can be expected that for discomfort the relationships of objective measures with discomfort would be stronger than for comfort, as the link between discomfort and objective measures of physical exposure, dose or response is more direct. Not all authors stress the difference between comfort and discomfort. Kuijt-Evers et al. (2004) showed a difference between the construct of comfort/discomfort in using hand tools (such as screwdrivers and pliers) and comfort/discomfort in seating. In many of the seating comfort studies, comfort and discomfort are different entities with different underlying determinants. However, in the use of hand tools adverse body effects such as cramped muscles, blisters and inflamed skin, underlie both comfort and discomfort. Additionally, comfort is mostly determined by functionality and physical interaction using hand tools (Kuijt-Evers et al., 2005). This could be explained by the different nature of seats and hand tools. Unlike sitting, the use of hand tools is mostly accompanied by discomfort. Helander and Zhang (1997) argue that when discomfort factors are present, comfort factors become secondary in the perception of comfort/discomfort. Because discomfort factors are present in hand tool use, comfort may be dominated by discomfort. Therefore, respondents may also think of comfort while using hand tools in terms of the absence or reduction of discomfort. Clustering the factors for hand tools was also possible (Kuijt-Evers et al., 2004). Based on statistical analysis, six comfort factors could be distinguished (functionality; posture and muscles; irritation and pain of hand and fingers; irritation of hand surface; handle characteristics; and aesthetics). These six factors explain 53.8% of the variance, and can be classified into three meaningful groups: Functionality; physical interaction; and appearance. In conclusion, in the use of hand tools the same descriptors are related to comfort as to discomfort; descriptors of functionality are most related to comfort, followed by descriptors of physical interaction; and descriptors of appearance become secondary in determining comfort when using hand tools. For seat design with a focus on comfort, it is important to decide whether the focus is on the physical aspects to reduce discomfort or on emotion and luxury to increase comfort. In hand tools functionality, physical interaction, and appearance are important for discomfort as well as comfort. Perhaps the difference lies in its function. In products

444

Product Experience

enabling tasks, like hand tools, comfort and discomfort could be each others’ opposite on one dimension, while products that are indirectly related to performing a task like seats, could distract in a physical sense and cause discomfort, which is different from the wellbeing experience.

3. COMFORT AND DISCOMFORT ASPECTS OF IMPORTANCE FOR DESIGN There are several aspects of comfort and discomfort important in designing for comfort. Each aspect will be elaborated further in this section. The first aspect is the difference between short-term and long-term (dis)comfort. The first comfort impression is important for fair trades and showrooms, but long-term effects are important in using the product. This difference is important because the comfort in the showroom could be experienced differently from the comfort in long-term use. This also has consequences for the tests. In fact, tests on short-term (dis)comfort are needed, as well as tests on long-term effects. Several studies describe long-term (dis)comfort and others the shortterm experience (Vink, Bronkhorst and Goossens, 2006). The effects of a product on (dis)comfort experience are not always similar. Kuijt-Evers (in Bronkhorst et al., 2001) showed that 49 experienced office workers evaluated one out of four office chairs negatively based on the visual information (a brown traditional chair). Contrary to what was expected, this chair was evaluated positively after for some hours of use. This means that explicit attention is needed for short-term and long-term experiences. A second aspect concerns the subjective character. This is an aspect which is not under debate in the literature (de Looze et al., 2003). This means that everyone has his or her opinion on the comfort of a product, and designing a product that is comfortable for everyone will be difficult. This is of importance because it strongly relates to how to cope with comfort in design. A product in itself can never be comfortable; it becomes comfortable (or not) in its use. The user decides whether or not a product is comfortable, or leads to discomfort, by using the product. This complicates the construct of comfort because it is not known how every individual will react to a product. For instance, for Passenger 1 on a long-distance flight, knee space is important. Passenger 2 wants a reduction in noise, and Passenger 3 needs more shoulder space. This complication in predicting comfort is one of the reasons why group user tests in different phases of the product design are of importance. A product should be studied at group level, as individuals may have different opinions and a product used by groups should fit the majority of demands of the group. The end-user should be involved, because he is the expert in experiencing the product. This end-user involvement is studied in the area ‘participatory ergonomics’. With the help of participatory ergonomics, the design process can be defined with special attention on how to take the participants into account, especially the end-user. A definition of participatory ergonomics is: It is the adaptation of the environment to the human (that is ergonomics) together with the proper persons in question (participants) (Vink, 2005). Proper means, in this case, the stakeholder needed for a good design, such as the end-user, designer, experts, and other representatives involved in buying and using the end-product. Defined in this way participatory ergonomics is more an umbrella under which different approaches are found. The common characteristic is that in a design process attention is paid explicitly to the role of designers, employees, end-users, and others involved. Subjective as well as objective methods could be used. Subjective methods such as questionnaires, interviews, and observations are very helpful. An often used method is the Local Postural Discomfort (LPD) questionnaire, which helps in finding out the part of the product that could be improved to reduce discomfort (see Figure 18.2). The LPD method is

445

Crucial elements of designing for comfort

Localized Postural Discomfort Q

R

S A

T Y

Z

X

0

⫽ no discomfort at all

0.5

⫽ extremely little discomfort (hardly noticeable)

D

F

G

1 2

⫽ very little discomfort ⫽ little discomfort

C

V

3 4

⫽ moderate discomfort ⫽ somewhat high discomfort

5

⫽ high discomfort

6 7 8

⫽ very high discomfort

O

J

B

P

K

9 10

⫽ extreme discomfort (almost maximum)

FIGURE 18.2 A method often used in discomfort research: local postural discomfort (LPD). After learning the scale by holding an object of 5 kg in a 90-degree elevated arm, subjects are asked to score discomfort (from 0–10) in the body map (left).This LPD map of the body shows the separate regions to be given a discomfort score on a 0–10 scale.

probably best validated, as it is used often and checks whether it measures what it should measure (Delleman, 1999). Test-result reliability appears to be good (van der Grinten and Smitt, 1991). In fact a subjective method is enough to define whether or not comfort is experienced, as it is a subjective phenomenon. However, if the results should be related to a product, and if explanations are needed, it is useful to have objective methods as well. A frequently used objective method in seat design is pressure distribution (de Looze et al., 2003). This can be related to the form and softness of the seat. In cabin design the ergomix (a system mixing drawings with real subjects), virtual reality, and mock-ups can be used. Successes have been achieved in projects where full-scale objects, mock-ups, and prototypes have been used. These achievements can be explained by the similarity of the tools to the devices that participants use in their daily lives. The deployment of this type of design method means that the situation and environments that are forged are not merely attempts to verbally describe and imagine the future use of an environment, but instead serve as a method of experiencing that world through direct physical involvement (Ehn, 1988). A third aspect concerns the factors influencing the comfort experience. In the model of de Looze (see Figure 18.1) an important environmental influence on comfort could be the temperature; a physical feature could be the foam, measured by pressure distribution; and aesthetics could be the color of the product which is noticed by our visual input. Most papers in the scientific literature concern thermal comfort (Vink, 2005). This is especially important in controlling the climate in buildings. A pleasant climate is often not noticed, but a high or low temperature attracts attention and the discomfort is perceived. Inner climate comfort is also important because it is related to productivity in offices. A high inner environment temperature of 27ºC reduces comfort and the measured productivity is 30% lower than at 21ºC (Kosonen, 2004). Apart from temperature, visual input, noise, vibration, humidity, pressure, and posture/movement do influence our comfort experience (Vink, 2005). It is important to distinguish the most important underlying factor influencing comfort in the design as it influences the way of measuring. Vibration, noise, temperature, and humidity measurement systems are often used, and various systems are available to measure these influencing factors objectively (see Figure 18.3).

446

Product Experience

FIGURE 18.3 Five systems to measure factors that influence comfort – upper left: a decibel measurement device; upper middle: a system measuring the vibration on the seat; upper right: measuring the temperature; lower left: a system to measure the heat transmission in a cushion; lower right: a system to measure the humidity transfer in a cushion.

Measurement of pressure and posture/movement have recently received more attention. Various studies show the relationship between pressure and discomfort in sitting (e.g. Goossens, Teeuw and Snijders, 2005). Goossens et al. (2005) increased the force applied to a buttock by pushing a circle of 30 mm in diameter upwards. They found a clear correlation between the level of pressure (force increase) and the perceived discomfort. To feel pressure we have sensors located in the skin. Generally, a better distribution of pressure between a seat or handle and the human body leads to less discomfort. A literature survey (de Looze et al., 2003) showed that of all objective measures, the distribution of pressure has the clearest relationship to discomfort in sitting. In this area Goossens et al. (2005) showed that participants are able to perceive small differences in pressure on their bottoms and could translate this to discomfort. For hand tools the relationship between pressure and discomfort was also established. For a hand tool which requires high forces and large movement trajectories (Kuijt-Evers, 2007), higher average pressures and larger pressure contact areas are related to discomfort. In fact pressure is one of the aspects of touching an object. This tactile experience is extensively studied by Sonneveld (2007). It’s an interesting area strongly related to comfort, as a thick carpet is experienced differently from a concrete

Crucial elements of designing for comfort

447

floor by our bare feet, and the textures of handles have an influence on the feeling of comfort. Sonneveld (2007) describes how we can take these feelings into account during the design process. The posture and movements determined by the product can also lead to discomfort. In the long run, local perceived discomfort could even result in musculoskeletal disorders (Proper, Bongers and van der Grinten, 1999). Back pain is found in one-third of all European workers. Neck and shoulder pain is found in almost onequarter of European workers. Not only is the human factor a problem, but so are the costs. In The Netherlands, musculoskeletal injuries cost €8 billion each year (Bongers, 2000). Thus, the problem is large enough to require that something should be done about it. This is an opportunity for designers: Design products that reduce musculoskeletal injuries. In other words, use experiments during the design process to avoid discomfort, and as a result prevent musculoskeletal injuries. To avoid discomfort and related musculoskeletal loading, several measuring methods are available. Apart from the subjective method Local Postural Discomfort (see Figure 18.2), there are many objective methods to measure posture and movements. Systems to measure posture could be goniometers, electromyography (EMG) or tracking devices for parts of the human body. All these aspects show that, for the physical side of (dis)comfort, knowledge and measurement methods are available. So, in previous studies, most attention was given to discomfort. For measuring comfort there is room for additional methods. Probably these methods should be more of a subjective nature (Kuijt-Evers, 2007). Factors like ‘is easy to use’, ‘has a nice feeling handle’, and ‘feels clammy’ are difficult to measure with objective systems. However, there are also factors that influence comfort that can be measured in some way in future projects, such as ‘fits in hand’, ‘is functional’, ‘has a good force transmission’.

4. COMFORT IMPROVEMENT IS POSSIBLE As there is no such thing as a general notion of comfort or discomfort, it is difficult to improve comfort. On an individual level it is especially difficult, as everyone has his or her opinion about comfort. However, at a group level, comfort can be improved and it is possible to do so. Positive effects on comfort of newly designed products are found, showing that improvements are possible. Kuijt-Evers, Krause and Vink (2003) found that excavators and wheel loaders were rated significantly more comfortable than machines 4 years older. Blok, Kamp and Vink (2007) found that the older type aircrafts of the Airbus A300 and A310, Boeing 757 and 767 score significantly lower in comfort than the newer Airbus A330, A340 and next generation Boeing 737 and 777. Also, for hand tools the same effects are described. The new paint scraper was experienced significantly better than an old one (Eikhout et al., 2005). These cases have in common that explicit attention has been paid to the comfort aspects, and end-users were involved in testing, at different stages of the design process.

5. THE CASES The theoretical aspects described above are of importance in comfort design. However, translating these into a design is difficult. Kuijt-Evers (2007) defined a flow chart based on several studies to be applied in hand tool design (see Figure 18.4), but even within a smaller area like hand tool design there are no general valid underlying factors for comfort. Therefore, Kuijt-Evers proposes in her PhD thesis (2007) a flow chart to find the

448

Product Experience

FIGURE 18.4 Flow chart to support designers and researchers to focus on the appropriate comfort descriptors in hand tool evaluation and design (Kuijt-Evers, 2007).

appropriate comfort descriptors and thereby the focus, which can be applied to screwdrivers, paintbrushes or handsaws. The first decision is based on the task (low or high intensity), which illustrates the difference between screwdrivers and handsaws, on the one hand, and paintbrushes on the other. The second decision is based on the force direction on the hand (shear forces or normal forces), which is the main difference between handsaws and screwdrivers. For other hand tools further research is needed, although the system of finding the focus can be applied to all products where comfort is important. This means that first underlying descriptors are defined, then the task is analyzed, and when applicable the physical

Crucial elements of designing for comfort

449

aspects are defined. Combining this flow chart with the above mentioned theoretical considerations, the following steps could be defined for the designer: 1. Description: Define the underlying factors of the comfort design aspect, with main questions such as discomfort or comfort? Short-term and long-term? 2. Task: Find out when and how end-users are using the product. 3. Physical and experience characteristics: Define whether measurements have to be carried out, such as using subjective methods (LPD, questionnaires, interviews) and/or objective methods (pressure distribution, force recording, temperature, etc.). After a description of the cases below, attention will be given to whether these three steps were applicable and if so, how knowledge was gathered and if it was helpful for the design.

5.1. Case I: Discomfort on an assembly line The case Some designers focus on workstation design. In this area it is important to take comfort and discomfort into account. In this case (for more extensive description see de Looze et al., 2005) a workstation where razor machines were assembled was improved. As this is sitting work, comfort and discomfort could be separated. In this case the focus was on physical workload and therefore discomfort was chosen as an effect parameter. The hypothesis was that if the workstation fits better to the human reach envelopes the discomfort will reduce. The background To survive the world-wide competition, assembly enterprises are forced to produce ever-increasing volumes. Meanwhile, they experience an increasing demand for product variation and shorter delivery times. There is also an increasing pressure to reduce the cost price, while retaining quality. This pressure on companies is likely to increase the pressure on the workers. Among the human factor problems in assembly is the high rate of musculoskeletal neck and shoulder disorders (Hagberg and Wegman, 1987). Static neck flexion and repetitive elevation of the arms due to poor workstation design are important risk factors. As is stated above, there are indications that discomfort is a precursor of neck and shoulder disorders. Ideally, productivity increase is combined with discomfort reduction. Philips DAP had much interest in optimizing the assembly line and designed an improvement; they performed tests together with TNO. The current assembly line Philips DAP is a division of the Philips Company (the Netherlands) that produces electric razors in very large volumes. In this company, an experiment was performed to study the effect of workplace adjustment on the productivity and the discomfort of the workers. The production line of Philips is designed with sophisticated VR software tools. However, further improvements are often still possible. For instance, the location of supply containers, from which components have to be picked, is far from optimal. This is mostly due to the large numbers of different components that need to be at hand simultaneously. This leads to frequent, extended reaching and much arm elevation, which consumes time. In studying the logistics, experts in the field of ergonomics saw that reaching and lifting the arms could lead to discomfort and could be improved. Consequently, risks of shoulder discomfort and pain could decrease. In a session with experts, engineers, management, and workers, improvement possibilities proposed by experienced ergonomic experts were discussed and one improvement was chosen, focused on lowering the reach heights and reducing the reach distances. In this case effects of discomfort were recorded, as they were

450

Product Experience

FIGURE 18.5 The ‘high’ (left picture) and ‘low’ (right picture) workplace condition. In the low workplace condition the storage containers for the plates (A) and blades (B) are placed lower and closer to the subject, while the product carrier (C) is also lowered.

assumed to be related to the working height and distances. To find out whether this investment would be cost-effective and better for health, an experiment was carried out. The most important characteristic of the workstation was the lowering of the picking heights. The test Fourteen female assembly workers, all working at Philips DAP, participated in the experiment. We studied these workers when performing the task of assembling the heads of razors. The actions consisted of picking the components from small supply containers, assembling the razor heads, and placing the products onto a product carrier sitting along the normal assembly line. The duration of one cycle (of two products) was roughly 10–15 seconds, depending on the overall work pace. We tested the subject under two conditions. Condition high reflected the current situation, where the supply bins and trays and the product carrier were located at positions requiring considerably high upper-arm elevations. In condition low, these positions were significantly lowered (see Figure 18.5). In the low workplace condition, the storage containers for the plates and blades are placed lower and closer to the subject and the product carrier was also lowered. All 14 subjects performed the task in both the high and the low condition. These performances took place on two consecutive working days, at the same time of day. The sequence of exposure to the low and high conditions was systematically varied across subjects. During the performance of the tasks, a psychophysical protocol was applied in order to determine the maximum acceptable work pace (MAWP) in each situation. The MAWP is the work pace considered by workers to be the highest pace that is still acceptable for an 8-hour working day. Also, real productivity in terms of products was measured. Every 5 minutes during the 50-minute protocol, local musculoskeletal discomfort in various body regions was measured (van der Grinten and Smitt, 1992). This approach applied a 10-point rating scale, where 0 ⫽ no discomfort and 10 ⫽ extreme discomfort.

451

Crucial elements of designing for comfort

TABLE 18.2 Maximum acceptable work pace (MAWP) expressed in number of products per minute and average localized musculoskeletal discomfort in the neck and shoulders as measured at the end of the test (after 50 minutes) Condition (sd)

Average MAWP (range)

Average discomfort

Workplace high Workplace low

9.4 (6.7–13.0) 10.4 (7.2–14.6)*

1.1 (1.0) 0.4 (0.6)*

*Significant difference t-test for paired comparison (p ⬍ 0.05).

Results We observed a significant effect on the workplace condition (see Table 18.2). In the low workplace condition, the maximum acceptable work pace was significantly higher than in the high workplace condition. The difference between the high and the low condition was 1.0 product per minute. This implies a difference in work pace of 10.6%. Despite the relatively low measured discomfort levels in general, we were also able to find a significant reduction for the effect of workplace condition on discomfort. Preliminary results from recent studies by Hamberg (2007) show that very low levels of LPD could even be predictors of musculoskeletal complaints. Musculoskeletal discomfort in the neck and shoulders was significantly lower in the low workplace condition compared to the high workplace condition. Discussion This experiment shows that it is possible to achieve a productivity increase in combination with a discomfort reduction. The redesign of the workstation was responsible for the effects as was shown in this experiment. This was important information for the management in enabling them to make the decision to invest in new production workstations. Therefore, the conclusion is that in high-volume manual assembly operations there is a clear link between workstation design and worker discomfort. In the case discussed here, optimization of the workstation layout reduced discomfort and increased productivity, which is of benefit to both the employees and the company. Regarding point 1 ‘description’, the focus of the comfort design aspect was discomfort, because it was related to musculoskeletal complaints. The focus was on long-term comfort, as the first sight was not the main focus, but the effect after working a day in the workstation. Regarding point 2 ‘task’, it was clear that it considered the specific task that was performed in this workstation. Related to point 3 ‘physical and experience characteristics’ the focus was on the physical characteristics. Soft issues were also explicitly considered, but these regarded the participatory process of development involving the end-users explicitly in the test and involving them in the selection at the end.

5.2. Case II: Comfortable paint scraping The case In designing hand tools, comfort could be an important aspect as well (Kuijt-Evers, 2007). Hand tool use is associated with the occurrence of musculoskeletal disorders (Mital, 1991) and frequently causes feelings of discomfort. However, increasing comfort could have positive effects on the sales of the hand tools. Therefore, in this case a hand tool is designed with specific attention to comfort and discomfort in different stages of the design process. The effect of the design process is described in this case (for a more extensive description see Eikhout et al., 2005).

452

Product Experience

The background Painters are a high-risk group for health problems. According to European and Dutch data from the building industry, painters suffer from more complaints of the upper extremities than the control group in the construction industry (Eikhout et al., 2005). A few years ago, all Dutch painters used the traditional triangular scraper (see Figure 18.6). A study was done regarding the workload in painters including working with the paint scraper. Paint scraping was typified as one of the heaviest tasks by the painters (Eikhout et al., 2005). The main problems in working with the traditional triangular paint scraper were (Eikhout et al., 2005): (1) frequent extreme wrist postures; (2) high forces of the arm, also due to the large moment arm; and (3) awkward postures in the back, shoulder, neck, and elbow. Also, the painters saw blade grinding as inefficiency in the task. Discussing alternative designs In a meeting with designers, painters, and human movement specialists, solutions for the main problems were discussed. Based on this discussion, several drawings were made of potential improved paint scrapers (see Figure 18.7). One idea was to use two hands to grip the paint scraper. Another concept was to reduce the distance between the scraping area and the hand. This reduces the moment arm, and thereby reduces the forces on the hand. The third idea was to change the angle between the grip and the scraper, which improves the position of the hand, wrist, elbow, and shoulder. These redesign proposals were discussed with painters and experts, and some ideas were incorporated in a new design close to Figure 18.7B, but with attention to the force direction. Testing the end-design These redesign ideas led to collaboration with Bahco Tools of Sweden. Bahco produces the so-called ergotool line, in which much attention is paid to ergonomics to arrive at comfortable hand tools. The ergotools have been developed in close collaboration with end users, following an 11-step design process (Bobjer and Jansson, 1997). The new scraper (see Figure 18.8) that was developed in the 11-step process has a blade of hard sintered metal, so that blade grinding is no longer needed. The shape of the handle is also different. The triangular scraper has the largest handle diameter close to the blade, whereas the new scraper’s handle is thickest close to the body. This reduces

FIGURE 18.6 The traditional triangular shaped paint scraper.

453

Crucial elements of designing for comfort

slip between hand and tool. The form of the handle is also better adapted to its use in the scraping task by improving the grip. The question is whether this improvement is also noted by the end-user. Therefore, before market introduction the effects were measured. Twenty professional painters from seven different painting businesses volunteered to participate in this field study. The selection criteria required painters to have a large number of scrape activities during the test period and to have no history of health complaints. All subjects were males with 14 ⫾ 11 years of painting experience; age 33 ⫾ 12 years; height 1.82 ⫾ 0.07 m; and weight 81 ⫾ 11 kg. After an introduction, the painters used the new scrapers in their normal work for two to three weeks. After one week the subjects were visited for guidance, and the test period ended with a test day.

A

B

C

FIGURE 18.7 Three ideas to improve comfort in paint scraping: (A), reducing the load by dividing it over two hands; (B), rotating the handle makes it possible to put the hand in the ideal posture; (C), reducing the moment could reduce the load.

454

Product Experience

FIGURE 18.8 The new paint scraper.

The discomfort was recorded by the LPD method described in Case I. Additionally in a questionnaire, questions regarding comfort experience were asked. Physical load was measured by recording posture, movements, effort, and torque during scraping. Forces and torques were recorded by a trial of strength. A questionnaire was used to record the opinion of the subjects. Results of the end-design test The new scraper showed a tendency in lower torques on the wrist, but further improvement is possible because the torques are still higher than health limits of a daily physical workload (Eikhout et al., 2005). Force measurement showed significant differences. It appeared that sometimes painters used burners. When using the burner, the mean forces were 17% lower on the new scraper than on the triangle. For the new scraper, measured without burning the paint before the scraping, 10% torque reduction was found. Twenty painters completed the questionnaire. The new handle was evaluated as ‘very well manageable’ under separate conditions of dust, grease, and moisture by 74%, 71%, and 72% of the painters, respectively. Fewer blisters, less strength required for use, and a better hold were reported for the new scraper. Grinding time was reduced to zero, which is seen as an advantage, because grinding the old triangular scraper blades took much time. Perceived discomfort was reduced (see Figure 18.9) in working with the new scraper compared with the old scraper; 70% evaluated the new paint scraper as ‘very comfortable’. In the final evaluation, the new scraper was given 8.5 and the triangle 7.0 on a scale of 1 to 10. This difference was significant as well. Discussion This experiment shows that it is possible to improve a tool which has not been changed for years. Also, in this case a productivity increase in combination with a

455

Crucial elements of designing for comfort

Shoulder/neck

Upper arm* Bahco triangular Wrist

Fingers* 0

10

20 30 40 % FIGURE 18.9 Local perceived discomfort (in % of the maximal discomfort averaged over 20 painters) working with the traditional triangular shaped paint scraper and the new Bahco paint scraper (*significant, paired t-test, p ⬍ 0.05).

discomfort reduction was achieved. However, comfort was also improved. The redesign was responsible for the effects as was shown in this experiment. This was important information for the management and marketing department, enabling them to make the decision to invest in new production lines and find a focus for the marketing strategy. Regarding point 1 ‘description’, the focus of the comfort design aspect was discomfort, because it was related to musculoskeletal complaints, but also comfort, because it had to be attractive. Also, short-term as well as long-term comfort had to be considered. This increased the complexity of the design process, as it had to be a good looking, good feeling product at first sight, and had to be beneficial in long-term use. Regarding point 2 ‘task’, it was clear that the main task was scraping. However, it was worthwhile to study the whole work process, as it provided input in improving the grinding operation. Painters didn’t like this task and productivity could also be increased by using the sintered metal blade. Regarding point 3 ‘physical and experience characteristics’, the focus was on the physical characteristics as this was the main problem with the old scraper. Comfort experience was also part of the design process and tested with a questionnaire after working with the prototype. A total of 70% rated the new scraper as very comfortable.

5.3. Case III: Aircraft interior comfort The case In buying airline tickets, comfort can play a role. According to Brauer (2004) airlines can increase their financial margin by reducing maintenance costs. However, a reduction of 14% of the maintenance costs results only in a 1% increase in margin, while an increase of 1% more passengers has the same result. To attract more passengers, the selection behaviour of passengers was studied by Brauer (2004). It appears that passengers first select on point-to-point transport, time, and price. Then, aspects like marketing (frequent flyer programmes) come into play, followed by comfort, past experiences, and delays. For short distances the delay aspect is more important and for long hauls the comfort aspect plays a more important role. Nowadays it is possible for passengers to find the comfort ratings of different airliners on the Internet. Thus, it is important to pay attention to comfort for airliners and their suppliers.

456

Product Experience

FIGURE 18.10 The aircraft interior comfort has influence on the sales of tickets.

Many changes The aircraft interior is changing. Lightweight seats, new materials, better cabin air, new entertainment, and new lighting systems are all to be seen. Most of these innovations are driven by technology. However, the question is whether the passenger experiences these innovations as improvements, and whether these improvements really solve the main problems passengers experience during their flight. The opinion of Internet passengers A study among 11,513 trip reports gathered from the Internet showed that the main problems according to passengers concern legroom, delay, lost luggage, and rude flight attendants (see Figure 18.11). The same study also showed that the comfort rating was significantly higher for the new planes compared with the older airplanes (see Table 18.3), showing that the innovations have their effects. In older planes, one had more complaints concerning legroom and width of the seat. For instance, in some A310 older planes nine seats were positioned next to each other, while in the same fuselage of the newer A330, rows of eight seats were installed, which results in more shoulder space. Older planes also score lower regarding flight entertainment and hygiene. It was also mentioned that newer planes were fresher and bright looking, having a strong relationship with comfort and showing that impression influences comfort. It was also interesting to see that charter flights are rated significantly lower regarding comfort compared with non-charter flights. From the trip reports it became clear that charter flights often have less legroom and shoulder space than the other flights, explaining for a large part the difference. This study also showed that a delay of several hours influenced the comfort scores so much that the ratings became around 3.5, while the average rating was 7.0 (on a scale from 0–10, where 10 ⫽ maximum comfort). Therefore, it is no use improving legroom when a company flies a route with frequent delays.

457

Crucial elements of designing for comfort

flight cancelled problems with entertainment system minimal legroom rude flight attendant

lost luggage

delay

FIGURE 18.11 From 11,513 website reports, a selection was made of the reports that contained the service and comfort rating, the airline name, information on the type of airplane of a flight between 1 May and 1 September in 2006. These 291 selected reports showed the main problems mentioned in the figure. TABLE 18.3 Differences in comfort experience of passengers caused by length of the flight, first/business class, leisure flights and old airplanes Comfort value (scale 0–10)

Long distance versus others First/business class versus others Charter flights versus others* Old airplanes versus others newer ones*

Special group

Others

p-value

6.67 7.91 6.29 5.88

7.18 6.99 7.33 7.26

0.17 0.11 0.00 0.00

Comfort could be rated on a scale from 0–10, ten being the highest comfort. The differences are tested with the t-test; p ⬍ 0.05 is seen as significant (marked with*).

Legroom From the 11,513 website reports it became clear that many passengers focus on the pitch in choosing the seat. Of course the pitch has influence on the legroom. However, new developments show that legroom can also be increased by making the seat thinner (Bronkhorst and Krause, 2005; see Figure 18.9). This has been applied by Swiss Air? for instance. Their website reports: In Swiss Business and Swiss Economy the new seats are thinner yet more comfortable than their predecessors, thanks to innovative carbon technology which makes both parts of the cabin feel even roomier. And the literature pouch on the seatback in front of you has been positioned higher to create extra legroom. (http://www.swiss.com/web/IE6/sw-op-ob-product-reconfig-a320)

Opinion on conceptual improvements For the main problems, a large number of drawings were made. Based on feasibility and effects on the problems three were selected and further elaborated. In Figure 18.12 an example of an elaboration is shown. The themes for the three designs were: More legroom; more movement; and more privacy. These designs were presented to 14 passengers flying frequently. In interviews the subjects were asked to give their preference

458

Product Experience

FIGURE 18.12 The possibility of having more legroom and the same number of seats (design I. Kamp).

and give additional comments. Regarding comfort, 42% of the passengers preferred the concept of more knee space. The concept of aircraft interior increasing knee space was found interesting as it increases comfort and would not increase the ticket price, as there was no reduction in the number of seats. A problem mentioned was that the knees of the passenger behind you could be felt. The theme ‘more movement’ was chosen by 28%. This concept of aircraft interior was meant to stimulate movement, to encourage shopping and getting a drink, but also making it possible to work with your laptop leaning. This was seen as an advantage. Another advantage of this system that was mentioned was that tickets could be cheaper, because of the larger number of passengers. The concept ‘privacy’ was chosen by 30%. This concept of aircraft interior consisted of a kind of hood to increase personal space to sleep, read or listen to music. The passengers mentioned that this would be especially useful for long-haul flights. Discussion The aircraft interior industry was successful in improving the passenger comfort experience. New airplanes are rated significantly better compared with the old airplanes regarding comfort. However, there is still room for improvement, especially in the leisure (charter) airplanes. Possibilities to innovate concern the need for more legroom and personal space. Examples show that this is even possible within the current airplanes with at least the same number of seats. Regarding point 1 ‘description’, the focus of the comfort design aspect was comfort, because it was related to attracting more passengers by giving them positive impressions that were attractive and luxurious. Also, short-term as well as long-term comfort had to be considered. This increased the complexity of the analysis and concept design process, as it had to provide a good-looking, good-feeling interior at first sight, which

Crucial elements of designing for comfort

459

had to be comfortable in long-term use as well. Regarding point 2 ‘task’, it was clear that the main task was sitting. However, it is worthwhile to study all tasks in the airplane and adapt the environment to these tasks. In a previous study (Bronkhorst and Krause, 2005) regarding a passenger seat, a significant increase in comfort experience was achieved by adapting the seat to the four most observed activities (sleeping, reading, talking and waiting). Related to point 3 ‘physical and experience characteristics’ the focus of the improvements was on the experience (in-flight entertainment, light, form, and looks) and physical environment. However, in our study we also see that regarding the physical aspects, especially knee space, improvement is possible.

6. CONCLUSION Despite the frequent use of the term comfort, there is no such thing as a general notion of comfort or discomfort. Some aspects of comfort are generally accepted: The subjective nature and the fact that the comfort experience is a reaction to the environment. There is a distinction between comfort and discomfort. For seats, discomfort is more related to physical aspects, and comfort is more related to emotions and perception of luxury. It is questionable whether this distinction is useful for all products. For instance, for hand tools the distinction between comfort and discomfort was not so clear. In the three cases, the first one primarily considered discomfort as it was related to musculoskeletal injuries. This was true for the second case as well. Additionally, comfort was considered to influence sales where there was improvement in comfort. In the third case the focus was primarily on comfort, because all aspects of flying, including a good-looking interior, were involved. All three cases showed that it was important to do the measurements on the tasks in which the comfort plays a role, and the three cases also stress the usefulness of paying attention to the underlying factors to determine the way the measurements are performed. It is important to consider comfort in the design process, as many studies on airline comfort, vehicle comfort, and hand tool comfort show the positive effects of this attention.

REFERENCES Armstrong, T. J., Buckle, P., Fine, L. J., Hagberg, M., et al. (1993). A conceptual model for work-related neck and upper limb disorders. Scandinavian Journal of Work, Environment and Health, 19, 73–84. Blok, M., Vink, P. and Kamp, I. (2007). Comfortabel vliegen (Flying comfortably). Tijdschrift voor Ergonomie (accepted for publication). Bobjer, O. and Jansson, C. (1997). A research approach to the design of ergonomic hand tools. The 11-point programme. In: P. Seppälä, T. Luopajärvi, C. H. Nygård, and M. Mattila (Eds.) From experience to innovation. Proceedings of the 13th Triennial Congress of the International Ergonomics Association, Helsinki, June 29–July 4, 1997, vol. 2, pp. 193–195. Helsinki: Finnish Institute of Occupational Health. Bongers, P. M. (2000). Risicofactoren voor lage rugklachten: resultaten van een longitudinaal onderzoek (Risk factors for lower back pain: results of a longitudinal study). Hoofddorp: TNO Arbeid. Brauer, K. (2004). Reinventing economy class – Improving passenger satisfaction and yields by better matching seating and fares. Presentation at the aircraft interior EXPO 2004, Hamburg. Bronkhorst, R. E. and Krause, F. (2005). Designing comfortable passenger seats. In: P. Vink (Ed.) Comfort and design: principles and good practice, pp. 155–168. Boca Raton: CRC Press. Bronkhorst, R. E., Kuijt-Evers, L. F. M., Cremer, R., et al. (2001). Emotie en comfort in cabines (Emotion and comfort in cabins). Rapportage TNO Basisfinanciering 2000, Team 40, Hoofddorp: TNO Arbeid. Delleman, N. J. (1999). Working postures: prediction and evaluation. PhD thesis, Free University, Amsterdam. de Looze, M. P., Kuijt-Evers, L. F. M. and van Dieën, J. H. (2003). Sitting comfort and discomfort and the relationships with objective measures. Ergonomics, 46, 985–997.

460

Product Experience

de Looze, M. P., van Rhijn, J. W., Schoenmaker, N., van der Grinten, M. P. and van Deursen, J. (2005). Productivity and discomfort in assembly work: the effects of an ergonomic workplace adjustment at Philips DAP. In: P. Vink (Ed.) Comfort and design: principles and good practice, pp.129–136. Boca Raton: CRC Press. Ehn, P. (1988). Work-oriented design of computer artifacts. Stockholm: Arbetslivscentrum. Eikhout, S. M., Bronkhorst, R. E., Vink, P. and van der Grinten, M. P. (2005). Towards a comfortable paint scraper. P. Vink (Ed.) Comfort and design: Principles and good practice, pp. 181–191. Boca Raton: CRC Press. Goossens, R. H. M., Teeuw, R. and Snijders, C. J. (2005). Sensitivity for pressure difference on the ischial tuberosity. Ergonomics, 48, 895–902. Hagberg, M. and Wegman, D. H. (1987). Prevalence rates and odds ratios of shoulder and neck diseases in different occupational groups. British Journal of Industrial Medicine, 44, 602–610. Hamberg, H. (2007). Personal communication, 19 February 2007. Helander, M. G. and Zhang, L. (1997). Field studies of comfort and discomfort in sitting. Ergonomics, 40, 895–915. Kosonen, R. (2004). Assessment of productivity loss in air-conditioned buildings using PMV index. Energy and Buildings, 36, 987–993. Kuijt-Evers, L. F. M. (2007). Comfort in using hand tools: theory, design and evaluation. PhD thesis, Delft University of Technology, Delft, the Netherlands. Kuijt-Evers, L. F. M., Krause, F. and Vink, P. (2003). Aspects to improve cabin comfort of wheel loaders and excavators according to operators. Applied Ergonomics, 34, 265–272. Kuijt-Evers L. F. M., Groenesteijn, L. de Looze, M. P. and Vink, P. (2004). Identifying factors of comfort in using hand tools. Applied Ergonomics, 35, 453–458. Kuijt-Evers L. F. M., Twisk, J. Groenesteijn, L. de Looze, M. P. and Vink, P. (2005). Identifying predictors of comfort and discomfort in using hand tools using the comfort questionnaire for hand tools (CQH). Ergonomics, 48, 692–702. Mital, A. (1991). Hand tools: injuries, illness, design and usage. In: A. Mital and W. Karwowski (Eds.) Workspace, equipment and tool design, pp. 219–256. Amsterdam: Elsevier. Proper, K. I., Bongers, P. M. and van der Grinten, M. P. (1999). Longitudinaal onderzoek naar rug-, neken schouderklachten. Deelrapport 5: Lokaal ervaren ongemak. De relatie met en de voorspelling van klachten aan het bewegingsapparaat. Hoofddorp: TNO Arbeid. Richards, L. G. (1980). On the psychology of passenger comfort. In: D. J. Oborne and J. A. Levis (Eds.) Human factors in transport research, vol. 2, pp. 15–23. London: Academic Press. Slater, K. (1985). Human comfort. Springfield, IL: Charles C. Thomas. Sonneveld, M. H. (2007). Aesthetics of tactual experience, PhD-thesis, Delft University of Technology, Delft, the Netherlands. van der Grinten, M. P. and Smitt, P. (1992). Development of a practical method for measuring body part discomfort. In: S. Kumar (Ed.) Advances in industrial ergonomics and safety IV, pp. 311–318. London: Taylor and Francis. Vink, P. (Ed.) (2005). Comfort and design: Principles and good practice. Boca Raton: CRC Press. Vink, P., Bronkhorst, R. and Goossens, R. H. M. (2006). Comfort and design: Measuring the effect. In: H.-J. Wilke (Ed.) Herausg. Ergo Mechanics 2, pp. 97–109. Aachen: Shaker Verlag. Winkel, J. and Westgaard, R. (1992). Occupational and individual risk factors for shoulder–neck complaints: Part II: The scientific basis (literature review) for the guide. International Journal of Industrial Ergonomics, 10, 85–104. Zhang, L., Helander, M. G. and Drury, C. G. (1996). Identifying factors of comfort and discomfort in sitting. Human Factors, 38, 377–389.

19

CO-EXPERIENCE: PRODUCT EXPERIENCE AS SOCIAL INTERACTION KATJA BATTARBEE IDEO, Palo Alto, CA

ILPO KOSKINEN University of Art and Design, Helsinki, Finland

1. FROM USER EXPERIENCE TO CO-EXPERIENCE Since the late 1990s, many designers have embraced the concept of experience (or user experience) as the focus of their design goal, philosophy and even methodology. Experience has been a logical extension beyond usability centred design, where meaning, pleasure and delight were long neglected by researchers and developers. However, even the improved term user-centred design was found to be restrictive – people are more than users, and designers need to make that distinction matter. Designing for experience requires awareness and empathy for sensory experience, emotion and action, as well as for the evolving values and meaning in products and their social, material and cultural contexts. Researchers have had to develop new tools and practices for designing for experience (Buchenau and Fulton Suri, 2000; Sanders 2001). Marketing literature was the first to pick up on the importance of this shift in focus (Leonard and Rayport, 1997; Pine and Gilmore, 1998) and to prove the business value of having experience as a design focus. The success of the concept of product experience has been particularly apparent in the domain of designing interactive products such as websites (see for example, Shedroff, 2001; Garrett, 2002), although consumer products have also been systematically pushed into the realms beyond usability (cf. Jordan, 2000). The same experience boom has also continued to resonate in design research, with much new work on the various aspects that extend traditional approaches towards more all-encompassing and experiential ones. These research directions are often based on an academic discipline; for example, emotions are possibly the largest research area in psychology applied to design research Product Experience Copyright © 2008 Elsevier Ltd.

461

462

Product Experience

(especially so in the human–computer interaction field). The complex and holistic nature of experiencing seems to require a broader view than any one academic discipline can support alone. However, different sciences have advantages in different stages of design. Research on emotion has focused more on evaluation of design, whereas social sciences are being adapted to the early, ‘fuzzy front end’ of design in order to understand people and products more holistically. Each discipline offers a different way of researching and defining experience, and a different set of methods and tools for the purpose. However, human experience is elusively large, and cannot fit into the framework of any discipline completely and exhaustively. The approach chosen needs to be selected to fit the needs of the particular project, addressing the available resources, the purpose of the research, and the contexts of experiencing where research has to take place. This is frequently where practical experience research and academic experience research part company, as academia has more time and the ability to publish, but often a narrow or artificial purpose, whereas a company has a very clear purpose, but perpetually suffers from a lack of time and other resources. In our view, the best experience literature has been created where these two worlds meet. This paper describes our work on the notion of user experience. Essentially, we elaborate the concept by situating it in social interaction. The paper proceeds in three parts. First, we relate our work to other work in this area. After this brief theoretical excursus, we illustrate our work through two examples. The first is more research-focused, intended to illustrate conceptual aspects of our work. The second is more design-oriented, illustrating how sensitivity to social aspects of experience can be taken into account in interpreting user research and integrating it into actual design work.

2. INTERPRETATIONS OF USER EXPERIENCE IN DESIGN RESEARCH Design researchers agree that experiencing is something subjective and private, as pointed out by Buchenau and Fulton Suri (2000), and no one can know exactly what an experience is and feels like for another person. Experiences are also unique, since each moment changes people slightly but irrevocably, making it impossible to repeat the same experience. While people’s perceptions and experiences evolve, the products that are the props and facilitators of experiences generally remain the same, or suffer from age and wear. Three main strategies can be identified for defining the relationship between design and experience, as described in detail by Battarbee (2004). These strategies may be productfocused, human-focused or interaction-focused, each implying a different approach to studying experience. The purpose of specific experience frameworks can be generative as they inform and inspire designers, but such frameworks can also be evaluative, such as the quality of user experience framework (Alben, 1996), which included aspects of the design process in its evaluation, something that end users rarely see or care much about, but which was of interest in that context. The first and simplest way is to focus on the product as the source and cause of experiences. Such frameworks may be very particular (for example, the information architecture of designing web pages by Garrett, 2002) or broad (all physical products in the ‘scene of experiences’ discussed by Jääskö, Mattelmäki and Ylirisku, 2003). The second way is to focus on people and their needs, and the kinds of experience they have and desire. People have an infinite capacity for experiencing, which products can either support or hinder. People also have universal drives and needs, such as those described by Maslow (comp. Jordan, 2000; Chapter 5). Products satisfy these emotional

Co-experience: Product experience as social interaction

463

needs. Most of these models have a psychological basis, such as needs or emotions (see Desmet, 2002). The third strategy focuses on experiencing as a process, in some ways a means of integrating both models into a timeline. A process view of the product is exemplified by Rhea’s product role life cycle (Rhea, 1992), where products can become increasingly important through events, as well as fade into oblivion. Although the product itself does not change, people and the usage context do, and so does the experience. This provides methodological opportunities for taking the process of change into account. There are a few models or approaches that have been particularly influential in the field of designing for experience. Sanders, who describes experience as a moment of action with reflection on the past and anticipation of the future (Sanders, 2003), suggests using observation for learning about the immediate present, and talking for finding out about the recent past and near future and making, i.e. constructing, artifacts of knowledge to address experiences from the past and dreams for the future. Forlizzi and Ford (2000) break down the moment of the experience into various interaction experiences, ranging from subconscious, to cognitive, to those involving storytelling. Subconscious experiences can only be observed, cognitive ones discussed and analyzed, and storytelling ones obviously unfold as meaning is attributed to the experiences through reflection. Both models show how people make sense of what they do in various ways and on various time scales through different actions, reflecting on the past and anticipating the future. While none of these models deny the importance of the presence of other people in experiencing, they treat the individual as central and only hint at social contexts by referring to storytelling. We go a step further, claiming that interacting with other people is the basis of making sense of experiences at all. To learn about product experiences, phenomena should not be studied in isolation of sensations, actions and emotions, without which meanings are inert and short-lived, lacking roots, direction and consequences. Observe these in the context of social interaction, however, and the meaning of the experiences will change and evolve in unpredictable yet consistent ways. This aspect of user experiences as social interaction, called co-experiencing (see Battarbee and Koskinen, 2004), treats experiencing as a process that is done by individuals in social interaction – experiencing is still subjective and private, but its meanings can be shared and communicated to others either implicitly or explicitly. Although people can and do engage in self-talk, and in doing so address themselves as a social object, without interaction with other people there is little reason to seek meaning in experiences or challenge an existing meaning with a new interpretation. Experiences come truly alive in social interaction.

3. A PHILOSOPHICAL DETOUR The principles of the pragmatist philosophy are to observe the world and to focus on its practical matters. This pragmatic principle is not only reserved for the focus of observation, but also to the desired end results. Pragmatic philosophy should respect and build on prior knowledge whenever possible (James, 1995, p. 56). The observations that prompted the search for the definition and concept of co-experience were of children enjoying using devices together more than alone, and coming up with more divergent and creative uses together than alone (see Mäkelä et al., 2000). Making sense of the experience was a fun social thing for them, and tied to the meanings and opportunities they discovered through the products. These observations prompted first a search through the growing body of user experience literature, and then a search for a way to learn, describe and communicate the significance of the observation. Several other field studies later, it was clear that

464

Product Experience

using and exploring together had very different qualities than using alone, and not only for children. Finding out what a device is good for is something that is quite crucial to many design research activities, especially when involved with actual design and product development work. The solution was to look for a theory that makes sense of meaning-making by individuals in social interaction and is based on observations in natural settings. Blumer’s symbolic interactionism is a theory in sociology that focused in the 1930s on the study of interaction between people and brought in field studies as the data collection method of choice (Blumer, 1968). Symbolic interactions are intentional and convey meaning – Blumer leaves out unintentional, unsymbolic ones such as reflexes. For example, a sneeze itself would not be included, but the behaviors of politeness and hygiene associated with sneezing are definitely symbolic, and used to convey meanings to others. According to Blumer, the basic principles of symbolic interactionism are: 1. That people act towards things (such as physical objects, people as well as abstract ideas) on the basis of meanings they ascribe to them. That is, for one person a chair is for sitting, while for someone else the same chair is a treasured part of a collection of Le Corbusier pieces. 2. These meanings are created in interaction with other people. When a guest informs the unknowing host that the chair is an original Le Corbusier, the host’s perception of the chair changes. 3. These meanings are handled in and modified through an interpretive process with things people encounter. When the proud host tells other visitors that the chair is a Le Corbusier, and gets compliments and hears stories about its value, he learns to appreciate the chair more. However, if another visitor points out a detail in the materials that reveals that the chair is merely a beautiful copy, the host now has to find a way to deal with the new situation and the types of disappointment – both with the chair itself and with all the people who have been part of the real versus copy experience. To interpret Blumer in terms of user experiences, there are two stages of processing an experience. One is the internal senses and feelings, and the other is deciding what they mean and how to relate to them. For example, to be able to interact with others successfully, ambiguous emotions are observed, regulated and shaped through social reflection processes that focus on the self (Rosenberg, 1990). Then, consciously or not, emotions are expressed through sentic modulation through culturally and personally determined gestures and behaviors (Picard, 1997, p. 25). In addition to the inner emotions, any message that people communicate to others comes accompanied by a host of supporting clues and behaviors that aim to direct others to understand the person in the intended way (Manning, 1992). Thus, through our behavior and by observing the meaning-rich behaviors of others, we quickly learn about the do’s and dont’s of the world. This is not to say that the end result is a consensus. Rather, the importance of this model is that although prior meanings exist, these are open to reinterpretation by anyone at any time in a continuing negotiation process. Any significant change in the situation, environment or activity prompts a re-evaluation of the meanings that people entertain. Blumer’s symbolic interactionism makes use of sensitizing concepts, which act as a scaffold for constructing understanding but, like a scaffold, are not a part of the final structure and are taken down before construction is complete. A sensitizing concept orients and supports observation and interpretation activities without dictating the end result. Co-experience is offered as such a sensitizing concept. Using the concept of co-experience can help to set up observations and identify interpretations in findings, especially when the focus is broad and fuzzy, as in the early stages of product design.

Co-experience: Product experience as social interaction

465

4. CO-EXPERIENCING MOBILE MULTIMEDIA AS A PROCESS Co-experiencing is the process of learning, maintaining, and modifying meaning in social interaction. This happens through all that people do or fail to do, non-verbal and verbal communications, and behavior. Co-experiencing consists of three key processes, or types: Lifting up; reciprocating; and rejecting. Each is discussed here to show how social interactions are crucial in the emergence and shaping of experiences with and concerning products and how these three types of action are all a necessary part of the process of co-experiencing. Implicit in this analysis is that interaction is a process that generally proceeds in a turn-wise fashion. Turns are easy to identify in a conversation between two people, but may be more difficult when observing how a group of people navigate across a crowded building. Further, the turns may be hard to spot in casual everyday exchanges, because much of everyday interaction is geared towards proceeding according to shared expectations. At the same time, best friends and total strangers share the fact that for opposite reasons they have the broadest range of choice in how they act towards one another and what meanings they choose to convey. Strangers have nothing to lose and thus have more latitude, whereas friends have come to trust each other over time and may have developed their own style of humor and their own expectations of appropriateness, which may exceed common norms and customs. However, the key is that whether people come to define something (i.e. a product experience) as good or bad, or ‘for me’ or ‘not for me’, depends on these processes of interaction. Experiences change, evolve, fluctuate, and grow in social interaction, and people are also quick to learn and observe from the experiences of others. Social interaction lifts things out of fluent, ordinary experience, keeps them as focal points of experience, and then removes them from common focus. Social interaction largely explains how things migrate between the levels of experience described by Forlizzi and Ford (2000).

4.1. Lifting up How is it that people move from a state of just doing and being to a state of reflecting and describing what they are doing or have done? How is the ongoing flow of doing and responding crafted into a meaningful, describable message? The argument here is that this process of lifting up happens in social interaction, since people find occasion through encountering others to inspect their experiences and tell them about their experiences. How does lifting up happen? People communicate many things at once, even in the simplest, smallest gestures and exchanges. Social interaction is built on turns and turn taking, which builds on what was said and done before, following loose rules of maintaining topic and relevance to the previous turn. The initial emotional response is usually only a trigger, and the actual story is told so as to interpret the situation or event appropriately for the recipient. The sociologist Erving Goffman refers to this as ‘impression management’ (Goffman, 1958). For example, in Figure 19.1, Maria has just taken a picture of herself with her camera phone, and sends it to Liisa, her friend. Her pose is nondescript, a point she elaborates in the text. We can interpret this message to mean that she labels her mood as boredom as she shares it with a friend. More importantly, she also turns expressing mood into a possible topic of discussion as she lifts her own experience up to Liisa. Breakdowns and surprises are what make good stories for people to share easily with designers (Erickson, 1995). At the lower level, some emotional responses can be extremely fleeting; for example, certain facial expressions may be as short as 125 milliseconds

466

Product Experience

FIGURE 19.1 A picture of mood.

FIGURE 19.2 Replying to the picture of mood.

(Hatfield, Cacioppo and Rapson, 1994, p. 19). At a higher level, the changes in reporting or communicating emotions can become more intentional: Something frustrating may be used humorously to entertain others or may be communicated to seek attention and sympathy. In other words, people interpret and communicate their emotions in ways that offer a desired picture of self to others, and that offer other people desired ways to respond. There are two kinds of response that either align with or challenge the interpretations offered or desirable behaviors. For co-experience, we look at the extreme ends of these responses as reciprocating and rejecting the interpretation.

4.2. Reciprocating Individual things and events are constantly lifted up to shared attention without any longlasting impact. If we want to talk about co-experience properly, we need to focus on those things that are focused on in interaction. We call this process reciprocation. Reciprocation is a positive response to something that has been lifted up. The socially expected norm is to respond to a gesture with a like gesture, to acknowledge, or even replicate the type of experience or thought another person has offered. Reciprocating not only aligns with the interpretation offered, but also responds with a similar one. Many ritualistic interactions, such as those depending on the concept of gift giving, centre on reciprocity, but are apparent in any small greetings and favors as well (see Taylor and Harper, 2002). In talking about product experiences, people often find a story to tell that relates to a similar kind of experience or similar interpretation, a ‘that reminds me of’ type of story. In product experiences, reciprocation also builds on the work of others. One person may experiment with a product and come up with a clever use – someone else may accept this, take the idea and modify it even further, accepting and taking the idea as their own. The process is a creative one. Figure 19.2 gives Liisa’s response to Maria’s message displayed in Figure 19.1. She responds to Maria with another gloomy face and a rhetorical question that shows that she is perhaps not in the best of moods either. However, she also manages to do several

Co-experience: Product experience as social interaction

467

things in her message. For instance, her response has a humorous, personal tone, which suggests that Maria’s previous message was not that serious, or that it should lighten up a little. Thus, her message is a response, but it does many other things as well. Of course, this is Blumer’s point: In interaction, experiences are taken up, twisted, interpreted, and recast in many ways, none of which are final or conclusive and which can be only judged against the responses and interpretations they invite from others. The process is not mechanical, but dependent on the participants’ wit in doing things. Especially in the case of technology that facilitates communication, the need for reciprocating in social interaction blends seamlessly with the search for the meaning and purpose of technology. This is very clear in communication technologies and products, which are both part of your surroundings, but also facilitate interaction with others further away (see for example, Battarbee and Koskinen, 2004). When a product or technology allows or even encourages reciprocation, it creates an instant platform that allows people to explore its meaning together. In this sense, reciprocation is the key element in co-experiencing. Reciprocation is also something that allows feelings of intimacy and closeness to be expressed and shared, with or without products involved (Battarbee et al., 2002). Things that might discourage reciprocation are sometimes subtle: A cost that is perceived to be too high, a task sequence that requires too much time and attention or any other failure along the way, such as delivery or connectivity that is unreliable or of unacceptably poor quality. Plain usability problems can also stump co-experiencing. The hindrances may also be of social quality, a perceived mismatch between the relationship and communication purposes and the perception of what the technology or device is good for. However, the search for what a technology is good for is an ongoing process, not a static state or a clearly definable time span. As the study by Muller et al. (2003) of a communication product (instant messenger) shows, people’s usage patterns, the number of people they connect with, and the purposes of use grew and broadened over the study period of two years, which is much longer than most design studies would ever last. However, while reciprocation is generally a positive force in co-experiencing, people may also amplify each other’s negative interpretations, creating a vicious circle in which they reinforce each other’s negative observations and judgments of a technology or an experience. The way a product or an experience falls out of grace within a group of people may be surprisingly quick and catastrophic.

4.3. Rejecting Experiences do not remain in joint attention forever. Most are simply passing moments in the stream of life. There are many reasons for this. Experiences are sometimes bound to events that have a natural end, while at other times they may fade because other things show up that capture the participants’ imagination. In most cases, co-experiences have a short lifespan simply because people return to things they were doing previously. However, occasionally things get more complicated, and people have to actively choose to end something that otherwise will not stop or that is no longer desirable. For example, behavior and interpretation may go beyond the bounds of propriety, and the experience has to be actively rejected. We have studied two types of rejection in our work. There is passive rejection, in which case a possible line of interpretation is ignored in favor of another. Then there is active rejection, in which people indicate to others, subtly and implicitly or bluntly and explicitly, that they do not like or approve of their interpretations and actions. The problem is that active rejection is always a loss of face in interaction that requires a repair action in order to restore the situation, even if the chiding or rejection is gentle.

468

Product Experience

Maintaining face (as discussed extensively by Goffman, 1967) is a key motivator for everyday social interaction, which is always a fine balance between keeping interactions pleasantly predictable and asserting self for personal gain at the expense of the interaction. As Goffman also mentions, the severity of the loss of face depends on the social situation and the ability of people to deal with the break – the problem with these situations is that people do not know how to behave and feel awkward, embarrassed or angry. Goffman describes many tactics that people employ to deal with this, ranging from ignoring the situation to exclaiming ‘oops’, apologizing, making fun of themselves before others can, drawing attention away from the problem to something else, or when all else fails, cutting their losses by ending the interaction and breaking up the situations by changing the focus onto actually dealing with the cause of the disruption. Each product and new technology has to adapt and find its accepted uses and behaviors. It is worth noting that all the publications and announcements of mobile phone etiquette were and are geared towards encouraging politeness to those in the vicinity of the user of the phone. People had to learn through trial and error when it is acceptable to receive or make a call and how one should excuse oneself from others, and sometimes made these rules explicit for all. In a meeting space at IDEO, the rules of brainstorming are posted in beautiful lettering on the wall. An extra rule, ‘No cellphones’, has been posted with a marker on a sticky note. Similar messages can be found in restaurants, cinemas, meeting spaces, and hospitals. Communities find ways to deal with new challenges through trial and error. The particular processes of co-experiencing can be revealed and understood by observing how people lift up, reciprocate and reject experiences with each other, and by empathically understanding what and why they do so. In many cases, however, organizing such observations is not straightforwardly easy – observing social interaction requires analytical skill and observational tact, and co-experience prototyping requires technologically advanced prototyping or clever use of analogous experiences and products.

5. MORPHOME: DESIGNING FOR CO-EXPERIENCE WITH PROTOTYPES The primary reason for coining the term co-experience instead of preferring longer and more precise expressions is to provide designers with a convenient shorthand for paying attention to the social context of experience as they interpret what they learn and see. Our methodological proposal rests on Buchenau and Fulton Suri’s notion of experience prototype (2000), but elaborates particular aspects. In designing with co-experience in mind, following a few principles is more important than developing a highly sophisticated functioning prototype to represent the product experience. The following paradigm describes the conditions required for studying social interaction for the purposes of design (Battarbee, 2004; Kurvinen, Koskinen and Battarbee, submitted for publication). • Ordinary social setting. More than one person has to be involved to create the conditions for social interaction, which has to take place in a real context, not in a studio or a laboratory. • Naturalistic research design and methods. The research setting has to be naturalistic: People have to be able to author their own experiences to allow for creativity. Data from people must be gathered and treated using empirical and up-to-date research methods, and requires that several methods or tools be used in parallel, because the complexity of experiencing cannot be captured by one method alone. • Openness. Research design has to be open: The prototype should not be thought of as a laboratory experiment, but rather as an intervention. The designer’s task is to

Co-experience: Product experience as social interaction

469

observe and interpret how people use, explore, and create, experiences with the prototype, which is still undefined to its new users and contains many possible meanings and interpretations. • A sufficient time span. The prototype usage ought to be observed for long enough, typically for a few weeks at least, since it is difficult to get an idea of how people explore and redefine the technology in their actions where the study period is shorter. The usage time needs to reflect frequency of use and social interaction. • The sequential unfolding of events. Designers have to pay special attention to how events unfold over time and in context, and what may hinder or enable people’s ability to co-experience. The intention of this setup is to create conditions in which a social organization involving the representation (i.e. co-experience prototype) emerges, so that this organization (i.e. meanings that are interpreted from interactions with the prototype) can be observed and described in detail. This understanding can be used as a driver in design, and may even be modeled. The key thing in creating a research design for prototyping co-experience is that people must have time and opportunity to create meanings to prototypes together with other people, i.e. to lift these things into attention and then let attention wane (Sengers and Gaver, 2006). The next section shows that it can in fact be relatively easy to build prototypes, even of future technological visions, and situate them in a social setting to see what courses they may come to follow in that setting. These examples show that people pay attention to new things in their environment and elaborate their experiences with them with other people. That is, using co-experiencing terms, they lift things to each other’s attention, reciprocate these experiences, and may even come to contest them under certain circumstances. The following section is based on a study called Morphome, which focused on socalled proactive information technology in the home. In a nutshell, proactive technology is ambient technology that gathers data from people using sensors, and uses this data to react to events pre-emptively, or in any case faster than a human could or would, thus doing things for people in a proactive fashion (Tennenhouse, 2000). Proactive technology suggests the promise of creating a calm future environment in which people do not have to continuously give commands to electronic devices (Want, Pering and Tennenhouse, 2003; Weiser and Seely Brown, 2004). In Morphome, several different types of prototype were used to study the interaction and design issues of proactive technology in the home, with each turn experimenting with prototypes, collecting and interpreting qualitative data (see Seale, 1999), and developing better hypotheses for design. It is important to keep in mind that the study was not about testing ideas with data, but rather about designing representations to be used as social interventions in the highly sensitive home environment, to learn about co-experiences with the prototypes and thus to inform the design of the next set of prototypes. The perspective of the research matters more than actual methods or theoretical assumptions. In industrial work, in which it is not usually possible to spend months on gathering and analyzing data, researchers may have to rely on ‘discount’ methodologies and streamlined or abbreviated studies. In Morphome, we installed our prototypes in real homes using the paradigm described above, but gathered participants’ experiences in interviews and scenario-assisted interviews rather than using ethnomethodological or even ethnographic methods. The aim is not to argue that co-experience must be studied with any particular methodology, since methods of study need to be adapted to the situation in hand. The importance lies in creating the right conditions for co-experiencing to happen.

470

Product Experience

5.1. Design prototype: The ‘IKEA’ style study and system scenarios The ‘IKEA’ style study was conducted by placing a design prototype in the home context for a period of time. The prototype was an IKEA-esque lamp that reacted to sound levels, and thus to particular actions and behaviors of people in the home. The lamp had four 36 watt light tubes (two colored, two normal) and four states: A normal lamp with button for adjusting light; a 10-minute cycle in which colors changed from warm to cold; a sensor designed to keep light constant in the lamp’s surroundings; and a state in which red and blue LEDs reacted to sounds. It was also possible to attach other electrical devices to the lamp. The design was deliberately simple in technical terms, since the aim was only to provide people with the experience of what living with proactive technology might be like. The logic is that the design would thus be unremarkable enough for people to focus their attention on the behavior of the lamp instead of thinking about it as an artistic object (Figure 19.3). The prototype functioned as it was designed to do. However, since our technology was simple in comparison to Tennenhouse’s vision of proactive technology, we took the participants’ experience with the lamp only as a starting point, and enriched the study with scenario interviews (see Carroll, 2000) in which we probed both technology and social action in more detail. In these scenarios, the experience of participants with the lamp was situated in various technological and use situations. As expected, people were willing to delegate some of their ‘dirty work’ to the lamp installed in the network, and let it automatically control some functions. People related some of their real life experiences to the system concepts, suggesting for example that a vacuum cleaner should power down when the phone rings. The design prototype of the lamp did not contribute in an identifiable way to the participant’s ability to evaluate the scenarios and build on them. However, the situation changed as we started to explore how proactive technology would affect social affairs. For example, we asked whether the lamp would be an appropriate feedback mechanism about sound level, whether its behavior should change in the course of the day, and what kind of feedback it could give people about sound. In this case the experiences with the lamp in the home had already provided some

FIGURE 19.3 Left: The IKEA Lamp Design: electronics and its states. The UI is the upper part of the picture on the left. Right: Examples from scenarios. Top: the priming question sent a week earlier, the sound world of the home (preparatory question); Middle: the lamp reacting to sound, Memory Trace (a lamp that remembered who had been in the room, and communicated that to newcomers); Bottom: the lamp in social context, the lamp as a control for other technology. Scenarios drawn by Kristo Kuusela.

Co-experience: Product experience as social interaction

471

co-experience opportunities, and participants were able to judge both positive and negative repercussions. In particular, people thought it would be a nice, playful addition to the home in situations like entertaining guests. However, when we probed what people would think about a lamp that had a Memory Trace, a function which dimmed the light (the warm chair effect), and whether they would allow it to be linked to a home-wide network, people became reserved, the particular issue being privacy. We originally explored the idea whether the lamp could sense the presence and absence of people, and communicate by, for instance, getting increasingly dim over time when no one was present. However, people thought that this solution would simply translate into a form of surveillance technology. They also were worried about how neighbors and criminals would use cues from the lamp for the wrong purposes. In brief, when we raised social issues into the discussion using our scenarios, people felt that they were important in whether they would accept technology or not. These scenarios certainly indicate that reasoning about social affairs does enter into discussions of proactive technology. This social reasoning has both positive and negative aspects. That is, even if we cannot predict exactly how people might experience proactive technology, our interviews show that we cannot neglect social aspects of experience – co-experience in our terminology – in designing proactive systems. People saw proactive technology in social terms, sometimes in terms of real, sometimes imaginary, meanings. Some things they see as interesting in individual terms may seem less desirable when they think about them in a social context, and vice versa, illustrating the complexity of the home environment. As designers, we should pay attention to ways in which social meanings affect how things are lifted to attention, and what kinds of terms are used in this process.

5.2. The system prototype: Living inside a proactive home as co-experience Just how important co-experience may be in designing proactive systems is well illustrated in the third phase study of the system prototype. The participating homes were fitted with sensors and programmable behaviors using the X10 home automation system (www.x10. com), which uses existing electrical cables for communication between devices. The software was replaced by flexible open source software called Misterhouse (MH; www.misterhouse.net). MH combines the X10 hardware with a PC and offers a simple user interface, as well as some basic means for programming and essential object libraries. The logic of events and functions was programmed in Perl. The X10/Misterhouse study probed what it might be like to live in an environment that has proactive features. Figure 19.4 shows how lamps, the sound environment, and coffee makers were linked to each other in one case (details of the floor plan were changed at the request of the participant). In X10/Misterhouse, we went beyond just having a collection of the IKEA lamp style individual devices, and tried to convey the feeling of what it would be like to live in a system in which objects communicate with each other. To that end, we built experience prototypes (Buchenau and Fulton Suri, 2000) for two activities, waking up and going to sleep, and tried to program them into the system. Table 19.1 shows how our prototype modeled the waking up behavior of a proactive system. In interviews conducted after a two-week field period, participants confirmed that the routine had worked well. The key factor was that the process of waking up in most cases has a clear and unambiguous starting point, the alarm, which, furthermore, is easy to attach to a more encompassing electronic system. In terms of collective and individual variation, waking up is a far simpler and more repetitive routine than going to sleep, and in consequence is relatively easy to model. The system and ordinary practices were in far better correspondence with each other.

472

Product Experience

FIGURE 19.4 An example of a floor plan showing the placement of devices connected to the proactive home system. The operational elements are named in the floor plan. TABLE 19.1 The operation of the wake-up routine in each state is explained State

Function

Wake-up activated

The wake-up time has to be programmed prior to entering the state. The wake-up routine is ready to be started as the time condition is filled.

Preliminary state

The wake-up routine will start as the state is being entered. The bedroom lamp and the living room lamp brighten linearly for 15 minutes up to 50% of the maximum. Ambient sound is played in the bedroom and in the living room. The volume is first low, but slowly increases up to 50% of maximum.

Running state

The coffee-maker is switched on. The lighting of the bedroom and the living room, and the sound volume are further increased up to the maximum level.

Fading state

The lighting power of the bedroom and the living room lamps, and the sound volume level are decreased slowly.

Snoozing state

When this state is being entered, lights and sound fade. When the program jumps to the beginning of the wake-up routine, lighting and sound volume levels are increased. The snoozing state is on until the snoozing timer is expired.

Wake-up deactivated

The bedroom and the living room lamps, the coffee-maker, and all sounds are switched off.

The second system modeled (Table 19.2) was to support going to sleep by proceeding to dim the lights slowly and create a soothing sound world, offering a convenient environment for going to sleep. This proved far more difficult to program for two reasons. Unlike waking up, going to bed and falling asleep does not have a clear-cut starting point. There is also tremendous variation in this sequence, not just between homes, but also from one evening to the next. Even though this system for falling asleep worked out

Co-experience: Product experience as social interaction

473

TABLE 19.2 The operation of the going to sleep routine State

Function

Sleep activated

The sleep time has to be programmed prior entering the state. The sleep routine is ready to be started as the time condition is filled.

Preliminary state

The going to sleep routine will start as the state is being entered. The bedroom lamp and the living room lamp are turned on at the beginning (in the case they are not already on) and they are dimmed linearly for 15 minutes to 50% of the maximum lighting power. Ambient sound (sea, waves) is played in the bedroom and in the living room. The sound volume level is higher at the beginning, but is slowly decreased to 50% of the maximum.

Running state

The lighting power of the bedroom and the living room lamps and the sound volume are further dropped to the minimum level.

Sleep deactivated

The bedroom and the living room lamps and all sounds are switched off.

fine for individuals, the real complications followed from co-experience. How should we take complexities in social action into account; for example, the fact that people seldom act as a team, more often pursuing several activities in the same space, exhibiting various degrees of involvement with each other’s ongoing agendas? When people reflected on the social aspects of their experience with the prototype, they were able to formulate their experience in terms of rejection. Thus, in an interview, Vera, a participant, described the system as patronizing. Once such feeling was expressed and generalized in Vera’s family, and the system lost a good deal of its initial appeal: This procedure for falling asleep was in fact rejected in all the homes. The line between patronizing was crossed (laughs). I don’t want a system to tell me when I must go to bed. I know when I want to do that. The function that reminded me about it was unnecessary. (Vera, June 2005)

In consequence of opinions like these, always reciprocated in our family interviews, the going to sleep sequence was largely rejected. Although the idea of a proactive system aroused interest and did not scare people once they became familiar with the X10/ Misterhouse installation, this particular sequence was rejected unanimously. The contrast with waking up, an activity that has a far clearer sequential structure, was clear. Although similar indications could have been achieved in scenario interviews, the definiteness of the rejection was unmistakable once it had been co-experienced. Again, when we studied our experience prototype of a proactive system in terms of co-experience, we saw how it colored the way in which people relate to that technology. In reflecting their experience, people thought about it not just from their own, personal experience, but also in terms of how it would affect other people and joint action. When we used co-experience as a sensitizing term in our analysis, we had to pay serious attention to many problems inherent in proactive technology, ranging from simple things like people having different bed-times to more complex ones, like people thinking they have a right to decide for themselves what they want to do without technological interference.

6. DISCUSSION Throughout our work, we have taken it as an axiom that experience takes place in a social setting. We have coined the term ‘co-experience’ as a convenient shorthand for

474

Product Experience

this feature. In theoretical terms, this term is an elaboration of the user experience model introduced by Forlizzi and Ford (2000), which is mainly indebted to the pragmatic philosophy of John Dewey (1934). However, since our aim has been to push the social grounds of experience to the forefront, we have built primarily on Herbert Blumer’s thinking (1968) in our attempt to understand co-experience as social action. Shifting attention to the social grounds of experience has an additional methodological benefit. Unlike cognitive states, social action is directly observable. It can be studied by simple means without recourse to complex, contested theories of, say, how the brain functions or wearable technological devices for measuring and monitoring the body’s various states. As to the methodological aspects of our work, we are not promoting a major change in prototyping practices, but calling for a research paradigm (in Robert Merton’s 1968 sense) in design that differs from the experimental line that dominates in usability research and in studies of user experience. The three main differences are: 1. First, there has to be a theoretical framework that sensitizes the designers to how this social organization takes shape, and how it affects the way in which people experience designs. Although our work builds on symbolic interactionism (Blumer, 1968), other options for a theoretical foundation exist. One such is the well tried-out ethnomethodological and activity theoretical work in Computer Supported Collaborative Work (CSCW) (see e.g. Nardi, 1997; for a more elaborate statement, see Kurvinen, Koskinen and Battarbee, submitted for publication). The primary reason for having such a framework is that existing models of user experience do not prepare us well to see social organizations and processes, and few designers are trained even in elementary sociology. 2. The main features of research design have to make sure that there is social action to be observed. In creating prototypes, attention has to be paid to social aspects of experience. Prototypes must be considered in the context of a natural social organization that has to have time to evolve through opportunities for social interaction and communication over enough time to get through the ‘honeymoon’ phase of initial excitement. 3. Finally, there has to be an interpretive procedure for making sense of data. The main aim is not to ‘test’ whether the designers’ ideas about the design were right or wrong, but to install the design into a real world, and then see what happens to it, how people make sense of it, and how people make sense of their experience of it in interaction with others. More specifically, designers need to understand what kinds of thing people lift up from their experience to be reciprocated in discourse, and how they come to elaborate and modify these meanings in the course of interaction. Instead of treating social action as just an additional variable in models of user experience, we propose to take it as the starting point of exploration, and study co-experience with prototypes first, and individual variations second. It is important to note that we do not say that designers ought to follow one particular framework or methodology in studying co-experience, since one can use any number of design methodologies to do so. Similarly, one can rely on various frameworks for understanding it. The most important thing is that there be a sensitizing framework that helps designers in making sense of the intricacies of interaction and what it does to experience, and methodology that supports this study. For instance, in our own work, we have been relying on two theoretical frameworks, symbolic interactionism (see Battarbee 2004) and conversation analysis (Koskinen and Kurvinen, 2005; Kurvinen, 2007). In methodological terms, we have been gathering actual multimedia messages, but we have also been using interviews in our studies. In our work, however, we have always had a prototype that has been placed in a social setting. The analysis has consistently focused on social action and how it takes shape in the context created by the prototype.

Co-experience: Product experience as social interaction

475

The value of the Morphome project studies is that it is possible to conduct a discount version of co-experience. Ideally, at least from an academic point of view, the study of co-experience requires very detailed data on how people interact with each other in the context created by the prototype. This was the case with our study of mobile multimedia, in which we were able to collect actual messages as they where sent by 25 users (see Battarbee and Koskinen, 2004; Battarbee, 2004). The mobile multimedia study could have been more in-depth had it been supported by observations in the field or follow-up interviews, but the data itself creates substantial data on the context and people’s relationships. Similarly, in an ideal situation, the analysis of this data follows well-tried scientific protocols. For example, the mobile multimedia data has been analyzed from an ethnomethodological perspective (see Koskinen and Kurvinen, 2005). Finally, our point is conceptual, aimed at advancing a shift in design thinking rather than suggesting something totally new for the most advanced design practice. The leaders in design research (see Sengers and Gaver, 2006), as well as leading design companies (see Buchenau and Fulton Suri, 2000), follow a research paradigm that is much the same as the one described in this chapter. The approach advocated by us can easily be adapted to researching, say, interaction with robots or intelligent textiles. If for practical reasons one can do only one prototype, then it is wise to conduct research early on in the design process when design drivers are still open. However, as our examples have shown, research can be conducted at considerably later stages of the design process as well: In our study, the notion informed all stages of our iterative research. In the final analysis, the purpose of paying attention to co-experience is not so much about saying what the future product or system should be like in its details. Rather, it is about providing a more sensitive description of the social phenomena to inform designing technology.

ACKNOWLEDGMENTS We would like to thank the mobile phone operator Radiolinja for funding the original work that led to the concept of co-experience, and Esko Kurvinen for working with us in the Radiolinja project. Another debt is to Academy of Finland, which funded Morphome. The Morphome group consisted of several researchers. We would like to thank Kristo Kuusela, Anne Soronen, Jussi Mikkonen, Mari Zakrzewski, as well as Frans Mäyrä and Jukka Vanhala for cooperation in Morphome. Further, we would like to acknowledge our debt to pioneers in the field of user experience, especially Jane Fulton Suri, Jodi Forlizzi, Liz Sanders, to our most immediate coresearchers Turkka Keinonen, Tuuli Mattelmäki and Salu Ylirisku, as well as to our many other collaborators past and present.

REFERENCES Alben, L. (1996, May/June). Quality of experience. Defining the criteria for effective interaction design. Interactions 3(3), 11–15. Battarbee, K. (2004). Co-Experience. Understanding user experiences in social interaction. Helsinki: University of Art and Design Helsinki. Battarbee, K. and Koskinen I. (2004). Co-Experience: User experience as interaction. Co-design, 1, 5–18. Battarbee, K., Baerten, N., Hinfelaar, M., et al. (2002). Pools and satellites – intimacy in the city. Proceedings of Designing Information Systems DIS’00, pp. 237–245. New York: ACM Press. Blumer, H. (1968). Symbolic interactionism. Perspective and method. Berkeley: University of California Press. Buchenau, M. and Fulton Suri, J. (2000). Experience prototyping. Proceedings of Designing Information Systems DIS’00, pp. 424–433. New York: ACM Press. Carroll, J. (2000). Making use. Scenario-based design of human–computer interactions. Cambridge, MA: MIT Press. Desmet, P. M. A. (2002). Designing emotions. Delft: Technical University of Delft. Dewey, J. (1934). Art as experience. New York: Pedigree.

476

Product Experience

Erickson, T. (1995). Notes on design practice: Stories and prototypes as catalysts for communication. In: J. Carroll (Ed.) Scenario-based design. Envisioning work and technology in system development, pp. 37–58. New York: Wiley. Forlizzi, J. and Ford, S. (2000). The building blocks of experience. An early framework for interaction designers. Proceedings of Designing Information Systems DIS’00, pp. 419–423. New York: ACM Press. Garrett, J. J. (2002). The elements of user experience. User-centered design for the web. New York: AIGA and Pearson. Goffman, E. (1958). The presentation of self in everyday life. London: Penguin. Goffman, E. (1967). Interaction ritual. New York: Pantheon Books. Hatfield, E., Cacioppo, J. T. and Rapson, R. L. (1994). Emotional contagion. Cambridge, MA: Cambridge University Press. James, W. (1907/1995). Pragmatism. In: R. B. Goodman (Ed.) Pragmatism – a contemporary reader, pp. 54–75. New York: Routledge. Jordan, P. (2000). Designing pleasurable products. London: Taylor and Francis. Jääskö, V., Mattelmäki, T. and Ylirisku, S. (2003). The scene of experiences. In: The good, the bad and the irrelevant: the user and future of information and communication technologies. COST269 conference proceedings, pp. 341–345. Helsinki: COST and Medialab UIAH. Koskinen, I. and Kurvinen, E. (2005). Mobile multimedia and users: The domestication of mobile multimedia. Telektronikk, 101(3–4), 60–68. Kurvinen, E., Koskinen, I. and Battarbee, K. Prototyping social interaction. Design issues (submitted for publication). Kurvinen, E. (2007). Prototyping social action. Helsinki: University of Art and Design Helsinki. Leonard, D. and Rayport, J. E. (1997). Spark innovation through empathic design. Harvard Business Review, 75(6), 102–113. Mäkelä, A., Giller, V., Tscheligi, M. and Sefelin, R. (2000). Joking, storytelling, artsharing, expressing affection: A field trial of how children and their social network communicate with digital images in leisure time. Proceedings of Computer–Human Interaction CHI’00: Human factors in computing systems, pp. 548–555. New York: ACM Press. Manning, P. (1992). Erving Goffman and modern sociology. Palo Alto, CA: Stanford University Press. Merton, R. (1968). Manifest and latent functions. In: R. Merton (Ed.) Social Theory and Social Structure: Enlarged Edition. New York: The Free Press. Muller, M., Raven, M. E., Kogan, S., Millen, D. R. and Carey, K. (2003). Introducing chat into business organizations: Toward an instant messaging maturity model. Proceedings of GROUP’03, pp. 50–57. New York: ACM Press. Nardi, B. (1997). The use of ethnographic methods in design and evaluation. In: M. G. Helander, T. K. Landauer and P. V. Prabhu (Eds.) Handbook of human–computer interaction II, pp. 361–366. Amsterdam, The Netherlands: Elsevier Science. Picard, R. (1997). Affective computing. Cambridge, MA: MIT Press. Pine, B. J. II and Gilmore, J. H. (1998). Welcome to the Experience Economy. Harvard Business Review, (July/ August) 97–105. Rhea, D. K. (1992). A new perspective on design. Focusing on customer experience. Design Management Journal, 3(4), 40–48. Rosenberg, M. (1990). Reflexivity and Emotions. Social Psychology Quarterly, 53, 3–12. Sanders, E. B-N. (2001). Virtuosos of the experience domain. Proceedings of the 2001 IDSA Education Conference. Retrieved 7 November 2006 from http://www.maketools.com Sanders, E. B-N. (2003). Design for Experiencing: Scaffolds in communicational spaces. Design for communicational spaces conference proceedings. Retrieved 19 January 2004 from http://www.ualberta.ca/ COMSPACE/coneng/html/papers/Sanders.doc Seale, C. (1999). The quality of qualitative data. London: Sage. Sengers, P. and Gaver W. (2006). Staying open to interpretation: Engaging multiple meanings in design and evaluation. Proceedings of Designing Information Systems DIS’06, pp. 99–108. New York: ACM Press. Shedroff, N. (2001). Experience design. Berkeley, CA: New Riders Publishing. Taylor, A. S. and Harper, R. (2002). Age-old practices in the ‘New World’: A study of gift-giving between teenage mobile phone users. Proceedings of Computer–Human Interaction CHI02, pp. 439–446. New York: ACM Press. Tennenhouse, D. (2000). Proactive computing. Communications of the ACM, 43(5), 43–50. Want, R., Pering, T. and Tennenhouse, D. (2003). Comparing autonomic and proactive computing. IBM Systems Journal, 42(1), 129–135. Weiser, M. and Seely Brown, J. (2004). The coming age of calm technology. PhysComp Notes: Working notes on physical computing and embedded networking, Feb. 04, 2004. Retrieved 17 February 2005, from http://stage.itp.tsoa.nyu.edu/~tigoe/pcomp/blog/archives/000373.shtml.

20 AFFECTIVE MEANING: THE KANSEI ENGINEERING APPROACH SIMON SCHÜTTE

SHIGEKAZU ISHIHARA

Linkoping University, Linkoping, Sweden

Hiroshima International University, Hiroshima, Japan

JÖRGEN EKLUND Linkoping University, Linkoping, Sweden

MITSUO NAGAMACHI Kyushu University, Japan

1. INTRODUCTION Today’s products are often sold in saturated global markets in competition with many similar products. Each product therefore needs to offer features which make it distinguishable and attractive. Customers consider many aspects when deciding which product to buy, for example price, performance, features, ergonomics, brand, etc. One way of improving market shares on mature markets full of many similar products is to make a product ‘edgy’, i.e. easily recognizable and typical for the brand. Previously, industry coped with these demands through more active integration of the customers’ opinions in the designing phase. In the 1950s and 1960s the quality movement was born. Although functional aspects were in focus in the beginning, usability and intangible product characteristics soon became more important in the early 1980s (Childs, 2004). Customers do not base their choice on only logical reasoning, and the soft values, affections, emotions, and meanings play an increasingly important role in the purchase decision. Today’s development thus goes towards integration of affective meaning in products. Products must appear unique, reflecting an individual lifestyle. Also, if the customer has the choice between products from different manufacturers, which are equivalent in price and performance, a consciously built in ‘good feeling’ can trigger the final purchase decision. This can also be seen in today’s trends. Strong ‘super-trends’ such as hedonism, spirituality, downsizing and individuality (Jordan, 2002) abandon the traditional focus on functionality and concentrate on ‘softer’ issues such as hedonic ergonomics in design and pleasurable products and interaction (Helander, 2003). Product Experience Copyright © 2008 Elsevier Ltd.

477

478

Product Experience

Integrating such factors in product development means in the first place shifting focus from functional needs towards affective needs of the customers. Cars, furniture, jewellery, watches, etc., do not only have practical value, but also have affective properties giving the owner of products a good feeling and self-esteem. Kano, Seraku and Takahashi (1984) point out that the functional properties must be fulfilled as well, although they are not perceived as attractive attributes but more as ‘must-have’ attributes. In the beginning of the 1970s, the concept of Kansei Engineering was introduced in Japan (Nagamachi, 1989; Nagasawa, 2002). Kansei Engineering can be defined as a product development methodology, which translates customers’ and users’ feelings, impressions, and emotions into concrete design parameters. This chapter aims to give a background to Kansei Engineering and examples of how it may be used. Some of the application areas that will be covered are quality feeling, driving feeling of vehicles, and the subjective impression of comfort and safety.

2. KANSEI AND CHISEI Impressions, feelings, and emotions are often a result of external sensory stimuli. There are differences in the importance of the senses. Sight is often considered as the most important human sense. The other senses of hearing, smell, taste, and touch are usually used in a complementary manner. There are general models ranking and arranging them in accordance to the frequency of usage and importance (Schifferstein, 2006). In Japanese, the concept of sensing a situation or an artifact and building an individual emotional response is called the Kansei. The term ‘Kansei’ itself is a multi-faceted expression that does not have a complete equivalent in the English language. There are many different approaches to translating the word Kansei (compare Schütte, 2005). In the context of product development, the Kansei can be referred to as ‘the impression somebody gets from a certain artifact, environment or situation using all her senses of vision, hearing, feeling, smell and taste, as well as her cognition’. Lee, Harada and Stappers (2002) developed a complementing model where they clarify the role of the Kansei and its counterpart ‘Chisei’, which roughly can be translated as ‘reason’. Whereas the concept of Kansei is closely connected to affective, emotional values of human beings, Chisei ‘works to increase the knowledge or understanding which is matured by verbal descriptions of logical facts’ (Lee et al., 2002). Both have in common that they are triggered by a sensory input, which is mapped from both perspectives. The Kansei then builds affection, feelings, and emotions, which in turn lead to creativity; the Chisei or reasoning builds logics, recognition, and understanding which then become knowledge. Figure 20.1 displays this. C r e a ti v i t y

Affection feeling emotion

Sensory Input

Kansei Mapping Chisei

Building

Trigger Logice recognition understanding Knowle d g e

FIGURE 20.1 Model of the Kansei/Chisei (modified from Lee et al., 2002).

Affective meaning: The Kansei Engineering approach

479

3. INTRODUCING KANSEI IN COMMERCIAL PRODUCT DESIGN Designing affective meaning into commercial products is not a new idea. In fact, many industrial companies have done this for a long time. Methods have been developed and applied in product development such as quality function deployment (QFD), brainstorming techniques, etc., aiming to integrate affective meaning into new products. Other methods gather, rate, and assemble the emotions the users have of certain products (e.g. focus groups, interview techniques, survey techniques, etc.). The role product designers play in using those techniques is to merge together the customers’ and other stakeholders’ demands, (new) technical solutions, and their own ideas into new innovative products. In most companies this is done based on the experience and preferences of the product development staff and their interpretations of the customers’ desires. Often this process is considered to be more of an art than engineering or science. This is due to the fact that no rules of ‘how to do it’ are documented and the result is not falsifiable. Worse, the product might be an economical failure if this linkage is done based on wrong assumptions of the customers’ desires. On the other hand, a number of products not selling well were later considered to be way before their time. This shows that the product developers failed to properly interpret the voice of the customers, rather than that the technical specifications were incorrect. In order to avoid such failures, many companies seek for more reliable methods to grasp and translate the customers’ affective understanding into concrete product solutions. The Kansei Engineering methodology is, in particular, specialized in the translation of affective values into concrete product design parameters. To achieve this, Kansei Engineering uses Semantic Differential Scales (SD-scales) as a central pillar. This special type of scale was developed by Charles E. Osgood in the 1940/50s and measures the way a person feels about a certain object on a 5-point scale (Osgood, Suci and Tannenbaum, 1957). This data can then be treated using, for example, factor analysis, and underlying patterns of how the object is perceived can be extracted (Osgood and Suci, 1969). This information is then used for improving the perception of the object. Although Osgood used the method originally for political research, the same technique can be used, slightly modified, for consumer products. In the case of consumer products, how the customer interacts, under what circumstances, and for what amount of time are essentials for the depth of the Kansei built (Eklund and Kiviloog, 2003).

4. THE KANSEI ENGINEERING APPROACH Measuring the Kansei is not easy and will always build on a subjective basis, since the measurement methods are dependant on the reactions of the humans. However, Japanese researchers, with Mitsuo Nagamachi as a forerunner, invented a method in the 1970s which was able to grasp the Kansei and translate it into concrete product design solutions (Mitsuo Nagamachi, 1997). In the beginning it was called Emotional Engineering, as one of the first companies, Mazda Motor Corporation, applied the new methodology in their product development in the early 1980s. In a speech delivered in 1986 by Mazda’s chairman in Detroit, he referred to the methodology as ‘Kansei Engineering’ (Nagasawa, 2002) and coined the new methodology. In a Kansei Engineering context, a static view on the degree of importance of sensory systems is not beneficial. In fact, the significance of the sensory systems varies depending on

480

Product Experience

the products and concepts in question and individual preferences. In most studies using affective engineering methods in product development, all senses are needed (Nagamachi, 1989). For example, when determining the quality of a cup of coffee the visual appearance is less important than taste and smell. Also in a car compartment, the haptics of a textile surface is best evaluated by the sense of touch and vision working together. In the case of evaluating the sound of a car engine, hearing is the only relevant sensory system. Vision and smell can in fact distract the impression. The Kansei as an internal sensation is closely related to external senses (Nagasawa, 2002). The external senses deliver the input that is needed to build up a Kansei and react in an appropriate way. This principle is utilized in Kansei Engineering, where the affective meaning on external stimuli is measured (Lee et al., 2002). In this paper they show that the Kansei can never be quantified or described fully. In fact, methodologies such as Kansei Engineering use physical and external signs, such as people’s behavior, actions, spoken words, facial and bodily expressions, or physiological responses such as heart rate, EMG, EEG, etc. (Nagamachi, 1989). Kansei Engineering started with humble steps, but today at least six different types exist. Nagamachi (1997) collected all these applications on Kansei Engineering and grouped them according to the tools included and task areas. From these groups he identified so-called types of Kansei Engineering: • Kansei Engineering Type I: Category Classification. In Kansei Engineering Type I, a product strategy and a market segment is identified and developed into a tree structure identifying the customer’s affective needs. These affective needs or Kanseis are then connected manually to product properties. • Kansei Engineering Type II: Kansei Engineering System. Kansei Engineering Type II is often a computer aided system using interference engines and Kansei databases. The connections between Kansei and product properties are made using mathematical statistical tools. • Kansei Engineering Type III: Hybrid Kansei Engineering System. Kansei Engineering Type III is also a computer database system similar to the second type. However, it can not only suggest suitable product properties from an intended Kansei, but also predict the Kansei that product properties elicit, e.g. by using a prototype or mock-up. • Kansei Engineering Type IV: Kansei Engineering Modeling. The fourth type of Kansei Engineering focuses on building mathematical prediction models. These models are more strongly validated than the ones in the Types II and III. • Kansei Engineering Type V: Virtual Kansei Engineering. Kansei Engineering Type V integrates Virtual Reality (VR) techniques with standard data collection systems. This type replaces the presentation of real products with VR representations. • Kansei Engineering Type VI: Collaborative Kansei Engineering Designing. In Kansei Engineering Type VI, the Kansei database is accessible via Internet. Such design supports group work and concurrent engineering. More complete descriptions of these different types can be found in Nagamachi (1997).

5. THE KANSEI IS CHANGING The Kansei is not only difficult to quantify due to its individual character, it is also changeable. This means that Kansei data is valid only in certain limited contexts and during a limited time period. Four important reasons can be identified as influencing this change

481

Affective meaning: The Kansei Engineering approach

(compare also Schütte, 2005). People’s subjective perception of products usually depend on factors such as: • • • •

Personal interest and competence. Experience of interaction. Fashion and trends. Time dependency.

These aspects have crucial consequences for Kansei Engineering studies, but in spite of that are seldom addressed in international Kansei Engineering literature. If one of the factors mentioned above changes, the general Kansei of a customer group also changes. The significance of the change depends on other factors, such as the product, the customer group, etc. Kansei data must therefore be continuously updated in order not to be misleading. This means that the experimental design must be chosen carefully since it restrains the affective information. Schütte (2005) suggested a model explaining the way the affective data is treated (and limited) before they are perceived by the human senses. The model is based on Eklund and Kiviloog’s (2003) research and on a model used in artificial intelligence described by Picard (1997). Picard (1997) calls this the ‘affective channel width’. She concludes that computers affectively interacting with human beings have to provide certain signals, which can be sensed by the humans. In general it can be said that the more affective signals are given (the wider the affective channel), the better the mental picture (more intensive Kansei). Figure 20.2 illustrates the way the information travels when building a Kansei. This may be referred to as the affective flow. A certain product property possesses attributes that are transferred by different physical signals in the affective channel. This information is then received by the users’ senses and transformed into a Kansei. However, this is an ideal view. In praxis there are obstacles on the way, limiting or even cutting off the semantic flow. In Figure 20.2, these obstacles are referred to as Proximity of Presentation and Proximity of Interaction, acting as ‘affective windows’ that limit the affective flow. In Kansei Engineering, the ‘affective windows’ must be set in a way that information necessary for building up a sufficiently complete Kansei passes, but unnecessary information is blocked.

5.1. Proximity of presentation Product properties need certain senses to be transferred into a personal Kansei. In order to sense the Kansei fully, certain affective channels (Picard, 1997) must be used as shown in Figure 20.2. This means that the way a product is presented plays an important role. Product Product property

Affective channel

User sense

appearance

visual

sight

sound

acoustical

hearing

haptics

tactual

touch

odor

olfactorial

smell

tastants

gustatorial

taste

Kansei

FIGURE 20.2 The affective flow developed by Schütte, inspired by Eklund and Kiviloog (2003), Picard (1997) and Nagamachi (1997).

482

Product Experience

For example, the Kansei of a piece of chocolate is not satisfactorily transferred by a picture, since the smell and taste are important stimuli which are suspended in this case and the user can not interact with the product in the way she/he normally would. Hence, the ‘affective window’ must allow olfactory, visual, and gustatorial information to pass. A product may be represented as a picture, a 3D computer based model, a mock-up or as a prototype, all of which limit the opportunity of the user to build up the Kansei that the real product would have done. The ‘Proximity of Presentation’ consequently means the extent to which a user is able to perceive a product with all necessary senses without restrictions from the way it is presented. This is of importance in all user evaluations including Kansei Engineering studies. In general it can be said that the following two points have to be considered for each Kansei Engineering study. • Definition of channels needed in order to give a full Kansei. • Definition of necessary degree of interaction. These definitions automatically devise the ‘(minimal) Proximity of Presentation’.

5.2. Proximity of interaction When performing a Kansei Engineering study, the goal must be that all participants experience all aspects of the Kansei of the product in question. Otherwise, the impact of the affective values cannot be measured correctly. The ‘affective window’ must be set in a way that the information sent by the product is really sensed by the user. In certain situations, users can not interact fully with a product, e.g. if it does not exist or if it would be too costly or unethical to allow full user interaction. The ‘Proximity of Interaction’ consequently means the extent to which a user is able to interact fully with a product without restrictions of the way it may be used. This influences all kinds of user evaluations. Three points can be identified which have major influence on how well the Kansei is transferred. 1. Prior experience of the products. 2. Interest in the products. 3. Degree, time, and context of interaction. A study carried out on office chairs came to similar results. In this paper the expression ‘Proximity of Interaction’ was suggested for describing this phenomenon (Eklund and Kiviloog, 2003). Experience shows that groups with a high Proximity of Interaction, i.e. participants with good prior experience, high interest, and a high degree of interaction, usually deliver more relevant results than other groups.

6. KANSEI ENGINEERING PROCEDURE There are at least six different types of Kansei Engineering in use. However, the Kansei Engineering procedure is rarely described in literature available in English. Based on the work done in different areas of Kansei Engineering, Schütte et al. (2004) suggested a general model on how Kansei Engineering methodology works. Based on a literature review, the authors proposed a framework on Kansei Engineering methodology. Figure 20.3 presents this framework. Based on an earlier chosen domain the idea behind the product can be described from two different perspectives: The semantic description and the description of product properties. These two descriptions each span a kind of vector space. Subsequently, these spaces

483

Affective meaning: The Kansei Engineering approach

Choice of domain

Span the semantic space

Update

Span the space of properties

Synthesis

Update

Test of validity

Model building

FIGURE 20.3 A proposed model of Kansei Engineering (Schütte, 2005).

are analyzed in relation to each other in the synthesis phase, indicating which of the product properties evokes which semantic impact. After these steps have been carried out, it is possible to conduct a validity test, including several types of post-hoc analyses. As a result of this step, the two vector spaces are updated and the synthesis step is run again. When the results from this iteration process appear satisfactory, a model can be built describing how the Semantic Space and the Space of Properties are associated.

6.1. Choosing the domain Choosing the domain includes the selection of a target group, market-niche, and specification of the new product. Based on this information, product samples are collected, representing the domain. The Kansei Domain can be understood as the ideal concept behind a certain product. Despite the fact that a circle can never be drawn perfectly round, everybody knows what the perfect idea of a circle is. The Kansei domain is dealt with in the same way. It is an abstract super-ordinate mind structure, while the representative products are tangible, intangible or combined samples from this domain. As a result, a domain includes existing products, concepts, and even still unknown design solutions. The task in this first step is to define the domain and find representatives (products, drawings, samples, etc.) covering as big as possible a part of the domain.

6.2. Spanning the Semantic Space The Kansei is hierarchic. This means that one higher-level Kansei joins together several lower-level Kanseis and facilitates in this way the representation of the customers’ affective values. For example the low-level Kanseis of ‘slow’, ‘fast’, ‘indolent’, ‘agile’, ‘quick’, and ‘speedy’ can be summed up to a single higher-level Kansei ‘kinetic’. In Kansei Engineering only higher level Kanseis are connected to product properties in the synthesis phase in order to achieve a better generalization of the results. Spanning the Semantic Space identifies these higher level Kanseis from a great number of semantic expressions. Although the expression ‘Semantic Space’ originates from Osgood et al. (1957), more methods than his Semantic Differential Scales are available nowadays.

484

Product Experience

Collection of lowlevel Kansei words

Incoming data from choice of domain

Kansei structure identification

Compiling data

Outgoing data to syntheses

FIGURE 20.4 Procedure of spanning the Semantic (Schutte, 2005).

For practical reasons, spanning the Semantic Space has been subdivided into three steps as presented in Figure 20.4. Using the desired domain as a starting point, low level Kanseis, typically adjectives, are collected, describing the considered product semantically. From this set of semantic descriptions, higher-level Kanseis are identified in ‘Kansei Structure Identification’. In Kansei Engineering literature these higher-level Kanseis sometimes are also referred to as ‘Kansei Words’ or ‘Kansei Engineering Words’. Finally, the data is compiled in a standardized way in order to facilitate the following synthesis phase. If important Kansei Words are missed in this step, the result may have significantly limited validity. Hence, it is better to select a few more words than necessary. Collection of Kansei Words A Kansei Word is a word describing the product domain. Often these words are adjectives, but other grammatical forms are possible. For example when describing the domain ‘fork lift truck’, adjectives like effective, robust, quick, etc., but also verbs and nouns such as ‘accelerate’/‘acceleration’ can occur. In order to get a complete selection of words all available sources have to be used, even if the words emerging seem to be similar or the same. Suitable sources can be magazines, pertinent literature, manuals, experts, experienced users, related Kansei studies, etc. An important point is to translate ideas and visions into Kansei Words because non-existing solutions should also be considered. In this way Kansei Engineering can be used as a creative product development tool, which generates innovative solutions. The task is to describe the domain, not existing products. Depending on the domain considered, the number of existing Kansei Words generally varies between 50 and 600 words (Nagamachi, 1997). Since it is of great importance to cover the whole Kansei, the word collection is continued until no new words occur. The data gathered will critically influence the validity of the results if important words are missing. Tools for semantic structure identification For the identification of the Kansei structure, different methods are developed, tested, and made available for use. Generally, two different types of methods can be found. Manual methods are mainly used by experts and experienced users of Kansei Engineering. The Kanseis are grouped and summarized according to the participant group’s preferences and needs. Established methods are: • Affinity Diagram (Bergman and Klefsjö, 1994). • Designer’s choice. • Interview technique.

Affective meaning: The Kansei Engineering approach

485

The major disadvantage of relying on experts is that customers can have different opinions. An alternative or complement is to base decisions on user data. Typically, this is done by questionnaires or interviews. This requires other evaluation tools, such as statistics. The following statistical methods are available for use in Kansei Engineering today: • • • • • •

Principal Component Analysis (Osgood and Suci, 1969). Factor Analysis (Osgood and Suci, 1969). Cluster Analysis (Hair et al., 1995). Quantification Theory Types II, III, IV (Tsuchiya, 2004). Neural Networks (Ishihara, Ishihara and Nagamachi, 1998). Rough Set Analysis (Tsuchiya, 2004).

6.3. The Space of Properties as a counterpart of the Semantic Space As shown in Figure 20.3 the product domain is described both from a semantic perspective and a physical perspective. Both perspectives are presented in the form of vector spaces. However there are significant differences in the theoretical background of the two spaces. Whereas the semantic descriptions possess a well researched theoretical background based on, for example, Osgood’s Semantic Differential Technique (Osgood and Suci, 1969), there is no similar theory for the Space of Product Properties. Hence, there is no consistent way of developing the Space of Properties. At the same time, the selection of properties is essential (Nagasawa, 2002). However, few studies really evaluate the affective impact and the importance of the product properties on the user. Often, they are assumed to be relevant, given by the client company, or even chosen randomly. In the majority of cases however, the product properties are chosen on the basis of the feasibility of producing product examples to present in the study (Kanda, 2002). How can it be ensured that the properties chosen are really relevant to the user/ customer in the examined context? What happens if a feature chosen for selection is not important to the user? To illustrate, in a study the participants are asked to make a statement on the quality impression of a postal service. The samples differ in many of their properties and especially the delivery time and ability to track the batch may be of importance. If these properties are not chosen for evaluation the final result will not give a correct answer. Worse, it will not be possible to determine that there is a property missing. Consequently, it is necessary to rate the importance of the different product properties and make this a criterion for the selection. There are certainly methods capable of making an adequate selection of product properties for Kansei Engineering, but the problem is that they are not structured, nor generally tested for this task. This section will make an attempt to assemble methods for different studies, but also use methods from other areas used for similar purposes. Probably the most important demand is to provide a structured approach for constructing the Space of Properties. Spanning the Space of Properties The systematic collection of Kansei Engineering Properties, i.e. properties usable for a Kansei Engineering study, follows the model of the collection and selection of Kansei Words. It roughly can be subdivided into three steps as shown in Figure 20.5. In the collection step, inspirational material regarding a product domain is collected from a variety of sources and potential properties are identified. In a second step they are sorted according to certain rules. The number of properties is narrowed by prioritization. Only properties with high affective impact are chosen for further evaluation.

486

Compilation

Selection

Collection

Product Experience

Collection of product samples of existing products

Creation of new concepts

Definition of company image

identification of product properties

Identification of new properties

Select image properties

Determination of importance by customers

Determination of importance by expert group

Determination of importance by Company

Selection of items for evaluation

Selection of representing product

FIGURE 20.5 Spanning the Space of Properties (Schutte, 2005).

Finally, example products are found possessing those properties chosen, representing in this way the Space of Properties. Depending on the method used for relationship identification, the assembly of products can vary. In conformity with the building of the Semantic Space, the raw data is collected from different sources. Typically, existing products provide a wide variety of potential properties, which can be integrated into new products. Getting inspiration from already existing products is one of the most common ways of identifying relevant properties. Sources for collection of properties are usually found in literature, technical datasheets, magazines, etc. For identification, often an assembly list of properties is sufficient. The determination of importance and selection of properties with the highest importance and affective value is preferably done by customer representatives. To facilitate the work of obtaining raw data, tools such as focus groups or one-to-one interviews can be used. For determining the importance, e.g. Pareto diagrams (Bergman and Klefsjö, 1994) can be useful. In almost all Kansei Engineering studies carried out within industrial product development projects, a central specification must also fit the company image. Companies therefore tend to integrate unique features in their products. The right column in Figure 20.5 identifies company-specific product properties which are characteristic for the company’s brand image. Together with company marketing experts, the relative importance of these properties is determined. Usually the number of image properties is so small that no special tool needs to be deployed. The central column in Figure 20.5, however, is the integration of new product concepts. Kansei Engineering has been criticized for not being innovative. This part displays how creative thinking and new ideas can be integrated into Kansei Engineering as a method. As a main source, designers’ minds are used. Designers can make mock-ups, sketches or prototypes of the whole products or parts of them. Thereby they create potentially new properties, which are appraised and selected by an evaluator group. However, Figure 20.5 also displays that these processes do not necessarily take place separately and in isolation. To the contrary, they influence each other as indicated by the

Affective meaning: The Kansei Engineering approach

487

arrows. The designer might get inspiration from both existing products and the company’s image, which is then developed into a new solution. This new concept in turn might influence the company’s decision-making about which product properties to select as a relevant image. Also, new trends identified by the designers may influence the choice of product properties from existing products. Finally, all selected properties are brought together to one set of product properties from which representative products are determined or mocked up to be used in the following synthesis step.

6.4. Synthesis In the synthesis step the Semantic Space and the Space of Properties are linked together as displayed in Figure 20.6. For every Kansei Word a number of product properties are found affecting the Kansei. Also the size of the affective impact of the product properties on each Kansei is quantified. This synthesis is the core of Kansei Engineering technology. Whereas the identification of the semantic and physical structure is also carried out in different forms in other contexts such as Semantic description of environments (SMB) (Küller, 1975) or Semantic Differential Method (Osgood et al., 1957), the translation of the determined Kansei is exclusively performed in Kansei Engineering. Due to this fact this part of Kansei Engineering has been in focus for research since the beginning of its existence. A number of tools have been developed and are used for this part. Even here the same categorization can be made into the three different areas: Manual methods; statistical methods; and other methods. Manual methods for connecting the Kansei and the different product properties are easy to perform and require comparatively small resources. These are the oldest tools and are preferred by practitioners. One tool is: • Category identification (Nagamachi, 1997). As in semantic structure identification, statistical methods are used for treating great amounts of data from questionnaires. The tools used here have to be modified in order to fit the requirements of Kansei Engineering. Some possible tools for statistical treatment are: • Regression Analysis (Schütte, 2005). • General Linear Model (Arnold, 2002). • Quantification Theory Type I (Komazawa and Hayashi, 1976). Other tools use ranking and rating methods. These methods are mainly based on intelligent computer systems, and are able to sort and find similarities in the data. Those methods are: • Genetic Algorithm (Nishino et al., 1999). • Fuzzy Set Theory (Shimizu and Jindo, 1995). • Rough Set Theory (Mori, 2002; Nishino, Nagamachi and Ishihara, 2001).

FIGURE 20.6 Synthesis phase.

488

Product Experience

6.5. Model building and test of validity Finally, a mathematical or non-mathematical model is built depending on the synthesis method chosen. However, before using the model as a prediction model for future products, it has to be validated. At present, not only are validation methods for the Semantic Differentials available, but there is a need for a more integrative validation concept.

7. APPLICATIONS OF KANSEI ENGINEERING IN INDUSTRY Kansei Engineering can be applied in a number of different areas and for various products. The remainder of this section illustrates that with some brief examples. There are many more products, not only tangible but also intangible on the market, developed by using Kansei Engineering methodology.

7.1. The Mazda Miyata case One of the most famous Kansei Engineering cases is probably Mazda’s MX5, which became the world’s most sold sports coupé through consequent application of Kansei Engineering in many details. During its design process, Kansei Engineering was constantly deployed. The aim of the project was to build a sporty low-price car for younger male drivers. Also the new car should fit the needs of not only Japanese, but also European and American drivers. Mazda’s CEO, Mr Yamamoto, asked Nagamachi to support the technical design of a new sports coupé aiming at the above described target group. A special team was assembled dealing with the affective aspects of the new car. This team firstly examined the behavior of young male drivers. In practice they observed drivers’ manoeuvring in real traffic situations and documented it on film. The outcome from this analysis was further analyzed, and the start at intersections was identified as one of the most crucial situations. The material gathered was then presented to a board. In several brainstorming sessions, the team members wrote down keywords on cards relating to the displayed situations.

FIGURE 20.7

Mazda Miyata (Source: Mazda Motor Corp.).

489

Affective meaning: The Kansei Engineering approach

Kansei Zero

1st Tight feeling Direct feeling

HMU Speedy feeling Communicative

2nd.........n

Sensation Characteristics Physical traits

Body size Width Vision Engine Height Hearing Chassis Seat Smell Tactile Steering yaw Steering Noise design Organic Vibration Frequency

FIGURE 20.8 Kansei Engineering Type I as used in the Mazda case (Nagamachi, 1997).

Around 600 keywords were collected and arranged in the tree structure shown in Figure 20.8. Cards with similar meaning were grouped, these were arranged in a tree and main headings were found (affinity diagram, compare Bergman and Klefsjö, 1994). In principle, this method is used in order to create the higher levels from which the results are translated into concrete design parameters. The top card was named ‘Human–Machine Unity’ (HMU), which means that the driver imagines the vehicle as a natural extension of the body. This means that the car reacts in the way the driver expects. The HMU concept at the first level was represented by a combination of tight feeling, direct feeling, speedy feeling, and communication between the vehicle and the driver. In the case of ‘tight feeling’, the tree had to be developed into two levels. From this point the team carried out a small Kansei Engineering experiment where most of the participants voted for a body length around 4 meters and only two seats. In fact the Miyata has a body length of 3.98 meters and 2 full-size seats. In the same way, the sub-concept ‘direct feeling’ was evaluated. Here the gear shift lever was identified as essential for that part of driving impression. Especially two aspects, the length and the weight, were examined. For this part, Kansei Engineering Type II using statistical data in combination with experiments was applied. A variety of levers – different in length and weight – were presented to selected users. They were asked to rate the different concepts on semantic differential scales. These scales evaluated Kansei words correlating to ‘direct feeling’ and ‘self-controlling lever’. As a result of this experiment, the gear shift lever of 9 cm in length was selected as the best fit of the intended Kansei. In the case of ‘speedy feeling’ the engineers faced a problem at first. In general, there is time lag between pressing the accelerator and feeling the car accelerating. Such behavior is considered to be not sporty. Hence, the power train team worked to shorten this time gap by modifying the transaxle and engine control system. Also the speedometer was designed to give a very early response. In the end, the time lag was not detectable by subjective evaluation. Other parts evaluated were the design of the engine, exterior and interior details, and the exhaust pipe (Nagamachi, 1997).

7.2. Developing a new hair treatment In 1999 Nagamachi and Ishihara were asked by Milbon Inc. to help developing a new hair treatment product. Nagamachi participated in the work of improving composition of chemical contents according to the users affective needs (Nagamachi, 2002), whereas Ishihara conducted research on containers (Ishihara, Ishihara and Nagamachi, 2000). Nagamachi proposed to perform a Kansei survey about ladies’ hair problems. The project team, mainly consisting out of Milbon’s staff, visited a number of hair-salons interviewing female users concerning women’s hair issues. A structured survey and Kansei questionnaire was used. The survey was conducted with 300 women in the age range between 20 and 50. From this pre-study, typical hair problems, hair styles, and individual

490

Product Experience

demands on cosmetic products were found and documented. The zero-level Kansei concept, in this study was then derived from the data. It was called ‘Soft and Breezy or Rustling Hair’. The zero-level concept was broken down to the sub-concepts in the nth level in the tree structure. Figure 20.9 illustrates the breakdown process related to the chemical bulk development. Several sub-concepts in the nth level were selected with concrete definitions and transferred to the research institute. It constructed about 600 different chemical compositions and examined and tested the material on mannequin hair. In the next stage, selected chemicals were tested in a field study on voluntary female participants. The participants in this part of the study filled out a Kansei questionnaire. The members of the development group evaluated each test and compared the result to the intended Kansei. Finally, only two kinds of chemical compositions fulfilled the affective demands of the users. The final choice was then made by the team based on previous experience. In parallel, an evaluation of containers was carried out by Ishihara. In his experiment, Ishihara asked participants to evaluate hair treatment samples on a 5-point SD scale. In total 43 samples were collected and used for evaluation on standard Kansei questionnaires. Fourteen female college students aged 19 or 20 participated. Thirty-nine Kansei Words were used for evaluation on the SD scale. The evaluation took about 3 hours in total. Analysis was done with arboART, a self-organizing neural network based on hierarchical clustering (Ishihara et al., 1998). It performs hierarchical clustering with smaller errors than traditional computation methods. The input vector for arboART was the averaged evaluation values between subjects. All 43 different hair-treatment and shampoo containers were used as evaluation samples, and evaluated on Thirty-nine Kansei Word pairs in the questionnaire. By clustering with arboART, the samples were hierarchically grouped and formed a pattern corresponding to the subjects’ affective response to products. The outcome from the synthesis of the semantic space and the product properties (in arboART called classification) identified three major clusters, two small clusters, and eight clusters having a single sample.

Hair care survey --QTIII

Container development 1st level 2nd level

Selection of container

Kansei words

Kansei experiments

Analysis

Presentation to designer

Decision

3rd level

Zero-level concept

Selected Chemical Kansei definition

Soft and breezy hair

Salon research Mr. Saito

Chemical combination and test products

Test examination

AD

A"

A"

BE

B"

B"

CF

C"

C"

Monitor evaluation

Monitor evaluation

Final evaluation

Salon research

FIGURE 20.9 Development flow chart for Milbon’s Deesse’s.

Final products

Shampoo and Treatment

Affective meaning: The Kansei Engineering approach

491

FIGURE 20.10 The newly developed containers; shampoo (left) and treatment (right).

The first major cluster corresponded to the Kanseis of adult, feminine, and polite. Sample products belonging to this cluster had the colors light pink, light blue or white. The shapes of the containers did not seem to affect the final result. The second major cluster corresponded to the Kanseis of calm, adult, premium, and individual. Containers belonging to this cluster were yellow-brown, deep blue, and deep green. One was white, but its cross-section was almost octagonal which affected the rating. The main feature for the third major cluster was a bright blue to green pastel tone in color evoking the Kanseis of bright, healthy, and light. The members of the first small cluster were similar in shape, neutral in color and product name written horizontally in small font. They were rated calm, adult, and simple. The second small cluster had a horizontally written name in small font in common. This cluster corresponded to high-grade, simple, and beautiful. Milbon’s aim was to make a container which corresponded to premium and high-grade Kansei. At that time, Milbon’s products were colored yellow-brown and were regarded as adult and high-grade. A deep green or blue container was suggested as an alternative choice from the results. In addition an angulated design derived from an octagonal bottle and combined with a one-line small font logo reinforced the intended Kansei. Milbon’s designer made concrete designs based on the studies’ outcome. The new color chosen was deep blue-green, the shoulder was made longer than the bottom, and the product logo was written as one line with a smaller font (Figure 20.10). The newly designed hair-treatment products were named ‘Deesse’s’. These products were a huge success, and sales of Deesse’s reached 75 million USD in 5 years.

7.3. An example of engineering design for driving feeling One of the world’s leading forklift truck manufacturers, BT Industries, now owned by Toyota, investigated ergonomics aspects of low lifting forklift trucks. It was found that the old design of the driver platform contributed to several problems. Taller drivers perceived their work posture as uncomfortable when standing in the middle of the platform, so they preferred to stand on the rear part with their heels outside the platform. Further, the platform was hinged without suspension in the front, and consequently did not damp vibrations or jerks. Driving on uneven surfaces, e.g. when loading or unloading lorries,

492

Product Experience

transmitted vibrations and jerks to the spine. This gave rise to discomfort and eventually back pain for the drivers. This was identified as a second reason why the drivers preferred to stand on the rear part of the platform, since the suspension was softer there (Figure 20.11). One of the most common causes of injury when driving low lifting forklift trucks was heel injuries. Standing with the heels outside the platform substantially increased the risk of heel injury when driving too close to and hitting objects in the warehouse. A new design of a platform was proposed in order to eliminate the problems described above. By taking anthropometric considerations in the design and placement of the handle, the driver posture became more comfortable for tall drivers, and allowed the drivers to stand comfortably in the middle of the platform. In order to improve the suspension for a new generation forklift, the platform was hinged in the front part of the forklift. In this way, the whole platform could be suspended, not only the rear part (see Figure 20.12).

FIGURE 20.11

A typical platform for a low lifting forklift truck (Source BT-Industries).

FIGURE 20.12 The prototype platform suspension, showing the experimental conditions.

Affective meaning: The Kansei Engineering approach

493

However, in order to finalize the engineering design, the suspension and damping needed to be determined in engineering terms in order to provide driver comfort, safety and satisfaction. Not just the stiffness of the spring and the damping properties, but also the combination of these needed to be decided upon. A study was initiated to give the necessary input to the engineering design. Kansei Words were identified through interviews with 25 forklift truck drivers, from journals and information material. In total, 70 Kansei Words were identified that described desired properties of the platform suspension. These were reduced by manual clustering to six words, namely damping, calm, robust, jerky, rigid, and hard. A forklift prototype was built for three springs with different stiffness, which could be combined with three damper levels. In total, the prototype generated nine experimental conditions. In addition, a tenth condition was included, namely the original forklift platform. An experimental drive cycle was designed to simulate usual tasks for this type of forklift truck. In total, 17 forklift drivers participated in the experiment, in which they rated their driving feeling for each of the 10 experimental conditions and for the six Kansei Words, using semantic differential scales. The analysis was performed with Quantification Theory Type I (QTT) in order to identify to what extent the design elements (experimental conditions) contribute to the driver ratings. The results showed that damping was perceived as a favorable condition which mainly was associated with the low damper and to some extent with a stiff spring. Calm and robust were somewhat associated with a stiff spring. Jerky was closely associated with the damper, and perceived positively (less jerky) for the low damper. Rigid was associated to both the damper and the spring, and perceived positively for the low damper and the stiff spring. Finally, hard was associated to the damper and perceived more positively for the low damper. The combination of a stiff spring and a low damper was overall perceived as the most favorable and the experimental platform was perceived clearly more favorably than the original platform. The combination conveyed feelings of a damped, calm and robust platform, which was not jerky, rigid and hard. The findings from this study have resulted in the market introduction of a new low lifting forklift, which included a new platform designed according the principles described above. Further information of this study can be found in Axelsson, Eklund and Nagamachi (2001).

8. REFLECTIONS ON KANSEI ENGINEERING METHODOLOGY 8.1. Is Kansei Engineering innovative? Critics of the methodology have claimed that Kansei Engineering has a lack of innovativeness. However, Kansei Engineering has been involved in the development of many successful products, such those presented above, but the argument is that Kansei Engineering itself cannot contribute to the new features. Kansei Engineering methodology is often used in order to evaluate already existing products. It can determine which product features are important for a certain impression or Kansei and give recommendation about how to combine them with each other in order to achieve the intended Kansei or even combinations of different Kanseis. The reason for this is that the customers/users of a product must have sufficient experience in order to determine the Kansei. This is not the case if the product type is completely new. It is difficult for people to have opinions of products they have not interacted with. Hence, Kansei Engineering is more useful for evaluating products and prototypes after long-term usage. People get thorough experience of the products and obtain clear opinions about them.

494

Product Experience

They can also evaluate new features in old products. Even if Kansei Engineering cannot directly take part in the generation of new ideas and concepts, it presents suggestions, delivers directions and creates a ground of discussion which triggers creative processes in product development staff. This is the way it has been used in the cases of BT, Mazda and Milbon. In most cases product development is not about revolutionizing the product type. It is more about attractive quality creation (Kano et al., 1984), i.e. a gradual improvement of the products’ affective values by adding a few attractive qualities and at the same time ensuring one-dimensional and must-be-quality. Kansei Engineering is designed to accomplish this. However, there are development opportunities in Kansei Engineering which also have been addressed previously. One is the integration of creativity tools. These are techniques collecting and ‘harvesting’ new ideas. Some of them have already been suggested for usage in Kansei Engineering, such as brainstorming, Pareto-diagrams, Card-systems and Affinity diagrams, and more methods can be included.

8.2. Reinforcing exterior design using interior qualities Sometimes Kansei Engineering is used in order to develop or confirm the exterior design of products in order to create attractive products. However, in limiting applications to exterior design, Kansei Engineering can probably not show its full potential. In fact, other methods might be more powerful in this respect. Attractive design does not only deal with good exterior appearance, but also with a good balance with interior qualities (Garvin, 1988; Kano et al., 1984). As the examples above show, the strength of Kansei Engineering is to optimize properties which are not directly detectable or visible as e.g. the atmosphere of a concert hall, the concepts of good driver feeling or quality feeling by modifying the (engineering) properties of the products. Together with the exterior design the desired affective impression can be reinforced using Kansei Engineering. On the other hand, if there is a discrepancy between e.g. exterior design and engineering properties, this could be perceived as dishonest design. In conclusion, the application of affective engineering on the engineering design of the products and the technology used in them is a key factor in order to obtain feelings of e.g. usability, quality, durability, safety, comfort, engineering perfection, and driver feeling.

8.3. Reductionism versus holism It is a characteristic for reductionistic approaches that the reality is reduced to a (mathematical) model. In most natural sciences (including engineering) this is the preferred approach to explain phenomena isolated from their context in order to simplify and purify. Kansei Engineering in its original meaning uses a reductionistic approach. Both the semantic descriptions and physical descriptions are broken down into their essential parts and prioritized. Thereafter, a model is constructed which is only based on the most important factors of the products. This procedure does not consider the importance of other contexts, and ignores factors with minor impact on the total Kansei. This is probably due to the prevailing paradigm in the application areas of Kansei Engineering, i.e. product development departments. One reason why Kansei Engineering sometimes appears attractive to engineers might be the fact that it uses a similar approach in problem solving as most staff in product development use. Advantages of a reductionistic approach are that Kansei Engineering can be spread more easily to practitioners in product development departments. Reducing the

Affective meaning: The Kansei Engineering approach

495

number of influencing factors to a minimum, i.e. finding the most essential semantic and physical descriptions of the product domain, facilitates the understanding, enhances the knowledge of the product and makes the affective needs of the customers clearer to the designers. On the other hand, certain details of the Kansei cannot be caught using Kansei Engineering because the small factors are purposely omitted. This is a direct consequence of its reductionstic view. In humanistic science there might be holistic approaches, which can deal with the missing parts of the Kansei and their interactions. In order to develop Kansei Engineering into a bridge between technical and psychological science, more holistic approaches should replace or complement the traditional techniques. One way to go might be to develop approaches for qualitative studies in order to shift from strictly quantitative data to a combination of quantitative and qualitative data.

8.4. Moving from quantitative to qualitative approaches Kansei Engineering studies have in most cases been performed with quantitative data and statistical tools. The basis of the method is the identification of the relationship between the affective impression of the users and the product design parameters. There is no problem in using qualitative data instead of quantitative data within the framework of Kansei Engineering. The use of Rough Set Analysis is a tool supporting such a study. The use of qualitative data enables a deeper understanding of complex relationships that form the basis of the reactions of the humans. The qualitative methods and the quantitative methods should be seen primarily as complements to one another. Many authors point to the advantages of combining a qualitative and a quantitative approach (see e.g. Ernzer and Wimmer, 2002). In that way, better understanding of complex phenomena can be obtained.

REFERENCES Arnold, K. (2002). Towards increased customer satisfaction. Unpublished Master Thesis, Linköping University, Linköping. Axelsson, J. R. C., Eklund, J., Nagamachi, M., Isihara, S., Rydman, K. and Sandin, J. (2001). Suspension and damping of a lowlifter platform. Applications of Kansei Engineering. In: M. Smith and G. Salvendy (Eds.) Systems, Social and Internationalisation Design Aspects of Human–Computer Interaction, Vol. 2, pp. 333–337. New Jersey: Lawrence Erlbaum Associates Publishers. Bergman, B. and Klefsjö, B. (1994). Quality: From customer needs to customer satisfaction. Lund: Studentlitteratur. Childs, T. H. C. (2004). Personal communication. Presentation at LiTH. In: K. E. G. a. LiTH (Ed.) Linköping. Eklund, J. and Kiviloog, L. (2003, August 24–29). Kansei ratings and time dependencies. Paper presented at the Proceedings of the XVth Triennial Congress of the International Ergonomics Association, Seoul, Korea. Ernzer, M. and Wimmer, W. (2002). From environmental assessment results to Design for Environment product changes: an evaluation of quantitative and qualitative methods. Journal of Engineering Design, 13, 233–242. Garvin, D. A. (1988). Managing quality. New York: The Free Press. Hair, J. F., Anderson, R. E., Tatham, R. L. and Black, W. C. (1995). Multivariate data analysis with readings. London: Prentice-Hall. Helander, M. G. (2003). Hedonomics – affective human factors design. Ergonomics,46(13/14), 1269–1272. Ishihara, S., Ishihara, K. and Nagamachi, M. (1998). Hierarchical Kansei analysis of beer can using neural network. Paper presented at the Human Factors in Organizational Design and Management – VI, Amsterdam. Ishihara, S., Ishihara, K. and Nagamachi, M. (2000). Kansei analysis and product development of hair treatment. Paper presented at the Ergon-Axia 2000, Warsaw, Poland. Jordan, P. W. (2002). Designing pleasurable products: An introduction to the new human factors. London: Taylor and Francis.

496

Product Experience

Kanda, T. (2002). Classification of menus on home dining tables based upon human meal kansei. Kansei Engineering International – International Journal of Kansei Engineering, 3(4), 9–14. Kano, N., Seraku, N. and Takahashi, F. (1984). Attractive quality and must be quality. Quality, 14(2), 39–44. Komazawa, T. and Hayashi, C. (1976). A statistical method for quantification of categorical data and its applications to medical science. In: F. T. de Dombal and F. Gremy (Eds.) Decision making and medical care. New York: North-Holland Publishing Company. Küller, R. (1975). Semantisk Miljö Beskrivning (SMB). Stockholm: Psykologiförlaget AB Liber Tryck Stockholm. Lee, S., Harada, A. and Stappers, P. J. (2002). Pleasure with products: Design based Kansei. In: W. Green and P. Jordan (Eds.) Pleasure with products: Beyond usability, pp. 219–229. London: Taylor and Francis. Mori, N. (2002). Rough set approach to product design solution for the purposed ‘Kansei’. The Science of Design Bulletin of the Japanese Society of Kansei Engineering, 48(9), 85–94. Nagamachi, M. (1989). Kansei Engineering. Tokyo: Kaibundo Publishing. Nagamachi, M. (1997). Kansei Engineering: The framework and methods. In: M. Nagamachi (Ed.) Kansei Engineering 1. Kure: Kaibundo Publishing. Nagamachi, M. (2002). Kansei Engineering in Consumer Product Design. Ergnomics in Design, 10(2), 5–10. Nagasawa, S. Y. (2002). Kansei and business. Kansei Engineering International – International Journal of Kansei Engineering, 3(3), 2–12. Nishino, T., Nagamachi, M., Ishihara, K., et al. (1999). Internet Kansei Engineering System with Basic Kansei Database and Genetic Algorithm. Paper presented at the international conference on TQM and Human Factors, Linköping, Sweden. Nishino, T., Nagamachi, M. and Ishihara, S. (2001, June, 27–29). Rough set analysis on Kansei evaluation of color. Paper presented at the International Conference on Affective Human Factors Design, Singapore. Osgood, C. E. and Suci, G. J. (1969). Factor analysis of meaning. In: C. E. Osgood and J. G. Snider (Eds.) Semantic differential technique – a source book, pp. 42–55. Chicago: Aldine Publishing. Osgood, C. E., Suci, G. J. and Tannenbaum, P. H. (1957). The measurement of meaning. Illinois: University of Illinois Press. Picard, R. (1997). Affective computing. Cambridge: The MIT Press. Schifferstein, H. N. J. (2006). The relative importance of sensory modalities in product usage, a study of self reports. Acta Psychologica, 121, 41–64. Schütte, S. (2005). Engineering emotional values in product design. Kansei Engineering in development. Linköping University, Linköping. Schütte, S., Eklund, J., Axelsson, J. R. C. and Nagamachi, M. (2004). Concepts, methods and tools in Kansei Engineering. Theoretical Issues in Ergonomics Science, 5, 214–232. Shimizu, Y. and Jindo, T. (1995). A fuzzy logic analysis method for evaluating human sensitivities. International Journal of Industrial Ergonomics, 15, 39–47. Tsuchiya, T. (2004). Guest researching period. Sweden: Linköping.

21

THE USEFUL INTERFACE EXPERIENCE: THE ROLE AND TRANSFORMATION OF USABILITY JOHN M. CARROLL The Pennsylvania State University, University Park, PA

HELENA M. MENTIS The Pennsylvania State University, University Park, PA

Many of the products we use every day are partly digital; microwave ovens, audio systems, televisions, personal devices like MP3 players and personal digital assistants, and, of course, computers manage our information and services. Their user interfaces are digital displays, and we interact with them through digitally interpreted command gestures of various sorts. These products support a huge variety of activities – leisure and recreation, home care and family, education and learning, and work. Interacting with these products entrains myriad personal experiences – efficiency and control, achievement and satisfaction, confusion and frustration, curiosity and wonder. Increasingly, they support and transform our social interactions and experiences – friendship, trust, admiration, and suspicion. As these technologies become evermore ubiquitous in human lives, it is important to reflect on these experiences and understand better what we need to design for. Competing values of both usability and rich experiences can cause tension in any good interface design. Usability was once considered an encompassing term for interactive systems that were easy to learn, easy to use, and effective with respect to work activity. In the late 1990s, perspectives began to expand as to what a quality user experience really meant. Over time ‘usability’ and ‘user experience’ became fractured by the multitude of definitions and concepts on what usability does entail and, more prevalently, what it is thought to overlook. With incongruent definitions and relentless criticism of usability, one might be forced to feel they must choose one over another. This, we think, is an unnecessary battle. Why do we study experience for ‘useful’ interfaces? Isn’t usability good enough? This is the predominant thought – that usability is the first level of any design goal. Common practice usually shows that usability is a good place to start and ‘experience’ is something extra if you have the time and money. Separating the two makes as much sense as studying cognition without affect. That divisive notion is being overcome in the cognitive sciences and so should we consider experience as an integral part of usability – you can not have one without the other. Product Experience Copyright © 2008 Elsevier Ltd.

499

500

Product Experience

This is not a popular viewpoint. More often experience is held up as the cure to usability-focused practitioners and that experience does not need to consider usability. Beginning with Norman in The Psychology of Everyday Things aesthetics is stated as the antithesis of usability. Preece, Rogers, and Sharp (2002) echo the same sentiment in Interaction Design by stating that experience resides in the user while ‘objective’ usability resides within the product. IBM’s conceptualization of user experience states that user experience design (UXD) ‘encompasses traditional [human–computer interaction – HCI] design and extends it’ (IBM, 2006). HCI addresses interaction while UXD addresses everything else. This view basically speculates that you can have one without the other. However, there have been some who have tried to bridge the gap. Monk made a case during his closing plenary at HCI 2002 to extend the definition of usability to include fun, communication, and dependability. His premise was that designing for the home has highlighted the narrow focus of the concept of usability – primarily because technology uptake in the house is more dependent on attitudes of the consumer as opposed to the required use in the workplace. He states that these three issues of experience that are not captured under the traditional rubric of usability are just as important in the workplace, and that they should be merged back into the notion of what constitutes usability. Blythe and Wright’s (2003) introduction to Funology states the need to ‘extend’ usability to ‘encompass’ enjoyment; not add it on as an afterthought or consider it in spite of usability. Even the traditional usability pillars pay homage to an undeveloped notion of the effects of experience. The ISO 9241–11 lists three aspects to usability: Effectiveness; efficiency; and satisfaction. Effectiveness refers to achievement of a goal – which does not have to be an explicitly stated work goal, efficiency is the rate with which such goals can be accomplished, and satisfaction is the degree to which a product meets a user’s needs – although more often this has been reduced to a low level of frustration. These pillars leave room for expansion of definition, as well as room for other aspects of what constitutes ‘usability’. In this chapter, we examine the experience of using a practical digital product. From the standpoint of human history, this is still a relatively novel category of experience, but one that is increasingly diverse and important in everyday life. Beginning with a historical perspective of the concept of usability and the rise of experience in human–computer interaction, we then review research on digital products that describe desirable, adverse, and unexpected experiences. This review of research will show how the design of useful interfaces affects the experience of a product – whether it is directly or indirectly, intentional or not – and that broadening the definition of usability is perhaps more useful than separating out experience.

1. USABILITY One of the most interesting threads of development in information science and technology through the past 25 years is the field’s evolving conception of ‘usability’. The phenomenon of usability, and the theoretical and methodological construction of that phenomenon in practical design concepts and methods, is most centrally what the interdisciplinary field of human–computer interaction (HCI) is all about. Yet the meaning of the term ‘usability’ has changed through the past two decades, and in all likelihood will continue to change. Initially, usability was taken to be synonymous with ‘easy’ or ‘simple’. The defining challenge for HCI design in the 1980s was to produce concepts and methods to help ensure that computer software and hardware would be easy to learn and easy to use. A concrete way to think about this period is to turn the clock back to 1980. An early empirical study of a leading-edge IBM minicomputer office system enumerated patterns

The useful interface experience: The role and transformation of usability

501

of usability problems for expert office workers learning to use the basic word processing functionality of the system (Mack, Lewis and Carroll, 1983): • People assume that many aspects of text editors will work just like typewriting. • People are naive about how computers work; they do not know what is relevant to understanding and solving problems. • People are not good at following directions, and this leads them to make many errors. • People try to construct interpretations for what they do or for what happens to them, but their interpretations often prevent them from seeing that they have a problem. • People are often confused by prerequisites and side effects of procedures they try to carry out; they are often confused by feedback messages and the outcome of procedures. • Errors often cascade and interact, snowballing into tangles of side effects, misconceptions, and further incorrect actions. • People do not always benefit from help systems because they do not know what to ask for; help information is often not focused on the person’s specific problem. • People experience frustration and blame themselves. From a contemporary perspective, these issues may seem too crude, too obvious. But this was the original usability challenge of introducing computing to office professionals. In the ensuing quarter century, user interface design has made huge advances in presenting information and functionality more transparently and more intuitively through better technology, better concepts, and better methods for measuring and assuring usability. However, a major factor in the story of usability is contextual. Today, technologically advanced means digital; people are immersed in digital information and digital devices. No one comes to a word processing application with the expectation that it will operate just like a typewriter. Indeed, many people today would have to understand a typewriter by thinking of it as a mechanical word processor. As understanding of people’s experiences with information technologies developed, and as the cultural baseline for these experiences became richer, the concept of usability was enriched with ideas from human development to include such notions as ‘cognitively stimulating’, ‘consistent with prior knowledge’, and ‘transparently useful in the work at hand’. During the 1990s, as collaboration became a major problem area for human– computer interaction, and as organizational issues became better understood, usability was further elaborated to incorporate notions like awareness of and access to other people in the performance of a work task, and support for existing workplace roles and practices.

2. USABILITY BEYOND SIMPLICITY Even during the 1980s, the notion of usability as simplicity tout court was questioned. This questioning came from two distinct perspectives. First, from the European traditions of work psychology (Frese, Ulich and Dzida, 1987), and more broadly of activity theory (Bødker, 1991; Bertelsen and Bødker, 2003), researchers cautioned that designing technology to support work activity unavoidably transforms the work itself. If simplification is the sole driving value in this redesign, then the work activity may be simplified in ways that make it less flexible with respect to exception handling, and therefore less robust, but also make it less valuable to the person performing the work. The former

502

Product Experience

consequence would make the overall work system more vulnerable to breakdown. The latter consequence, which is called ‘deskilling’, would make it less humane for the people involved. Deskilling is a familiar theme in the psychology and sociology of work going back to the first industrial revolution. This European line of critique for taking usability as simplicity argued that using a computer tool for a practical goal is always part of a larger activity system, and that analyzing human–computer interaction in isolation from that larger activity context deprives it of its meaning. From this it follows that organizational usability – designing rich and challenging job roles – is an integral part of HCI design. And conversely, designing user interfaces and applications to be easy to learn and easy to use is insufficient, or worse, unless designers directly consider the workplace context in which those systems will be used. Given this, European researchers realized that workers should always be directly involved in the design of information technology that would be incorporated into their work, and thereby eventually redesign their work. Including workers as actors in design processes for which they are stakeholders is a tenet of participatory design. However, this critique, though well known in Europe since the 1960s, was not widely appreciated in North America until after appearance of Susanne Bødker’s 1991 book, Through the Interface. The second perspective calling into question the view of usability as simplicity was active user theory (Bruner, 1996). In the cognitive psychology of the 1980s – which fairly directly shaped the first generation of HCI theory – there were two, somewhat complementary, research programs. One focused on the kinds of information processing operations that constituted human performance in tasks like HCI; this is where usabilityas-simplicity had its theoretical roots. Fitts’ Law, which describes how people are better at pointing to large, proximal targets than to small, distant targets (Fitts, 1954), and the keystroke model, a simple cognitive user model focused on low level tasks (Card, Moran and Newell, 1980), are hallmarks of the information processing program. The other program focused on describing how and why people actively strategize and manipulate the contents of their own thought to more successfully manage their own behavior and experience. Examples of this work in early HCI were Malone (1982) and Carroll’s (1982) investigations of computer games as models for ordinary HCI, and Carroll and Mack’s (1983) study of what they called ‘active learning’ of office systems. Active user theory argued that usability cannot be managed in design merely by making user interfaces easy to use and easy to learn. Rather, it contended that user interfaces and applications should be designed to present challenges, evoke curiosity, engage critical thinking, suggest analogies and metaphors, and most of all afford activities that users could try out and learn from. Active user theory shifted more attention to emotional qualities and concomitants of the user’s experience, that is, beyond direct cognitive consequences. For example, Carroll and Thomas (1987) argued that usability research and methods should more directly incorporate fun as a determinant of the experience of usability. By the beginning of the 1990s, a broader conception of usability – as user experience of technology – was already widely used and had begun to be articulated.

3. DIGITAL USER EXPERIENCE DIRECTIONS In an early paper specifically referring to the user experience for the ACM/interactions design award, Alben (1996) analyzed five aspects of the experience of a product, which if ‘successful and engaging’, the product is deemed ‘valuable to the user’ (p. 12). Three of the five aspects can be directly mapped onto the conventional usability criteria.

The useful interface experience: The role and transformation of usability

503

1. The way it feels in the user’s hands. 2. How well the user understands how it works (effectiveness). 3. How the user feels about it while using it. 4. How well it serves the user’s purposes (satisfaction). 5. How well it fits into the entire context in which the user is using it (satisfaction). In essence, Alben presented a framework for qualities technology should strive for in creating effective experiences. If the goals of the system are too narrow then it will be an experience-poor device. If the goals of the system are expanded to encompass not only what people want, but also what they do not want, as well as providing the flexibility to do with it as they may, then the system becomes experience-rich. However, weighing all of these design criteria to support traditional usability goals, as well as ever-expanding user-experience goals, means that no one type of design consideration should prevail over any other in a product’s conceptualization and implementation. Since then, the term ‘user experience’ has gained currency in the HCI literature, even in the absence of a precise definition and guidelines in designing for it. This has led to Forlizzi and Ford’s (2000) analysis of the literature which yielded three distinct uses of the term: Experience; an experience; and experience as a story (Forlizzi and Ford, 2000). ‘Experience’ refers to the physical moment of being in and of a situation. This is derived from the ecological approach to cognition, which conceives of people as immersed in and part of their environment. ‘An experience’ refers to an event which has a beginning and an end. ‘Experience as a story’ relies on the significance of the user’s reflection on his/her own experience; the unique relating of an experience reveals what the storyteller feels are the most significant aspects of the experience. Forlizzi along with Battarbee (2004) expanded upon this framework from a multidisciplinary view of the design of interactive systems by extending ‘experience as a story’ to ‘co-experience’, which included interacting with others through the technology. Identified by Battarbee and Koskinen (2005), this was seen as a missing perspective in most conceptualizations of experience. The authors justified the need for this perspective by explaining that ‘ [p]eople as individuals depend on others for all that makes them truly human’ (p. 7). Technology meaning and experience are a product of social experiences and are mutable based on those social interactions. More recently, Hassenzahl and Tractinsky (2006) proposed a research direction in user experience to focus on pleasure and promoting positive experience, rather than mitigating problems. They claimed that current literature on user experience falls into three perspectives: Addressing human needs beyond the instrumental; affective and emotional aspects; and the nature of experience. The first perspective sees user experience as building past task needs to seeing a more holistic HCI, by addressing both pragmatic as well as hedonic needs. Directions in affective and emotional aspect would be in contrast to the technology-driven affective computing field by focusing on the human side of the equation, as well as supporting positive emotions as opposed to minimizing negative emotions. The third perspective on the nature of experience discusses the temporality and situatedness of experiences. The authors refer back to Forlizzi and Battarbee’s three types of temporal experiences, as well as provide examples of how an object can elicit different experiences depending on where it is located. Finally, McCarthy and Wright’s (2004) book, Technology as Experience, is arguably one of the most cited and well developed attempts to lay out a theoretical understanding of experience and technology. McCarthy and Wright view experience as something that cannot be explicitly and deliberately designed for. Based on this they regard experience as residing outside of or beyond usability. In their view, the notion that a user experience can

504

Product Experience

be designed betrays a technological deterministic frame of mind – one that neglects the agency of people interacting with their environment and with the technology itself. Thus, the perception of a product’s experience can vary among people, as well as within the same person at varying times. McCarthy and Wright recommend that practitioners and researchers eschew the idea that they can design an experience in favor of designing for experience. Forlizzi and Battarbee’s framework, Hassenzahl and Tractinsky’s research agenda, and most importantly McCarthy and Wright’s thesis have laid the foundation of experience in technology, and a direction for further work and discussion. Along with many other attempts at theories and frameworks of digital product experiences there is still no unified conceptualization of experience for useful interfaces. In this chapter we make no claims at providing the all encompassing theory or a complete catalog of experiences with useful digital systems. The following is an overview of the state of research in experience with technology and a short catalog of the types of experiences designers can support – those users like, those users don’t like, and those which have pushed the boundaries of what an experience can entail.

3.1. Desirable technology experiences Much of the discussion and research on the user experience has centered on the encouragement of positive and desirable experiences. There are many experiences that people explicitly desire, need, and want with useful digital products that ought to be realized and addressed in future interface and systems design. Identifying desired experiences with specific technology can lead to actionable design criteria. These are the experiences which could be fulfilled within the use of various technologies. Websites are continually an important domain for both interaction and user experience designers. Credibility and trustworthiness are two emotionally laden perceptions which affect website experience. Fogg et al. (2001) conducted a large study to identify factors which influence credibility (or believability). They defined credibility as a perceived quality based on trustworthiness and expertise. After running a 51-question survey on 1410 people, seven factors were found to influence credibility (Figure 21.1). Ease of use, a usability derivative, was the second most important factor. From these seven factors, Fogg et al. (2001) outlined seven design implications (summarized in Table 21.1) to create a credible web experience. Related to credibility, perceived trustworthiness has also been a topic of interest in the web design community. Egger’s (2001) study begins with a theoretical account and

FIGURE 21.1 The effect of seven determinants on perceived web credibility (Fogg et al., 2001, p. 65).

The useful interface experience: The role and transformation of usability

505

TABLE 21.1 Seven design guidelines for influencing web credibility (Fogg et al., 2001, p. 67: http//captology.stanford.edu) 1. Design Websites to convey the ‘real world’ aspect of the organization Communicate the legitimacy and accessibility of the organization. 2. Make Websites easy to use A simple, usable Website is perceived as more credible than a site that has extravagant features but is lacking in usability. 3. Include markers of expertise List an author’s credentials or include citations and references. 4. Include markers of trustworthiness Conveying the honest, unbiased nature of the Website. 5. Tailor the user experience People think a site is more credible when it acknowledges that the individual has visited it before. 6. Avoid overly commercial elements on a Website We accept commercialization to an extent but become wary when it is overdone. 7. Avoid the pitfalls of amateurism Dazzling features do not overcome small errors.

model of trust for e-commerce. Previous work by Egger had formed the four dimension model of trust for electronic commerce – pre-interactional filters form an initial trust value, interface properties cause the user to reassess that initial trust value, consideration of the company’s competence from informational content leads to the third reassessment, and any further interactions over time through relationship management continue to have an affect on a e-commerce site’s trustworthiness. From this model he derived design principles (Table 21.2), which overlap somewhat with previous work on trust in user experience design, but more importantly highlight the importance of the user’s preconceived trust value before the interaction has even commenced (pre-interactional filters). After that, the subsequent interaction with the interface (interface properties), information (information content), and company itself (relationship management) leads to the user’s continual reassessment of their trust value. Swallow, Blythe and Wright (2005) examined the user experience of smartPhones with the goal of demonstrating methods for evaluating user experience design ideas. They employed a grounded theory technique and carried out three case studies over a three week period, and performed interviews at both the beginning and end of the studies. Four themes emerged from this process which indicate the need for the support of multiple experiences for any one technology (Figure 21.2). The most pervasive theme was that of Identity which encompassed expressions that were both private (e.g. personalized desktop pictures that no one else would ever see) and public (e.g. image conveyed by look or use of it). Sociability was another key theme which was unsurprising due to the very nature of the mobile phone. Sociability experiences centered on managing relationships such as saving text messages for confirmation of plans, using text messages to keep in touch asynchronously, and maintaining different inclusion rules for their contact lists. Security issues were a third emerging theme; but as opposed to system security, phone security experiences had more to do with personal security. Within this theme there were negative as well as positive security experiences. For instance, negative experiences included such issues as worrying about losing such an expensive device or having it stolen, while positive experiences included reassurance in unfortunate circumstances. Organization, the last theme which emerged from this study, highlighted the need for consideration of relevance of functionality for different desired

506

Product Experience

TABLE 21.2 Design principles for enhancing perceived trustworthiness in e-commerce (Egger, 2001) Pre-interactional filters Know the customers Examine attitudes towards the industry Analyze the company’s brand equity Take advantage of trusted sources of information Interface properties Take advantage of a familiar brand experience (traditional companies) Create an interactive brand experience (dotcoms) Convey a professional image Provide easy access Be customer-centric Let the customer be in control Informational content Create value Be credible Be transparent Present the company Describe the company’s achievements Communicate the company’s values Address security concerns up-front Provide reassurance in case of fraud Provide a privacy policy Let customers be in control of their data Be transparent about the fine print

ni

za

tio

cu

ga

n

Se

Or

rit y

ty

ili

Ide

ab

ci

nti

So

ty

Relationship management Provide different means of contact Handle customer inquiries efficiently Provide feedback about the order Choose trustworthy commercial partners Provide an effective after-sales service

FIGURE 21.2 SmartPhone experiential color wheel (Swallow, Blythe and Wright, 2005, p. 97).

user experiences. For instance, a calendar feature was deemed very desirable for a busy exchange broker but unnecessary for a student or mother of two. These case studies and reflection interviews showed that when considering one experience in relation to another, difficult design questions arise. For instance, how does one design for sociability while still maintaining security? Observations and interviews of elders and their caregivers (including family and friends) led to the finding that social, emotional, and environmental needs are important

The useful interface experience: The role and transformation of usability

507

in the aging experience (Hirsch et al., 2000). When assistive products do not support these needs, they are more likely to be rejected. Although these assistive products are necessary for continued health, independence, and most importantly safety, they often ignore the importance of the social and psychological dimension. Some interviewees shunned the use of some useful devices, such as a hearing aid or emergency-alert necklace, because of a desire to not be reliant on a device due to stigmatization. In addition, stigmatizing assistive products can cause social isolation, such as limiting interaction outside the home because of the use of a wheelchair. Hirsch et al. concluded that aesthetics and perception are an addition to usability. Both can lead to greater adoption and more positive experience outcomes. They assert that usability must not only entail functional use, but also how it fits holistically into the user’s life. There is a new direction in experience design and research to look beyond stated user needs. This is motivated by a need to enhance experience, not just mimic it. Boehner et al. (2005), follow the path of reflective design – engaging users in critical reflection on the role technology typically plays in their experiences – and designing for new experiences without reifying existing ones. In order to identify marginalized aspects of the user experience they hybridized tool and art by using bird songs as an indication of the absence of visitors in areas of a museum. This use of nature indoors was to entice visitors to explore other areas of the museum which are not as high trafficked. Through this system they supported new experiences and needs by encouraging the patrons to experience the museum in alternative ways, while at the same time not distracting the patrons from the experience of the art itself. Identifying needs and values can lead to very different experiences for different products. Many times a product can be defined by a multitude of conflicting or overlapping needs. Emotion seems to be an underlying drive behind all needs – and can even eclipse survival needs. Although needs and values can be stated by the users some experiences are unique to technology and are still waiting to be uncovered.

3.2. Adverse technology experiences Along the same lines as the research in the previous section, user experience research has also focused on the mitigation of negative and undesirable experiences. Discovering those experiences which users want and need might be an easier task than identifying what they do not desire or do not need. A common thought is that ease of use is the most important factor to mitigate undesirable experiences; however, the following are examples of experiences which people do not need or want, but engage in nonetheless with technology. Despite our assumption that usability is an important aspect of the user experience of useful products we do not deny that other factors can take precedence. Minor usability problems can be perceived as having a greater negative affect on the user experience than large usability problems. In Mentis and Gay (2003), the majority of remembered frustrating incidents with a variety of technology were attributed to incidents such as autoformatting, computer errors or bugs, a slow or dropped Internet connection, and pop-ups. These incidents are external to the user’s cognitive processing and interrupt the user’s task; thus, they take away control from the user. When users decide on what goal they want to achieve, they plan the steps that are needed to complete that goal; however, when there is an unanticipated interruption, the user has to compensate for that interruption, breaking the cognitive flow. These frustrating incidents were often very mild usability problems (see Table 21.3 for examples). Findings such as these can lead to design implications for creating a better overall user experience by not interrupting a user’s cognitive flow.

508

Product Experience

TABLE 21.3 User stated examples of interrupting frustrating incidents (Mentis and Gay, 2003) I hate the little pop-up windows that appear whenever you open the browser. You have to click them all closed so you can see the window you are working on. Using my webcam crashes my computer quite often. I hate it when Word formats a document automatically such as putting in bullets when it is not the format that you want. You keep having to change it. What’s frustrating is when my Internet connection goes bad.

This also would apply to environments outside of desktop environments. Bell and Kaye (2002) have noted that the history of technology in the home, particularly kitchens, has focused on streamlining functionality, without acknowledging that people’s fondest memories of kitchens have to do with smells, conversations, and food rather than how fast they were able to cook a meal. Taylorism’s reach has taken a workplace ideal and inserted it into an arena where efficiency is seen as intruding into the experience which is wanted and needed (Taylor, 2003). Many corporate research and design organizations have conceptualized the home of the future with images of uninhabitable, sterile, or ideal lifestyles, which look great in catalogs or on displays, but lose their allure when compared against a real home environment. Efficiency has become the underlying hallmark of most of these conceptualized homes. In contrast, academic research on home environments has been much more realistic, and has been centered on building inhabitable home environments for studying ideas and building requirements. Cultural differences in how a home is designed and how rooms are used was also raised by the authors. For instance, in Italy the kitchen was the family gathering and conversation area whereas, in the United States, the family room was typically the focus for such activity. The authors argue that in order to think about how to build technology in a home, the designer must understand and appreciate the experience of the home. They concluded that in order to design for experience over efficiency, a designer has to understand the use of objects in a cultural context. Their conception of enhancing the user experience is one in which technology facilitates and enriches experiences that are already enjoyable, without trying to replace or interrupt them. For researchers and designers it can be difficult to accept that the user’s experiential goal is not to interact with an interface for a digital product, but to use and enjoy the product. Designing technology for domestic spaces should not take away from the experience at hand. Cooking is a social and emotional activity for families, not just a prerequisite to eating. It might not benefit from technological interventions such as automation. In order to support new experiences in the kitchen without distracting from already engaging and enjoyable experiences, Bell and Kaye implemented a few examples of what people might desire to do in their kitchens. Robocrop was an implementation of a hydroponic kitchen garden for vegetables and herbs in order to support those who do not have the means or opportunity for an outside garden. PC Dinners used a barcode on prepackaged foods to indicate optimal cooking time in addition to an experience unique to the type of food to be enjoyed (e.g. music which is related to the country of origin). Mr Java was a coffee machine which knew the preferred coffee order of a user based on the RFID tag at the bottom of their favorite mug. Finally, inStink shared the smells of one kitchen with another. For instance, a child who has moved away from home can still feel connected to their family’s activities by the familiar smell of an old family recipe wafting into their kitchen at dinner time.

The useful interface experience: The role and transformation of usability

509

FIGURE 21.3 A patron’s explicitly expressed sense of isolation (Hsi, 2002).

Many times technologies are implemented within a framework of values and needs predetermined by requirements. Failures of experience from such projects can yield an amazing level of understanding about user desires. An example of this is the case of The Electronic Guidebook that was developed as a mobile resource in a hands-on museum (Hsi, 2002). The researchers gathered a set of user interactions to be developed based on previous research and literature, their own observations, and a determination to support learning of the installations at the museum. The latter requirement was the most important function as defined by the researchers, even though it was not explicitly stated as such. By predefining a set of interactions based on values not shared by museum visitors, the electronic guide undermined one of the most important aspects of the museum experience – social connection. Despite the evidence of users’ successfully switching attention or using the guide in concert or sharing with others, a sense of isolation was related by many of the users (see Figure 21.3 for one such quote). What are we learning about the user experience from these failed attempts or negative reactions to useful interfaces? That we are fundamentally social beings; that we need to connect and stay connected is probably more important than any other experience; and when we are within an experience – whether it is social in nature or not – we don’t like interruptions. In a sense, the best experiences are seamless – seamless with our felt lives and with other technology experiences. By understanding what experiences and interactions are important to the users – what goals are motivating their actions and what are their values – one can design a better system by supporting those needs and values. At the very least, one can ensure that other experiences are successful as long as they do not interfere with those which are more highly valued. In this case, conventional usability concerns may initially not be considered all that important, but bad usability can affect the overall experience by interrupting more important needs and values.

3.3. Unexpected technology experiences Finally, to take us a further step from conventional beliefs, there are some experiences which are captured by adhering to neither narrowly practical usability standards nor idealistically positive (hedonic) emotionality. The previous section discussed experiences users do not enjoy. This knowledge is oftentimes discovered post initial conceptualization. Product design begins with a product team’s motivation to support efficient interactions and oftentimes they are led by a naïve view which mandates that positive emotions lead to utopian experiences. Challenges to this view have been met with applause and have led designers to embrace some surprising experience outcomes. Unlike the previous two sections, a novel concept on the needs of the user is breaking previously held conceptions of user experience; not all which is unexpected is considered

510

Product Experience

undesirable. Unexpected technology experiences include both those experiences which surprise the user, as well as those experiences which emerge unexpectedly to the designer. Positive emotions, happy experiences, and directly useful technology are beginning to be shown to be just the tip of the experience iceberg. In their piece, Design Noir, Dunne and Raby (2001) argue for the inclusion of ‘darker’, more genuine human needs in the design of technology. As in film noir, with anti-heroes and alternatives to the happily ever after ending, design noir encompasses products which raise questions, provoke problems, and appeal to the emotional needs of the user. They do not try to meet any pragmatic need, nor do they attempt to fit into a sanitized world. Dunne and Raby refer to the real world where the misuse and abuse of electronic products exists and emotions are complex and nuanced. However, most useful interfaces are designed for an extremely narrow range of emotions. To this end, they embarked on the Placebo Project which aimed to develop objects which, although they technically could not counteract electromagnetic waves, could provide some comfort to those who are concerned with this ever-present force around them. One example of such an implementation was the Phone Table. When users returned home, they would place their mobile phones inside of the table which would glow when the phone was called. This transferred the electromagnetic waves into something useful – lighting the table – and thus provided a ‘more gentle language’ than the harsh ring of the mobile phone. Dunne and Raby’s view and their example implementations have spurred a new direction for experience design which addresses needs not normally considered ‘appropriate’ or ‘useful’. Isbister et al. (2000) demonstrated such a possibility in their work on a computer character to aid Japanese and American students in getting to know each other. Isbister et al. showed that socially appropriate topics (e.g. asking students to tell about their hobbies) are substantially less effective than socially inappropriate topics (e.g. asking users how much money they earn or whether they support abortion). The character’s inappropriate conversation provided a topic for the different cultures to discuss. Throughout the research in fulfilling experiences, the focus has been on supporting technology experiences. However, what about supporting experiences outside of technology with a technology experience? The idea that sanitized values are as important in the home as in the workplace was also challenged by Gaver and Martin (2000). Through a conceptual design workshop activity, they identified six values that are overlooked when porting technology design values from work to home or other locations. Impressionistic displays, diversions, influence, intimacy, insight, and mystery were described as experiences their designs were meant to encourage – although non-instrumental to the task at hand. Diversion is a particularly unexpected technology experience if one considers the usual goals of usability such as efficiency. Yet, diversions can lead to enjoyment through the encouragement of exploration, novelty, and trepidation. The (De)Tour Guide (Figure 21.4) was one such embodiment of these experiences where, through various cues, users discovered city landmarks almost by chance. Such a device can then be used to go astray from any directed sightseeing expedition or to become purposely lost. Gaver has also perpetuated the supporting of experiences through ambiguity – by directly designing for it or by being open to the occurrence of it (Gaver, Beaver and Benford, 2003). Ambiguity is posited as encouraging closer personal experiences through engagement (this echoes notions of the active user). As they point out, the world around us is rife with ambiguity, and yet people seem to manage. By capitalizing on this resource, designers can engage users, allow for multiple interpretations and support technological limitations. Through a number of examples they describe how these benefits are manifest in technology.

The useful interface experience: The role and transformation of usability

511

FIGURE 21.4 The (De)Tour Guide – experience through diversion (Gaver and Martin, 2000, p. 210).

FIGURE 21.5 Two ambiguous technology experiences (Gaver, Beaver and Benford, 2003, p. 234).

For example, the Sloganbench (Figure 21.5, top) and Imagebank (Figure 21.5, bottom) are two systems in the Projected Realities Project. The Sloganbenches were usable pieces of public furniture which contained fabric scrolls in the back. These scrolls displayed handwritten slogans by the elders in the area. The knowledge of where these slogans came from were only spread by word and little other context was provided. Thus viewers had to interpret what emotions and attitudes were being reflected. The Imagebank consisted of five monitors in a wood casing which showed images provided by the elders in the area. These images represented their lives and again had little context as to what was being displayed and why. Both objects mixed the familiar with the strange – attraction to this ambiguity engaged passersby. These examples and new directions raise the question of what is the role of usability in these types of experiences. Should we just throw out the notion altogether? Gaver et al. (2003) claim ambiguity is the ‘nemesis’ of usability and usefulness – but that still connotes a relationship.

512

Product Experience

4. USABILITY AS AN EVOLVING CONCEPT Usability is an ever-expanding, ever-growing concept; the complexity and richness of which reflect the development of the HCI field itself. We should expect it to continue to grow as we learn more about what sorts of design considerations are necessary and effective in various contexts. It is a sign that we are onto something important. The process of constructing a comprehensive and integrated analysis of the user’s experience of usability has led to significant technical progress. Further articulating the roles and interactions among a variety of important components of the user’s experience is itself likely to elicit a deeper and more significant consideration of each individual component. As the use of computers expands into leisure activity, family interaction, and civic life, our understanding of usability may broaden further to encompass qualities like eudemonic well-being (Ryan and Deci, 2001), collective efficacy (Bandura, 1997), cultural identity (Clifford, 1986), and social capital (Putnam, 2000). Perhaps the most significant consequence of human–computer interaction is larger-scale and/or longer-term than those investigated so far. In human–computer interaction, health and well-being are usually thought of with respect to workplace ergonomics – problems of physical posture and manipulation that can be long-term, and psychological stressors that are generally short-term. Technology clearly can have broader effects than this. People who report being happy when engaged in social interaction, report being bored and unhappy when watching television (Kubey and Csikszentmihalyi, 1990). Television viewing is linked to reduced physical activity, and poorer physical and mental health (Anderson et al., 1998). These are issues of usability-in-the-large. Indeed, concerns about impacts of Internet computing on health and well-being have already received much attention (Kraut et al., 1998). People’s beliefs about their own specific capabilities exert powerful influences on learning and performance outcomes (Bandura, 1997). Perceptions of high self-efficacy for an activity domain cause individuals to set more challenging goals, to work harder on difficult aspects of tasks, to master new competencies, and to achieve more. Similarly, group members’ beliefs about collective efficacy predict group performance. Collective efficacy is a function of interrelated personal efficacy beliefs, including both members’ appraisals of personal capability for functions performed within the group (for example, the belief that there is someone you can turn to for advice about handling problems with your family) and members’ appraisals of the group’s capability (for example, the belief that one’s community can improve the quality of public schools without help from the state government). The tools people use, such as computers and software, can affect their perceptions of both self-efficacy and collective efficacy, and thereby enhance or impair future learning and performance. Although it can seem like the world has been thoroughly homogenized, we are still clearly not all members of the same culture. Age, gender, ethnicity, race, nationality, residence, first language, education, occupation, family status, disabilities, and special needs – all entail folkways, mores, and concerns. Cultural identity contributes to the creativity of the self and to the diversity of society, but it is a complex social construction that must be accommodated, encouraged, and celebrated. It is under assault throughout the world by mass production of all sorts, including one-size-fits-all software and user interfaces. Although the risks of poor usability with respect to cultural identity are becoming better understood, it is still not well-understood how to manage the design of human–computer interactions to ensure preferred outcomes (Prabhu and delGaldo, 1999; Stephanidis, 2000). Social capital is a key concept in contemporary studies of the decline of civil society and the rise of utilitarian individualism (Bellah et al., 1986; Putnam, 2000). The creation

The useful interface experience: The role and transformation of usability

513

of social capital involves the establishment and maintenance of social networks, shared goals and values, and social norms of reciprocity. Social capital is not a transient state, like satisfaction or frustration, or a discrete achievement, either present or absent; it is a social resource that is developed. It is not an individual state or achievement; it is a collective good benefiting everyone who lives in the community. Systems and applications that enhance social capital will have greater long-term usability for the members of the community; those that diminish the social capital of organizations are less usable. Usability is the touchstone concept of human–computer interaction. As our understanding of the phenomena of human–computer interaction grows, the concept of usability has grown. The experience of usable products is far richer than merely not feeling overwhelmed by unnecessary complexity. In the past 25 years, we have learned quite a bit about the experience of usability, enough to know that we have much more to learn.

ACKNOWLEDGMENT The authors are grateful to Elsevier, San Diego, and ACM Publications for permission to reprint the figures.

REFERENCES Alben, L. (1996). Quality of experience: Defining the criteria for effective interaction design. Interactions, 3(3), 11–15. Anderson, R. E., Crespo, C. J., Bartlett, S. J., Cheskin, L. J. and Pratt, M. (1998). Relationship of activity and television watching with body weight and level of fatness among children. Journal of the American Medical Association, 279, 938–942. Bandura, A. (1997). Self-efficacy: The exercise of control. New York: W. H. Freeman. Battarbee, K. and Koskinen, I. (2005). Co-experience: User experience as interaction. CoDesign, 1(1), 5–18. Bell, G. and Kaye, J. (2002). Designing technology for domesticated spaces: A kitchen manifesto. Gastronomica 2(2), 46–62. Bellah, R., Madsen, R., Sullivan, W., Swindler, A. and Tipton, S. (1986). Habits of the heart: Individualism and commitment in American life. Berkeley: University of California Press. Bertelsen, O. W. and Bødker, S. (2003). Activity theory. In: J .M. Carroll (Ed.) HCI models, theories, and frameworks: Toward a multidisciplinary science, pp. 291–324. San Francisco: Morgan-Kaufmann. Blythe, M. and Wright, P. (2003). Introduction – From usability to enjoyment. In: Blythe, M., Overbeeke, K., Monk, A. and Wright, P. (Eds.) Funology: From usability to enjoyment. Bødker, S. (1991). Through the interface – a human activity approach to user interface design. Hillsdale, NJ: Lawrence Erlbaum Associates. Boehner, K., Sengers, P., Medynskiy, Y. and Gay, G. (2005). Opening the frame of the art museum: Technology as art and tool. Proceedings of Digital Arts and Culture Conference. Copenhagen, Denmark. Bruner, J. (1996). The culture of education. Cambridge, MA: Harvard University Press. Card, S. K., Moran, T. P. and Newell, A. (1980). Computer text-editing: An information-processing analysis of a routine cognitive skill. Cognitive Psychology, 12, 32–74. Carroll, J. (1982). The adventure of getting to know a computer. IEEE Computer 15(11), 49–58. Carroll, J. M. and Mack, R. (1983). Actively learning to use a word processor. In: W. Cooper (Ed.) Cognitive aspects of skilled typewriting, pp. 259–281. New York, NY: Springer-Verlag. Carroll, J. M. and Rosson, M. B. (1985). Usability specifications as a tool in iterative development. In: H. R. Hartson (Ed.) Advances in human–computer interaction. Norwood, NJ: Ablex. Carroll, J. M. and Thomas, J. C. (1987). Fun. ACM SIGCHI Bulletin, 19(3), 21–24. Clifford J. 1986. Introduction: Partial truths. In: J. Clifford and G. E. Marcus (Eds.) Writing cultures: The poetics and politics of writing ethnography, pp. 1–26. Berkeley: University of California Press. Dunne, A. and Raby, F. (2001). Design noir: The secret life of electronic objects. Basel, Switzerland: August/Birkhaeuser. Egger, F. N. (2001). Affective design of e-commerce user interfaces: How to maximise perceived trustworthiness. Proceedings of the International Conference on Affective Human Factors Design, Singapore, pp. 317–324. London: Asean Academic Press.

514

Product Experience

Fitts, P. M. (1954). The information capacity of the human motor system in controlling the amplitude of movement. Journal of Experimental Psychology, 47, 381–391. Fogg, B., Marshall, J., Laraki, O., et al. (2001). What makes websites credible? A report on a large quantitative study. Proceedings of the Conference on Human Factors in Computing, Seattle, WA, pp. 61–68. New York: ACM Press. Forlizzi, J. and Battarbee, K. (2004). Understanding experience in interactive systems. Proceedings of Designing Interactive Systems, Cambridge, MA. Forlizzi J. and Ford, S. (2000). The building blocks of experience: An early framework for interaction designers. Proceedings of Designing Interactive Systems, New York, NY, pp. 419–423. New York: ACM Press. Frese, M., Ulich, E. and Dzida, W. (Eds.) (1987). Psychological issues of human–computer interaction in the workplace. Amsterdam: North-Holland. Gaver, W. W., Beaver, J. and Benford, S. (2003). Ambiguity as a resource for design. Proceedings of the Conference on Human Factors in Computing Systems, Ft. Lauderdale, FL, pp. 233–240. New York: ACM Press. Gaver, W. W. and Martin, H. (2000). Alternatives. Exploring information appliances through conceptual design proposals. Proceedings of the Conference on Human Factors in Computing, The Hague, The Netherlands, pp. 209–216. New York: ACM Press. Hassenzahl, M. and Tractinsky, N. (2006). User experience – a research agenda. Behaviour and Information Technology, 25(2), 91–97. Hirsch, T., Forlizzi, J., Hyder, E., et al. (2000). The ELDer project: Social, emotional, and environmental factors in the design of eldercare technologies. Proceedings of the Conference on Universal Usability, Arlington, VA, pp. 72–79. New York: ACM Press. Hsi, S. (2002). The electronic guidebook: A study of user experiences using mobile web content in a museum setting. Proceedings of the IEEE International Workshop on Wireless and Mobile Technologies in Education, pp. 48–54. Washington, DC: IEEE Computer Society. IBM (2006). What is user experience design? Retrieved December 28, 2006, from http://www-03.ibm. com/easy/page/10. Isbister, K., Nakanishi, H., Ishida, T. and Nass, C. (2000). Helper agent: Designing an assistant for humanhuman interaction in a virtual meeting space. Chi Letters, 2, 1. ISO (1998). Ergonomic requirements for office work with visual display terminals (VDTs) – Part 11: Guidance on usability (ISO 9241–11:1998(E)). Kraut, R., Lundmark, V., Patterson, M., et al. (1998). Internet paradox: A social technology that reduces social involvement and psychological well-being? American Psychologist, 53(9), 1017–1031. Kubey, R. and Csikszentmihalyi, M. (1990). Television and the quality of life: How viewing shapes everyday experience. Hillsdale, NJ: Erlbaum. Mack, R. L., Lewis, C. H. and Carroll, J. M. (1983). Learning to use office systems: problems and prospects. ACM Transactions on Office Information Systems, 1, 254–271. Malone, T. M. (1982). Heuristics for designing enjoyable user interfaces: Lessons from computer games. In: J. C. Thomas and M. L. Schneider (Eds.) Human Factors in Computing Systems. Norwood, NJ: Ablex Publishing Corporation. McCarthy, J. and Wright, P. (2004). Technology as Experience. Cambridge, MA: MIT Press. Mentis, H. M. and Gay, G. K. (2003). User recalled occurrences of usability errors: Implications on the user experience. Extended Abstracts of the Conference on Human Factors in Computing Systems, Ft. Lauderdale, FL, pp. 736–737. New York: ACM Press. Monk, A. Fun, communication, and dependability: Extending the concept of usability. Closing Plenary at HCI2002. http://www-users.york.ac.uk/~am1/MonkHCI02.PDF. Prabhu, G. V. and delGaldo, E. M. (Eds.) (1999). Designing for Global Markets: Proceedings of the First International Workshop on Internationalization of Products and Systems. (Rochester, NY, May 20–22). Rochester, NY: Backhouse Press. Preece, J., Rogers, Y. and Sharp, H. (2002). Interaction Design: Beyond Human–Computer Interaction. New York, NY: John Wiley and Sons. Putnam, R. (2000). Bowling alone: The collapse and revival of American community. New York: Simon and Schuster. Ryan, R. M. and Deci, E. L. (2001). On happiness and human potentials: A review of hedonic and eudaimonic well-being. Annual Review of Psychology, 52, 141–166. Sidney, S., Sternfeld, B., Haskell, W. L., et al. (1998). Television viewing and cardiovascular risk factors in young adults: The CARDIA study. Annals of Epidemiology, 6(2), 154–159. Stephanidis, C. (Ed.) (2000). User interfaces for all: Concepts, methods, and tools. Mahwah, NJ: Erlbaum. Swallow, D., Blythe, M. and Wright, P. (2005). Grounding experience: Relating theory and method to evaluate the user experience of smart phones. Proceedings of the 2005 annual conference on European association of cognitive ergonomics, Chania, Greece, pp. 91–98, University of Athens. Taylor, F. W. (2003). Scientific Management. London: Routledge (original editions 1903–1912).

22

THE EXPERIENCE OF INTELLIGENT PRODUCTS DAVID V. KEYSON Delft University of Technology, Delft, The Netherlands

1. BACKGROUND Our world seems to be heading towards a situation where daily-life durable products will contain powerful chips, advanced sensors, inter device communication via intelligent telecommunication networks and advanced user-input and display technologies. This will apply to a very large share of the consumer and professional products in and around our homes, work, outdoors, and in leisure situations. Consider for example consumer products such as a home control system with programmable events for switching lights and heating on or off, media recording systems offering help in personal content selection, educational toys with memory and speech output capabilities, personal programmable devices such as mobile telephones, or in-car systems for performance control, information and entertainment. Given the combination of a physical product or service with: (1) technology that can recognize who the user is; (2) embedded experience knowledge relating to the user’s goal; and (3) an understating of context and a model on how to react accordingly in relation to user actions, the basic dynamics of user-product interactions and experience can be transformed in many ways. Not only may the user experience change and evolve over time, but the product itself may change in function, behavior, and personality. The degree to which the product adapts to the user may depend to a large extent on being able to accurately anticipate and accommodate changing user needs. While there is a wealth of product technology available, interactive intelligent product design, being distinctive in terms of encompassing one or more of the three above mentioned aspects, is at a very early stage in terms of generating successful real-world products. The current state of affairs may be largely attributed to the fact that technology Product Experience Copyright © 2008 Elsevier Ltd.

515

516

Product Experience

is emerging at a rate far higher than product developers are able to accommodate. Technology driven products are being produced at an increasingly higher rate with more and more features packed in, in order to compete in very dynamic markets. Little time is available to design intelligence into products in such a way that user goals can be supported (den Ouden, 2006). It could be argued that most product interaction occurs only at the product feature level rather than goal or task level. Consider for example, how often users replace their mobile phones to obtain the latest ‘observables’ features (Rogers, 1995). Mobile phones could do a lot more than serve as mobile database and communication devices, for example research is now being conducted on how to include food sensing in a telephone camera combined with knowledge on how to control obesity so as to advise the user and improve health (e.g. Pantech G-1300V Cell Phone). Examples of other products which could benefit from goal-level interaction include programmable consumer appliances, such as a DVD recorder or a combination microwave, convection, and grill oven. In short, most users are interested in achieving a certain goal, such as simply recording an upcoming movie, or cooking a certain dish, rather than interacting at the product function level which relates to hardware/software features. The basic usability of consumer products with embedded electronics continues to be a major issue. Technology remains far too hard for most folks to use and most people can only utilize a tiny fraction of the power at their disposal (Business Week, 2004). In a recent study conducted by den Ouden (2006) 85% of the consumer complaints related to new products were found to be due to usability rather than technical issues. Half of all ‘malfunctioning products’ returned to stores by consumers were in full working order, but customers could not figure out how to operate the device.

2. UNDERSTANDING INTELLIGENT PRODUCTS What are the central issues concerning the design and experience of intelligent consumer products? What makes an ‘intelligent product’ so special as compared to a product without embedded intelligence? Before addressing these questions, a common understanding of the term intelligent products is needed. In theory, any product could be seen as intelligent. For example, a well-designed chair is intelligent if it does not fall over when leaning upon it. In this chapter the notion of intelligent products is limited to those products which have some computing capability which in turn enables the product to be aware of the user, including some sense of past, current and likely potential states of interaction. Furthermore, the product has a user interface which could either be physically part of the product or distributed in the user environment. Increasingly, there is a range of real-world systems which are composed of inter-connected intelligent products, co-existing in the user environment. Such systems can be found in literature dealing with the topic of pervasive computing or ambient intelligence (Aarts and Marzano, 2003). The intelligent product experience may thus extend beyond a physical product and into the surrounding environment as products become more embedded in the user environment and able to communicate with each other. For example, Philips Medical Systems has introduced the use of mood tokens to set the ambient environment in areas equipped with Computerized Axial Tomography (CAT) Scanners. The child patient can choose a mood token in the waiting room. Upon entering the scanner area the lighting and wall animation is set according to the token (Aarts and Marzano, 2003). Similarly, some conference centers offer occupants the opportunity to set the conference meeting

The experience of intelligent products

517

room atmosphere by selecting color lighting choices. Meeting agendas could eventually be linked to such a system. Within the automotive industry one can see the development of cars equipped with Bluetooth personal area networks. The car interior environment and equipment settings can be set by reading the user’s mobile telephone identifying Bluetooth signal. Home entertainment systems such as Bang and Olufsen emphasize the interconnectivity aspects and ease of use of their products by providing single point access control to all home entertainment systems. A remote control is supplied which can recognize all the BandO products. Though several clear examples of intelligent consumer products can be found, the general notion of everyday products which are capable of remembering who the user is, and have some notion of desired states of interaction is relatively new. There is thus a lack of longitudinal or real-world studies in which the user’s experience has been well documented. As recently as 2004, the first panel session dealing with the very notion of intelligent products was held at the 10th annual Intelligent User Interface conference (de Ruyter et al., 2005). Previous work dealing with adaptive and intelligent systems has been mainly focused around computer based and robotic applications. In particular, the rise of the Internet has led to adaptive matching algorithms, such as those used in online shopping systems, including social filtering systems which provide recommendations to the user based on the history of other users’ choices. Gaining an understanding of the factors which may influence the experience of intelligent consumer products is a very complex issue, as interaction may span over time, users, space, and across products and into the environment. Nevertheless, by focusing on the human as the central player in the ever expanding arena of interaction, one can begin to discern key design issues and principles which are critical to the user experience.

3. INTELLIGENT PRODUCT FUNCTIONALITY AND THE USER In considering how an intelligent product may adapt or respond to the user, Bradshaw (1997) suggests nine potential capabilities of intelligent software agent, being: • Reactivity: the ability to selectively sense and act. • Autonomy: goal-directedness, proactive and self-learning behavior. • Collaborative behavior: can work in concert with other agents to achieve a common goal. • Inferential capability: can act on abstract task specification using prior knowledge or general goals and may have explicit models of user, situation and/or other agents. • Temporal continuity: persistence of identity and state over long periods of time. • Personality: the capability of manifesting the attributes of a ‘believable’ character such as emotion. • Adaptivity: being able to learn and improve with experience. • Mobility: being able to migrate in a self-directed way from one host platform to another. Other user interface capabilities which an intelligent software agent or intelligent product interface may include: • Flexible multimodal communication: ability to interpret user needs and select appropriate modalities of communication.

518

Product Experience

• Collaborative dialog: product can converse with user towards helping achieve specific or abstract goals based on embedded knowledge (DeKoven, 2004). • Sensing and influencing capability of user engagement.

4. USER EXPERIENCE ISSUES User experience issues in relation to intelligent products could be considered as separate aspects of interaction, but are actually highly intertwined in terms of shaping the overall user experience. Three key factors that make up the landscape of user experience are: (1) understanding and sense of being in control; (2) emotionally appealing and engaging; and (3) expected and perceived functional performance (Figure 22.1). Each of these elements is expanded upon below.

4.1. Perceived functional performance A rather basic experience issue is the degree to which an intelligent product appears or is expected to be initially useable, stable, and predictable. The physicality of a product to a great extent may influence initial perceptions about ease of use. Imagine a product that produces strange vibrations, emits unwanted scents or produces seemingly strange sounds when it is operated. The expected and perceived functional performance of a product may thus be influenced the moment it is viewed, and in more depth once it is engaged. Similar to a magic show gone wrong, credibility can be rapidly lost if the system crashes or cannot even be initially setup to a working level. To some degree a consumer product crashing is worse than a computer, since users have grown accustomed to the idea of re-booting a PC. If the time required to program a device such as a VCR or thermostat is too long in relation to the perceived gain, a user may never bother using programmable features, even though he or she may have initially bought the product for this reason.

Emotionally appealing and engaging Understanding and sense of control

Expected and perceived functional performance

FIGURE 22.1 Three central and intertwined aspects of intelligent product experience.

The experience of intelligent products

519

The product packaging can also set expectations in terms of the product functionality. For example, the Apple iPod won the I.D. Magazine Annual Design Review award in the ‘unpacking ballet’ and out-of-the-box experience (MacObserver, 2004). Users thus may expect a novel product experience based on the unique unpacking experience.

4.2. Understanding and sense of control One of the most important issues in the experience of intelligent products is the question of whether the user feels in control over the product, or does the user feel the product is in control? The issue of control should be seen along a continuum from a product taking no action, giving suggestions and collaborating on tasks, and at the far end, taking action completely autonomously. A great deal of criticism has been expressed towards intelligent products, such as cars equipped with adaptive cruise control, which take action without due regard to the user context and needs, for example when exiting the freeway no car in front is detected and the car thus speeds up (Norman, 2007). Ideally, an intelligent system should build up a sense of trust over time, such that the user will begin to delegate mundane or repetitive tasks to the product. This implies that the system has some notion of the user’s current activity and a model of cost versus benefits in taking a particular form of action. An important related issue is the degree of transparency afforded by the intelligent product, i.e. to what degree can the user understand, follow, and possibly change decisions or suggestions made by the system? Too much insight may become confusing, creating information overload, too little may create a black box syndrome, whereby the user is less likely to trust and engage the product. In the case of products with rich embedded functionality, a collaborative and narrative approach may be useful. In terms of tangible products, communicating what the product can or cannot do can be conveyed through the product physicality. Work by Djajadiningrat and colleagues has focused on the role of design in creating a balance between the physical product appearance and action potential, as communicated by perceived affordances (2004). In short, at the cognitive level the user should be able to understand what the intelligent product does and how it can be of help in performing a certain task or fulfilling a certain goal. In considering how user-product communication is currently supported, one is confronted with the problem of many of today’s products, being that they interact with the user at a low hardware-software function level (Figure 22.2). Product interaction should support higher-goal level interaction with the product. One should consider userproduct interaction in terms of the need to support a range or hierarchy of control acts from communicating rudimentary settings to higher-goal related acts (Rasmussen, 1986). In this chapter several design examples are presented which illustrate how user-product interaction can extend beyond low-level hardware/software feature control. The specific design examples are plotted in Figure 22.2 along the dimensions of increasing functionality and abstract control. Understanding and collaboration Collaboration in general involves both action and communication by the product and user or user group. Depending upon the relative knowledge of the user and system, the product agent may transition back and forth between a tutoring or more assistive role. In a more assertive role, the product agent may even attempt to persuade the user to follow a certain goal which otherwise may have been neglected. Intelligent products, such as an Ibo robot, are commonly perceived as complex due to the amount of hidden functionality and range of functions (Rijsdijk, 2006). However, studies conducted by Rich, Sidner and Lesh (2001) demonstrated that the perceived

520

Product Experience

Atmosphere Controller

l

ia

t en

t

t ar

Po

Aware Thermostat

Abstract control

Sm Music Cube

Most product UIs

Functionality

FIGURE 22.2 Common user interfaces and examples illustrated in relation to degree of functionality and the extent of control abstraction.

complexity of a product may be reduced by designing a more goal-based collaborative system rather than one based on controlling low-level functions. Goals can be communicated between the user and product such that the product can help the user meet a goal (Figure 22.3). The advantage of a fully integrated collaborative interface is that the user is still encouraged to explore and learn about the system, rather than become dependent upon a help or wizard agent. For example, the intelligent thermostat concept (Keyson, et al., 2001) shown in Figure 22.4 is based on models of collaboration stemming from the Collagen agent architecture (Rich et al., 2001). The user can communicate task level information via the right side of the screen by speaking in or selecting displayed sentences. The ‘Things to say’ list is updated based on user actions in relation to a hierarchical task model. For example, if the user says ‘go back’, then the system goes back to the previous task, rather than undoing the last button pressed at the function level. The user can

FIGURE 22.3 The Collaborative Paradigm (Rich, Sidner and Lesh 2001).

The experience of intelligent products

521

request help with a higher-level goal, such as ‘help me save energy’ and can also communicate with the product at the function-feature level (e.g. change room temperature) by selecting items via the touch screen or speaking in terms. Knowing what the user can say to a product is critical. For example DeKoven (2004) observed that without a ‘things to say window’ one user asked the intelligent thermostat who the prime minister of Canada was during a trial session. Recently, Mitsubishi Electronics Research Labs developed the concept of DiamondHelp (Figure 22.5) as a cross-product networked platform for providing collaborative interactive help for consumer appliances (Rich and Sidner, 2007). Usability studies conducted by Freudenthal (2002) found that elderly people in particular

FIGURE 22.4 The intelligent thermostat prototype.

FIGURE 22.5 Screen capture of the DiamondHelp interface for collaborative help in interacting with programmable consumer products.

522

Product Experience

appreciated being able to converse at the conversational level with the interface, rather than having to recall menu names and functions. Over-specification of product functionality when not necessary can lead to higher perceived complexity. Several home control systems have failed in the market place due in part to the complexity of the programming task. By placing the user goals central to the design rather than the product’s function, this complexity can be reduced. As discussed further below, in work by Ross and Keyson (2007), users were able to control lights, music and wall-art projections based on the common theme of desired atmosphere.

4.3. Emotionally appealing and engaging The perceived hedonic aesthetic aspects of a product such as novel and surprising (Hassenzahl et al., 2001) may evoke certain initial emotions and expectations about how the product works. As the user begins to engage the product, the sense of appeal, including ergonomic and hedonic aspects over time will play a role. Personalization and the notion of flow here is an important factor for gauging how the user experiences the product. If the required skills to use the product are too high, frustration will occur; on the other hand, a lack of challenge may lead to boredom (Csikzentmihalyi, 1975). To some extent, a user may be persuaded to change his or her attitude about the product by the product itself, and thus may be more willing to engage it. Similar to the use of advertisements, people’s attitudes can be changed in relation to interactive products, via a media. For example, McCalley et al. (2006) found that providing a short video clip on a washing machine display about the environment and the need to conserve energy, led to more frequent use of an energy savings button on the washing machine. Tangible interfaces offer the potential for designers to create a physically engaging product experience, by focusing on the aesthetics of interaction. Material expression, form, tactility, and gestures combined with form change can all play a role in shaping the how the product is experienced (Ross and Keyson, 2007). Appeal and tangible interaction Product appeal is central to the notion of creating an engaging experience (Figure 22.1). To consider this notion further, a study as summarized below was conducted to consider how the ergonomic and hedonic aspects of a physical product can influence appeal. The relationship was expected to be similar to those found for computer based screen applications, whereby ergonomic and hedonic qualities are additive and contribute directly to the sense of appeal (Hassenzahl et al., 2001). A prototype was developed called MusicCube, which contained algorithms for creating personal play lists, position and gesture interpretation for control, and color light displays relating to content, volume and song selection (Figure 22.6A and B). The lights are off when the MusicCube is off; shaking the cube causes the four color side lights to come on, indicating the cube is ready for interaction. A play list can be selected by placing the color side of the desired play list upwards. Rotating the top button allows scrolling through the songs in the play list. Music can be paused by pressing the button. The volume mode is activated by briefly holding in the button. Shaking the cube activates shuffle mode and a random song is played. Placing the cube on a surface with the top button facing downwards and pressing it causes the device to fall asleep, indicated by a fading relay activated rattle-like sound and dimming lights. In evaluating the appeal of the MusicCube an experiment was conducted in which the prototype product was compared to the iPod (Bruns and Keyson, 2005). A questionnaire along the dimensions of trust (e.g. familiarity and reliability in Jian et al., 2000),

The experience of intelligent products

523

engagement (e.g. challenge and involvement in Rozendaal and Keyson, 2005), ergonomic qualities (e.g. support and control) and hedonic qualities (e.g. interest and innovativeness) and appeal (e.g. pleasure and desirability, in Hassenzahl, 2000) was administered. The relations of the measures to common underlying factors showed high consistency with Hassenzahl’s results, whereby measures correlated with factors which could be labeled as ergonomic, hedonic, and appeal. As depicted in Figure 22.7, the Music-Cube, the version with speech feedback (MCs) was found to be more appealing on hedonic scales as compared to the iPod, while the I-IPO scored higher on ergonomic scales.

B

FIGURE 22.6 (A) MusicCube. (B) Interaction with the MusicCube (play, change play list, pause, scroll or change volume, and shuffle).

FIGURE 22.7 Results of measures for Ergonomic (EQ), Hedonic (HQ), and Appeal qualities for the MusicCube (MC) and MC with speech (MCs) compared to the Apple iPod.

524

Product Experience

The results of the MusicCube study suggest that products such as the iPod can be further improved in terms of appeal by focusing on the hedonic qualities of engagement. Novel forms of input and displays can play a key role here, given ergonomic quality is not impeded. Richness and engagement As discussed above, the notion of flow is central to creating and sustaining flow. An intelligent product may adapt so as to maintain a certain level of challenge while not exceeding the user skill level. In a study conducted by Rozendaal (2007) engagement was studied as a variable which could be systematically manipulated in a game by changing the degree of experienced control and perceived richness. Based on empirical studies, Rozendaal presents a conceptual framework in which: E ⫽ 冑RC. In taking the square root of the product of experienced richness (R) and control (C), the level of the experienced engagement (E) was found to increase when either the level of experienced richness or control was raised. Richness can be considered both in terms of behavioral or motor aspects afforded by the application functionality and visual aspects, including color, texture and form. As depicted in Figure 22.8, the game developed by Rozendaal increases incrementally in richness as the user progresses through the game challenges. Graphic and sound items are pre-scaled on a richness scale. As the sense of perceived control and richness increased, user’s sense of engagement was also found to increase. As shown in Figure 22.8, behavioral richness shifts from simple pattern making to moving and shooting objects, followed by two person games and multi-user collaborative tasks, supporting collecting behaviors. Each level includes functions from the previous level. Sound and richer visual effects are introduced as the player moves to the next level. In this manner game designers can structure and manage the introduction of new game behaviors and visual effects during the course of game playing.

Collecting

Competition/cooperation

Move/shoot

Pattern making

FIGURE 22.8 Pacman like game, with increasing levels of richness from pattern making to supporting collecting behaviors.

The experience of intelligent products

525

5. EXPERIENCE DRIVEN DESIGN As illustrated in the previous section, there is clearly a wide range of issues which may determine how well a product meets the user expectations in terms of meeting a desired experience. From a methodological viewpoint, in order to take the range of issues into account, intelligent products should ideally be designed in context. This may imply that products are developed in the field or in simulated lab environments. In essence context should drive the design process and be modeled as part of the product, rather than serve only as background information. A central aspect of an intelligent product is the ability to contain embedded formal representations about how users perform certain tasks and experience-derived knowledge on preferred styles of interaction the user may wish to perform. This assumes that experience based knowledge can be modeled, which in itself is difficult, given the lack of formal perceptual and commonsense reasoning models for everyday interaction in the environment (Minsky, 1988). Even more complex is the issue of how a product could learn over time and adjust to new desired experiences.

5.1. Guiding the experience design process Central to the experience driven design approach, is that the designer begins with understanding and modeling the user desired experience in terms of meeting a set of user goals, before actually considering a specific type of product. To illustrate how context and a desired user experience can drive the product creation process, the development of an atmosphere control system is illustrated here. The Carrousel (Ross and Keyson, 2007) is an expressive tangible user interface used to control home atmospheres. While the device interface is tangible, the underlying experience model can also be used to create rich speech and visual interfaces for atmosphere control (Vastenburg, Keyson and Ross, 2007). A 3-D model of atmosphere experience, as depicted in Figure 22.9, was developed via a series of steps (Figure 22.10), based on a research through design approach (Hekkert et al., 2001). The initial study focused on

FIGURE 22.9 Depiction of the atmosphere model (left view): a cubic space with the three model axes warmth, activity and attention and (right view) living room with atmosphere display.

526

Product Experience

FIGURE 22.10 Overview of steps taken in the development of a functional experienced-based prototype for controlling living room atmosphere.

FIGURE 22.11 Three examples of mood boards used to derive bipolar scales for rating atmospheres.

TABLE 22.1 Terms expressed by users in rating the mood boards, positioned on bipolar scales, for constructing the atmosphere model 1. 2. 3. 4. 5. 6. 7.

Chaotic Inspiring Exciting Cheerful Intimate Romantic Cozy

– – – – – – –

Orderly Uninspiring Boring Sad Distant Work Uncozy

–8. 9. 10. 11. 12. 13.

Relaxed Active Lively Natural Restless Warm

– – – – – –

Tense Passive Static Artificial Calm Cold

creating mood boards of living room atmospheres (Figure 22.11), which led to a descriptive vocabulary on atmospheres. Semantic Rating scales were then developed based upon most commonly occurring word groupings (Table 22.1) and were then used to evaluate the collages (step A and B, respectively). This led to a model for scaling atmospheres

The experience of intelligent products

527

FIGURE 22.12 The tangible expressive atmosphere controller.

as depicted in Figure 22.10 (step C). The model is populated with atmosphere markers containing music, art projections, and color light settings. The user can navigate between labels, and the system in turn can extrapolate the relevant content (Step D). The model was validated by having users rate the dynamic atmospheres using the same semantic differential scales derived from the collages (Step E). A tangible, a visual, and a speech interface were developed to facilitate the interaction between the user and the atmosphere control system (Step F). The Carrousel expressive tangible interface is depicted in Figure 22.12. The tangible interface allows users to ‘sculpt’ an atmosphere, similar to sculpting clay on a potter’s wheel. For example, a romantic atmosphere, which is high in warmth and low in activity could be expressed by slowing down the carrousel rotation speed using the hands, and by moving the flags inwards, with the flags’ wooden side facing outwards, and the metal side facing inwards. A happy, energetic atmosphere can be expressed by creating a high rotation speed, combined with upright flags that are directed outwards. Subjects were generally found to be able to sculpt atmospheres using the tangible interface on an individual basis, but less so across subjects, suggesting the need for a learning or customizable interface. In further testing it was found that in absence of the experience based atmosphere model, users were unable to create an atmosphere which they considered engaging (Vastenburg, Keyson and Ross, 2007).

6. CENTRAL DESIGN CONSIDERATIONS In this section two central shifts which may influence the way in which designers approach the experience design of intelligent products is discussed. The first being from ‘use to presence’ and the second being from ‘task-oriented to experience driven design’.

6.1. From ‘use to presence’ Closely related to the design of intelligent products is the vision of Ambient Intelligence which foresees a seamless integration of computational technology into our everyday lives (Aarts and Marzano, 2003 ). Technology would no longer claim users’ attention as it does now, but rather seamlessly merge into people’s everyday activities and environments (Weiser, 1991). Hallnäs and Redström (2002) describe this change of technology’s

528

Product Experience

relation to people as a shift from ‘use’ to ‘presence’. A ‘use’ centered view focuses on functional aspects of a system or device. Viewing a device or system from a ‘presence’ perspective touches upon a broader, existential definition of design in the life of a person. For example, a mobile phone can be described from a ‘use’ perspective by listing functional aspects (e.g. dialing, ringing, opening an audio channel, keeping an address book). From a ‘presence’ perspective mobile communication technology offers a feeling of social connectivity and mobile phones are a means of personal expression (Ling, 2004).

6.2. From ‘task-oriented to experience driven design’ In considering activities in leisure and work contexts, one needs to look beyond supporting the functional aspects of the task at hand. Considering interactive systems, the total experience of fulfilling a task should be taken into account. Social, personal, and emotional engagement and expression are salient factors in interaction, along with ease of operation and efficiency. For intelligent products to be accepted in the daily living context, values beyond system functionality need to be taken into consideration, such as playfulness, creativity and personal expression. One could argue that many previous approaches towards the design of intelligent products have incorporated one of the two above trends, but have not necessarily pursued an integral approach in which the technology presence and interaction experience are both taken into account. Most consumer products do not offer or provide the user with a level of interactive control on the input side, such that creativity and personal expression are fostered. There seems to be asymmetry between the input and output modalities of interaction. For example, in the Digital Family Portraits system (Mynatt, 2001), the everyday activities of older family members are shown in expressive photo-frames to provide younger relatives with a sense of peace of mind. The photo-frames themselves do not offer the possibility to express a response to the displayed information. Congruently, some products focusing on the user experience and personal creativity are geared towards providing mainly entertainment or mental stimulation, while neglecting to link experience rich interfaces to task based applications. An example of such a system is the BioMuse (Tanaka, 1993) installation that picks up bioelectrical signals from muscles of the body and translates them to a sound and videoscape, without any model or system understanding of what the user’s current task or context of interaction is.

7. FUTURE In many ways it is difficult to imagine how technology will be integrated via products into our everyday life in 50 years from now, given how much things have changed in just the last two decades, and how little we understand about the experience and implications of new products in our daily lives. More technology does not necessarily mean a better quality of life, indeed one could argue that developed societies have become burdened by the daily headaches of technology going wrong, and the marketing campaigns aimed at convincing consumers to buy the latest gadgets. With the mass production of electronic goods, offering more features for less money, profit margins have fallen. As such, large multinational companies have begun to shift their focus away from consumer electronics and towards consumer wellness and health related products, as a virtually untapped market. In many ways this is a refreshing shift for product developers, as the emphasis has moved from just selling products with more and more bells and whistles to considering how human well-being can be improved. In the

The experience of intelligent products

529

workplace research has begun to focus on the role of technology in creating awareness of and reducing stress. New technologies for increasing safety and health in fixed and mobile contexts are emerging. In Europe, large EU funded projects are aimed at the development of systems which can enable the elderly to live independently at home for a longer period of time, while maintaining social connectivity. Interactive local community television via Internet is emerging now as a way to bring people together via technology. Creative applications of new interactive technology may open more doors for users with special challenges. For example Hummels and her colleagues (2006) created an interaction device to help language learning for mentally challenged children, based on tagging real-world objects with tags evoking speech from a toy. Many ethical questions emerge here, such as to what degree will users become dependent upon technology, to what degree should users adapt to products or products adapt to people, and how will privacy be ensured? Increasingly, methods for collaborative design will be needed to help understand experience issues, and develop and validate new product designs over time and in context. From a methodological perspective, techniques will be needed towards understanding and formally modeling the user context of interaction. Towards this end researchers have begun to consider techniques such as diary tracking or ‘obstacle courses’ to help train context modeling systems (Intille et al., 2004). In concluding, the basic shift from technology for the sake of technology to technology to improve our lives places a high degree of responsibility on product designers. The need for an experience based approach to design is essential towards understanding the short and long-term implications of products which will co-exist in the user’s leisure and work environments, in support of daily activities. Consider here, the development of intelligent robotics which will be able to serve users. Laws on ethics pertaining to how robots are used will be required to avoid an over-dependence on them. At some point robots may even be considered as conscious entities, as they develop to the point they can hold what appears to be robust and well-controlled conversations in something like a natural language (Dennett, 1994). This may further challenge the idea of the user staying in control.

ACKNOWLEDGMENTS The author wishes to thank members of the ID Studio Lab working in the Social and Contextual Interaction Group, at the Technical University of Delft, who provided critical input for many of the studies presented in this chapter, and would also like to thank Mitsubishi Electronics Research for permission to publish the view on DiamondHelp.

REFERENCES Aarts, E. and Marzano, S. (2003). The new everyday: views on ambient intelligence. Rotterdam: 010 Publishers. Bradshaw, J. M. (1997). An introduction to software agents. In: Bradshaw, (Ed.) Software Agents. Cambridge, MA: MIT Press. Bruns, M and Keyson, D. V. (2005). MusicCube: making digital music tangible. In: G. C. van Veer (Ed.) CHI 2005: Proceedings and extended abstracts of the conference on human factors in computing systems, pp. 1176–1179. New York: ACM Press. Csikszentmihalyi, M. (1975). Beyond Boredom and Anxiety. San Francisco, CA: Jossey-Bass, 36. DeKoven, E. (2004). Help me help you: Designing support for person-product collaboration. Ph.D. Thesis, Delft University of Technology, Delft, The Netherlands. Dennet, D. C. (1994). The practical requirements for making a conscious robot. Philosophical Transactions of the Royal Society A, 349, 133–146. den Ouden, E. (2006). Development of a design analysis model for consumer complaints revealing a new class of quality failures. PhD Dissertation, Technical University of Eindhoven.

530

Product Experience

de Ruyter, B., Jain, Y., Keyson, D. and Rich, C. (2005). The usability crisis in high-tech home products: an opportunity for intelligent user interfaces? In: R. Amant, J. Riedl, and A. Jameson (Eds.) Proceedings of the 10th International Conference on Intelligent User Interfaces, 4 pp. San Diego, California, USA. Djajadiningrat, J.P., Wensveen, S. A. G., Frens, J. W. and Overbeeke, C. J. (2004). Tangible products: Redressing the balance between appearance and action. Journal for Personal and Ubiquitous Computing, 8, 294–309. Freudenthal, A. (2002). Designing transgenerational usability in an intelligent thermostat by following an empirical model of domestic appliance usage. In: C. Johnson (Ed.) 21st European Annual Conference on Human Decision Making and Control, Glasgow July 15–16. GIST Technical Report G2002-1, pp. 134–142. Department of Computing Science, University of Glasgow, Scotland. Hallnäs L. and Redström J. (2002). From use to presence: On the expressions and aesthetics of everyday computational things. Transactions on Computer–Human Interaction, 9(2), 106–124. Hassenzahl, M., Platz, A., Burmester, M. and Lehner, K. (2001). Hedonic and ergonomic quality aspects determine a software’s appeal. Proceedings of the SIGCHI Conference on Human factors in Computing Systems, pp. 201–208. The Hague, the Netherlands. Hassenzahl, M., Platz, A., Burmester, M. and Lehner, K. (2000). Hedonic and ergonomic quality aspects determine a software’s appeal. CHI, 201–208. Hekkert P., Keyson D. V., Overbeeke K. and Stappers P. J. (2000). The Delft ID Studio Lab. In: H. Acten, B. de Vries and J. Hennessey (Eds.) Design research in the Netherlands, pp. 133–142. Eindhoven: Eindhoven University Press. Hummels, C. C. M., van der Helm, A. J. C., Hengeveld, B. J., et al. (2006). Explorascope: An interactive, adaptive educational toy to stimulate the language and communicative skills of multiple-handicapped children. In: A. L. Brooks (Ed.) Proceedings Art Abilitation, pp. 16–24. Esbjerg, Denmark: Aalborg University. Ishii H. and Ullmer, B. (1997). Tangible bits: Towards seamless interfaces between people, bits and atoms. Proceedings of the ACM CHI’97 Conference on Human Factors in Computing Systems, pp. 234–241. Intille, S.S., Bao, L., Munguia Tapia, E. and Rondoni, J. (2004). Acquiring in situ training data for contextaware ubiquitous computing applications. Proceedings of CHI 2004 Connect: Conference on Human Factors in Computing Systems, pp. 1–9. New York, NY: ACM Press. Jian, J., Bisantz, A. and Drury, C. (2000). Foundations for an empirically determined scale of trust in automated systems. International Journal of Cognitive Ergonomics, 4, 53–71. Keyson, D. V., Freudenthal, A., Dekoven, E. and de Hoogh, M. P. A. J. (2001) European patent no. 1014792. Intelligent Thermostat. TU Delft. Ling, R. (2004). The mobile connection: The cell phone’s impact on society. San Francisco: Morgan Kaufmann. MacObserver. (2004). www.macobserver.com/article/2004/07/06.4.8html McCalley, L. T., Kaiser, F. G., Midden, C. J .H., Keser, M. and Teunissen, M. (2006). Persuasive appliances: Goal priming and behavioral response to product-integrated energy feedback. In: Y. de Kort, C. Midden, W. Ijsselsteijn, E. van den Hoven (Eds.) Persuasive 2006 Proceedings. Berlin, Germany: Springer. Minsky, M. (1988) The society of mind. New York, NY: Simon and Schuster. Mynatt, E. (2001, 31 March–5 April). Digital family portraits: Supporting peace of mind for extended family members. In: Proceedings of CHI ’01 Human Factors in Computing Systems, pp. 333–340. Seattle, WA: ACM Press. Norman, D. A. (2007). Design of future things. New York: Basic Books. Rasmussen, J. (1986) Information processing and human-machine interaction. New York: North Holland. Rich, C., Sidner, C. and Lesh, N. (2001) COLLAGEN: Applying collaborative discourse theory to human-computer interaction. AI Magazine, 22(4), 15–25. Rich, C. and Sidner, C. (2007) DiamondHelp: A generic collaborative task guidance system. AI Magazine, 28(2). Rijsdijk, S. A. (2006). Smart products: Consumer evaluations of a new product class. PhD Dissertation, Delft University of Technology. Rogers, E. M. (1995). Diffusion of innovation, 4th edn. New York: The Free Press. Ross, P. and Keyson, D. V. (2007) How to sculpt home atmospheres: design principles for expressive tangible interaction in control of ambient systems. Journal of Personal and Ubiquitous Computing, 11(2), 69–79. Rozendaal, M. C. and Keyson, D. V. (2005). Designing user interfaces with gestures and sound: Towards the performance and appeal of voice mail browsing. Journal of Design Research, 5(1), 1–14. Rozendaal, M. C. (2007). Designing engaging interactions with digital products. PhD Dissertation, Delft University of Technology. Tanaka, A. (1993). Musical technical issues in using interactive instrument technology with application to the BioMuse. Proceedings of the 1993 International Computer Music Conference – Tokyo, pp. 124–126. San Francisco, CA. Vastenburg, M., Keyson, D. and Ross, P. (2007). Reducing complexity of home atmosphere control via a user experience based approach. Universal Access in the Information Society 6-01. Weiser, M. (1991). The computer for the 21st century. Scientific American, 265(3), 94–104.

23

THE GAME EXPERIENCE ED S.TAN University of Amsterdam, Amsterdam, The Netherlands

JEROEN JANSZ University of Amsterdam, Amsterdam, The Netherlands

1. INTRODUCTION Playing digital games on a personal computer, a game console, a handheld device, or on the Internet is a relatively new, but massively popular form of mediated entertainment. In the public eye, games are generally constructed as the province of children, but one may wonder if this image is accurate (Williams, 2006). Research shows that the gaming audience has a wide age range, expanding well into adulthood. The game industry even claims that the average age of the gamer now is 33 (ESA, 2006). Men and women are unevenly represented among the gamers as most gamers are male. The gender difference becomes prominent during adolescence; at earlier ages the participation of boys and girls is rather balanced (ELSPA, 2006; Lucas and Sherry 2004; Nikken, 2003; Roberts, Foehr and Rideout, 2005). Gamers spend many hours on their favorite pastime, but it is rather difficult to determine the exact amount, as they are genuine multi-taskers who divide their attention across different media. They combine playing a game with, for example, listening to music, watching television, exchanging text messages on their cell phones, and chatting on the Internet (Roberts et al., 2005). A combination of sources from the industry and academic research warrants the estimate that a male adolescent generally plays about 11 hours per week, which is approximately one-and-a-half hours per day. Female gamers invest far less, and there is a substantial group of enthusiastic male gamers who spend about two-and-a-half hours per day (ESA 2006; Jansz and Tanis, (2007) Nikken 2003; Roberts et al., 2005). The popularity of games is underlined by figures from the industry: in the USA game producers reported a steady increase in sales over the past years, from 5.5 billion dollars Product Experience Copyright © 2008 Elsevier Ltd.

531

532

Product Experience

in 1998 to 7.0 billion in 2005 (ESA, 2006). The picture in Europe is comparable. The sales of entertainment software in the United Kingdom, for example, increased from 0.85 billion British Pounds in 2000 to 1.35 billion in 2005 (ELSPA, 2006); likewise, the Dutch market expanded from 83 million euros in 2000 to 186 million in 2005 (NVPI, 2006). The game industry has become a serious competitor of the movie industry. The game industry is growing faster than the movie industry and the market of digital games grosses more than cinema box office receipts. The movie industry, as a whole, still outperforms the gaming industry. This is attributed primarily to ancillary revenues, such as DVDs and sales to TV networks and cable companies. The gaming industry forecasts a further expansion of the market. The worldwide market for gaming is expected to grow from about 29 billion US dollars in 2005 to as much as 44 billion in 2011 (DFC Intelligence, 2006). The expected growth is partly attributed to the expansion of online Internet gaming. The increasing popularity of gaming went hand in hand with the public expression of concerns about playing digital games. Parents, teachers, politicians, and many others were particularly concerned about the violence in games and about the addictive properties of gaming. Empirical research partly confirmed the worries about possible negative effects of violent content, but failed to deliver substantial evidence for addiction to digital games (Anderson, 2004; Anderson and Bushman, 2001; Griffiths, 2005; Sherry, 2001). In many countries, the industry and government bodies addressed the public concerns by publishing rating systems that inform users in advance about (offensive) content and potential risks for young players. In this chapter, we address the experiences that result from actually playing a game. The massive popularity of digital games stands witness to the fact that playing games is attractive for many people. We propose that gaming is an emotional experience that is intrinsically rewarding. In other words, gamers are motivated by the unfolding of the game itself, and they enjoy the accompanying feelings. In our view, interest is crucial with respect to gaming. It dominates the gamer’s immediate experience during a game session as an emotion proper and it acts as a motivational disposition in-between separate gaming experiences. The first part of the chapter briefly characterizes the game as a product, and discusses the principal game features that may appeal to (potential) players. The second part is concerned with the actual game experience and the development of specific gaming expertise. We conclude the chapter by discussing the implications of being an expert gamer for the gamer’s identity. We also discuss the themes that must be addressed in future research about the game experience.

2. THE GAME 2.1. Digital games ‘Digital games’ is the umbrella term for interactive games that are played on different kinds of electronic media thus encompassing computer games, video games, games on mobile phones, and games that are played on the Internet. A digital game can be defined as an interactive rule-governed system based on computer processing power. ‘Interactivity’ means that the game requires a constant exchange of messages between the game and its player. When the player refrains from communicating, the game simply ceases to exist (Grodal, 2003; Kiousis, 2002; Sellers, 2006). The importance of rules for games was already emphasized in classical theories of play (Huizinga, 1950; Juul, 2005). In the case of digital games, the formal nature of the rules enables the implementation of games on a computer.

533

The game experience

TABLE 23.1 The major genres of digital entertainment games Genre

Description

Example

Action and adventure

The protagonist must complete a difficult quest. On this journey riddles must be solved, and attacks must be dealt with.

Tomb Raider Metal Gear Solid

Shooting and fighting

The player engages in combat. The beat ’em up subgenre is characterized by one-to-one combat against another player, or against (the AI of) the computer. First person shooters (FPS) derive their name from the first person perspective one has in the virtual world.

Tekken Mortal Kombat Doom Counterstrike

Platform game

One of the oldest genres, dating back to the earliest days of arcade-gaming. The characters are supposed to jump from platform to platform. Their movements are impaired by gravity and hostile interventions by other game characters.

Mario Bros Donkey Kong

Role playing game (RPG)

The role playing generally unfolds in a fantasy or science fiction context. The Internet is the stage for the large scale variant: the MMORPG (Massively-Multiplayer Online Role Playing Game).

Final Fantasy Legend of Zelda Everquest

Simulation

The player builds an artificial (social) world, and is supposed to solve problems that occur (unexpectedly). Sometimes called ‘god games’ because of the player’s apparent omnipotence.

SimCity Roller Coaster Tycoon The Sims

Sports and driving

Emulate the playing of physical sports, for example, (American) football, basketball, tennis, car racing, and car rally.

Madden NFL; FIFA2005; Gran Turismo; Need for Speed

Strategy

Achieving victory (under difficult circumstances) requires a lot of thoughtful planning and resource management.

Age of Empires Command and Conquer

Trivia and puzzle

Refers to the mini games that are generally played on the Internet. Examples include digital versions of traditional games, and brief versions of arcade games.

Bejeweled; Checkers; Mah Jong; Pac-Man

Digital games are available in an almost endless variety from trivial puzzle games on the Internet that can be completed in less than a minute, to highly realistic threedimensional (3D) console games involving a sequence of complex actions that take dozens of hours before the end is reached. Some games are aggressive of which some are popular and some are not. Furthermore, as illustrated in Table 23.1, many games are not aggressive at all. A differentiation in genres is a convenient way to bring some order into the diversity of games. We distinguish eight popular genres (see Table 23.1): Action and adventure; Shooting and fighting; Simulation; Sports and driving; Strategy; Role playing; Trivia and puzzle; and Platform games. Borders between genres are not very rigid. Many games are hybrids, combining characteristics of different genres. The successful title Grand Theft Auto San Andreas (GTA), for example, integrates many elements. The player assumes the role of a small-time crook with no means of subsistence, and, therefore, must accept all kinds of odd and dangerous jobs from mafia bosses in the city of San Andreas. Stealing cars is a fundamental characteristic of GTA; it allows one to cruise the city as if the player is partaking in a driving game. Also, completing the more complex criminal missions requires much planning and strategy on the part of the player. Finally, elements from shooting and fighting games are incorporated in the violent confrontations with the police and opposing gangs. The combination of elements from different genres has sometimes resulted in a new label

534

Product Experience

for a hybrid. Stealth games (e.g. Splinter Cell), as an example, are those that combine elements from action, adventure, shooting and fighting games. Different genres are characterized by different kinds of gameplay. Consider, for example, the difference between the fast-paced shooting games, and the simulation games that can be played at pace preferred by the player. The online shooting game Counter Strike, for example, is particularly popular among adolescent males. In this game, one collaborates with about six fellow players in a counter-terrorism unit. The goal is to neutralize one or more terrorists, resulting in a rapid sequence of attacks and counter attacks with much violence and mayhem. The players of CS are fascinated by its violent content and the tough nature of its competition, but they also enjoy the social interaction that is required to succeed (Jansz, 2005; Jansz and Tanis, in press). The Sims, in contrast, has a far wider demographic of male and female players of all ages. Gameplay in The Sims consists of managing a virtual dollhouse (of sorts?): The player arranges a life for one or more sim-people, designs clothing, a house, and furniture, and is intended to plan and control the lives and careers of these Sims. Its success as the best selling PC-game of all times is partly attributed to the unique combination of player’s roles. If they choose to do so, players can engage in the development of life narratives for their Sims, or concentrate on creatively designing clothing, furniture, and houses (Nutt and Railton, 2003). The examples of Counter Strike and The Sims underline that digital games are well equipped to satisfy widely different entertainment needs among the (potential) audience (Lucas and Sherry, 2004). They also illustrate the diversity in game content. Counter Strike is representative of the domination of male values and preferences in digital games. Many genres consist of stereotypical male activities such as: Fighting, football, and racing (Graner Ray, 2004; Williams, 2003). The Sims embodies a different set of values and activities, hence its appeal to a far wider demographic. The vast majority of digital games are played for entertainment purposes. Next to this, however, games are deployed for education and training (e.g. Prensky 2005). The digital games of reference in this chapter are entertainment games offering hours of gaming experience. Notwithstanding the differences between game genres, we will focus on features that are shared between games. In the same vein, we acknowledge that individual gaming experiences may differ from each other, but we aim to provide insight at a general level, and thus will concentrate on the commonalities in the experiences of gamers. Figure 23.1 presents an overview of the game experience as we will discuss it in this chapter. The two outer boxes represent the context of what happens in a single game session. The experience of a single game session is a function of repeated gaming (Multisession Context) and the player’s identity (Social Identity). Non-players’ or incidental players’ experiences during a particular session are different from those of typical gamers. The same goes for players whose social identity is only weakly tied to gaming. Identity and a history of repeated gaming influence the experience of a particular gaming session. The inner box represents events in the situation of a particular gaming session. Three descriptive levels correspond to mechanics, dynamics and aesthetics (Hunicke, LeBlanc and Zubek, 2004). Aspects of mechanics, dynamics and aesthetics will be called features. These are the vertical ellipses in the diagram. The experience as a whole (Single Session Game Experience), is represented by the field with mechanics as the lower limit and the top line of the inner box as the upper limit. Features are responsible for affordances (Gibson, 1979) of specific game experiences and they stem from the mechanics and dynamics of the game. Features are for now unspecified. They cluster to form affordance dimensions, which include: Sensory stimuli, rules, fantasy, mystery, control, and challenge.

The game experience

535

FIGURE 23.1 Overview of framework for describing game experience.

Features are shaped by multisession context, e.g. influence of practice, and social identity, e.g. motivation factors. The experience of gaming is dominated by emotions (horizontal ellipses). Most emotions are related to goals. For example, players experience interest in response to challenge and the prospect of achieving goals. Fear is experienced when an opponent threatens the player’s (game) life conflicting with the goal of staying alive. Goals are central elements in both affordances and emotions. They, too, are subject to influence from multisession context and social identity.

2.2. Game features and experience In an account of the game experience, the role of the game as a physical man-made product cannot be dispensed with. In order to properly grasp the (partial) dependency of gamer experiences and activity on the product, we have to distinguish levels of description in system design and behavior. LeBlanc and his team developed an approach to conceptualize the different components of digital games (Hunicke, LeBlanc and Zubek, 2004) that provides us with three descriptive levels. Their MDA framework (Mechanics, Dynamics and Aesthetics) looks at games from the perspective of the designer who aims to develop an appealing game employing digital technology. The first level of the MDA framework is the mechanics of the game. It describes the algorithms that determine the actions of game elements (events, characters, props, settings, and so on). At this level of analysis, the player’s activity is irrelevant. The mechanics are defined explicitly by the designer, and are thus predictable (Sellers, 2006). The dynamics is the second systemic level of the game. Here player input does matter. Interactions between specific mechanics are described that are partially pre-programmed, but are very conditional on player’s input (constituting choices and actions from their point of view). Consequently, the system’s ‘behavior’ (Hunicke et al., 2004) may result in emergent game play that was not anticipated by the designer. The third level of the MDA framework, aesthetics, describes the desirable responses of the

536

Product Experience

players. Game design aims to develop games that evoke positive emotional player reactions (for example, joy or interest), but obviously not all designs are successful in this respect. The aesthetic component is addressed in more detail in the second part of this chapter. A discussion of mechanics and dynamics is unnecessary given the aim of the chapter, and the authors’ lack of technical expertise required for an extensive treatment. The current section paves the way by describing the link between mechanic and dynamic levels on the one hand, and the aesthetic level on the other. The line of reasoning will be that a number of unspecified elements of game mechanics and dynamics that we will simply call ‘features’ enable specific experiences. The analytical link between features and experience is realized by the concept of affordance. This concept was originally developed by perception psychologist J. J. Gibson in his attempt to systematically link perception and action to the world in which the perceiver/actor functions. Affordance refers to the opportunities for action offered by a given environment to an organism (Gibson, 1979). An organism perceives the environment in terms of affordances. That is, physical or social features of the environment are automatically perceived as fit for, and inviting, action upon them. As an example, humans approaching a staircase perceive it as ‘climbable’ without explicit instruction or even thought. The concept of affordance highlights the relational structure in which perceiver and environment interact. Affordances also occur in man-made, artificial environments including the complex virtual environment of digital games (Linderoth, Lindström and Alexandersson, 2004). Conceptualizing (particular) game features as affordances allows for identifying their appeal to gamers. Obviously, the perceiver/actor side of any affordance involves capacities and skills. The features of the game are made into affordances when players have specific abilities that allow them to perceive the features and act upon these (Yates and Littleton, 1999). Most titles require players to have attained certain levels of cognitive development (e.g. literacy). Specific titles require specific abilities; for example, one must have background knowledge of soccer rules when playing FIFA2005. Features and coupled affordances of a game can be described at different levels of specificity. One option is, for example, to focus on the features of a specific title. Quite oppositely, another option is to focus on a set of abstract features shared by all digital games. We propose to describe the possible affordances at an intermediate level between the unique features of one game and the abstract features of games in general. We will follow Garris, Ahlers and Driskell (2002) who critically surveyed a number of earlier categorizations and proposed a framework of six broad game dimensions to capture the relevant game features at an intermediate abstract level: Fantasy; rules; sensory stimuli; challenge; mystery; and control.1 Fantasy is a feature of digital games that allows its players to engage in activities that are separate from daily life. Games create a special, distinct ‘reality’ in time and space where players’ actions have no consequences for the real world. In other words, players operate within the boundaries of the magic circle (Salen and Zimmerman, 2005). As a result, the game affords a safe, private laboratory where players can engage in activities that are neither possible nor tolerated in daily life (Jansz, 2005). Rules are a fundamental feature of digital games because they describe the procedures players must follow and are allowed to follow to reach the goal of the game. As an affordance, ‘rules are the most consistent source of player enjoyment in games’ 1 Other authors solved this issue in a different way. Sellers (2006), for example, proposed the abstract threefold characterization of interactivity, linearity and emergence and Klug and Schell (2006) took the type of player that is envisioned by the designer as a way to characterize digital games (for example, The Competitor, The Explorer, The Craftsman).

The game experience

537

(Juul, 2005, p. 55). When the game is played, gamers voluntarily submit to the rules. The dimensions Fantasy and Rules are captured under Juul’s observation that digital games are half-real (Juul, 2005): The rules are real because they define how to play the game, and winning or losing is a real event. At the same time, playing a digital game means participating in a fantasy world that is not real. Sensory stimuli: Digital games present the players with a rich and unique multimedia array of sensory stimuli. The processing power of contemporary game consoles and computers enables the drawing of millions of polygons per second, thus creating highresolution graphics (Poole, 2000). These technical improvements resulted in digital games with 3D visual features that approach the graphic quality of film or television. The auditory features have gained realism too in recent years. The game industry has always been quite successful in modeling mechanical sounds of, for example, guns and cars, and recent developments enabled the incorporation of realistic human voices (Smith, 2006). Digital games are generally accompanied by music. Many recent games offer a large musical score comparable to the soundtracks of films. In the Grand Theft Auto series, for example, each car is equipped with a radio. Players can choose their favorite station once they have stolen a car. Tactile features are provided by the controllers of console games. For example, the dual-shock controller of Sony’s Playstation 2 vibrates at appropriate moments in the game, thus affording a particular sensation. Nintendo’s new console, the Wii will be equipped with a wireless controller, which can be used as a handheld pointing device and can detect motion and rotation in three dimensions. Challenge: The leveled structure of digital games affords players a progressive degree of difficulty: Low levels are relatively easy to complete while higher levels demand much more effort. The game provides feedback about success and failure of the performances, and translates progress into a score. Challenge is among the affordances of a digital game, because it appeals to a fundamental urge for competence (effectance motivation (White, 1959)) and the experience of efficacy, that is, the satisfaction of having imposed an effect on the environment (Klimmt and Hartmann, 2006). Mystery: The mysterious features of a game encompasses more than the world of magic that is part and parcel of, for example, adventure games and role playing games. Here, we employ ‘mystery’ in a wider sense to refer to the inability to predict the future in a game, the violation of expectations, and the incompleteness or inconsistency of information (Garris et al., 2002). Mystery functions as an affordance because it appeals to the player’s curiosity or interest. Control is the final feature of digital games. It affords to regulate, direct, or command the activities within the game. Control is the unique selling point of many simulation games, which allow players the unique position to be in charge of all developments in, for example, the city (SimCity), or theme park (RollerCoasterTycoon). In other games, control can be maximized by playing in ‘god mode’. It is generally activated by a cheat code that makes the character invulnerable to damage. It is a common misunderstanding, however, that control in digital games is absolute (Klug and Schell, 2006). Part of the challenge of all games is, of course, that there are obstacles (e.g. hurricanes in SimCity) and opponents, both human and A(rtificial) I(ntelligence). In actuality, this particular affordance is an illusion of control. Control constitutes an important contrast with traditional ‘lean back’ kinds of mediated entertainment such as film and television. Once viewers have committed themselves to a particular film or program, it is impossible for them to influence the course of what is offered on screen. In summary, the dimensions fantasy, rules, sensory stimuli, challenge, mystery, and control describe game features that characterize digital games across specific titles and genres. Obviously, some dimensions are more salient within one particular genre than

538

Product Experience

within others, as illustrated in the discussion of simulation games and control. To what extent a feature actually functions as an affordance for individual players is also far more differentiated than this general discussion suggests. From a theoretical-methodological point of view, we may observe that there is a need for a proper systematic description of features and affordances. A kind of ‘grammar’ of features and affordances across genres is clearly missing and creating it seems a formidable task. Such a system should not only relate features to mechanics and dynamics, but should also explicitly convey similarities and contrasts and functional relations among dimensions. For instance, Challenge and Mystery seem to be the opposite of Control, while Rules seem to put constraints on Fantasy. These are crude intuitions emanating from Garris, Ahlers and Driskell’s (2005) proposal. However, as a contribution to a system of affordances, we propose that there is one structural feature of games that crosscuts most affordance dimensions, namely goals. Game playing is set apart from other activities in daily life in that the game world and its rules have extraordinary clear delimitations. This does not imply that players have at any moment a complete knowledge of all possible game states; discovering rules and possible states is part of all interactive games, which adds to the mystery. However, it does mean that they always understand that there is an ultimate aim to the game, and what outcomes constitute the aim. You have to find a ‘treasure’ somewhere and somehow, and it is necessary to stay alive under attacks of enemies. As a result (and in comparison to other activities) gaming situations are regularly evaluated in terms of desired final outcomes.

3. THE PLAYER 3.1. Gamers Game features are varied, as we have seen, to appeal to a mixed audience of potential players. The popularity of The Sims, as well as the success of the Trivia and Puzzle games on the Internet, shows that digital games can be attractive to players from different genders and ages. However, such successes are the exceptions to the rule: Most games are played by men. The Action and adventure, Shooting and fighting, and Sports and driving genres in particular seem to be the exclusive province of young men (Pratchett, 2005). They are the typical gamers, being adolescent or early adulthood males that outperform all other gamers with respect to the time and effort they devote to gaming. Typical gamers tend to be depicted as ‘nerds’ who prefer to play in social isolation in their proverbial attic, but research has shown the crudeness of this stereotype. Generally, gamers enjoy interacting with other gamers. The success of LAN parties, where gamers link their own computers to a top-speed Local Area Network (LAN) to play against each other, illustrates the enthusiasm of typical gamers for gaming as a social event (Durkin, 2006; Jansz and Martens, 2005). Nevertheless, others seem to prefer to concentrate on the virtual social relations in and around the digital game, particularly when playing an online role-playing game (Griffiths, Davies and Chappell, 2004). In general, playing with others in the virtual or real world provides social affordances (Yates and Littleton, 1999) that interact with the affordances embedded in the game features. Emotional preferences in gaming are likely to be highly gendered. Emotions that are part of the experience of the virtual world may differ between genders along the lines identified in research about gender and emotion (Shields, 2000) related to the different ways in which men and women experience (and express) emotions (Fischer, 1993; Jansz, 2000). More specifically, it may be that the affordances of challenge and control are more appealing to men, while fantasy is more appealing to women. This is a speculation

The game experience

539

that deserves further theoretical elaboration and empirical research, but there is some backing evidence. First, the desire for mastery over a game seems to be paramount for males, while females tend to be willing to play without regard to their score (Morlock et al., 1985; Raney, Smith and Baker, 2006). In addition, women may prefer more freedom in identifying and setting goals, and in the final convergence of goal-paths. This could justify their preference for playing The Sims over such games as Mortal Kombat (Lucas and Sherry, 2004; Nutt and Railton, 2003). Secondly, specific game features, such as advanced and convincing graphics, may be expected to be enjoyed more by men than by women (Ivory, 2006). Thirdly, gender differences in self-efficacy as to gaming have been observed. On average females have low self-efficacy beliefs with respect to games, and, according to Klimmt and Hartmann (2006), this may weaken their motivation to play. The account we propose here is only concerned with the experience of typical gamers who are regular players of mainstream or popular digital entertainment games.

3.2. A framework for describing the game experience Available frameworks for describing gaming include: (1) the uses and gratifications approach (e.g. Sherry et al., 2006); (2) effects research (e.g. Lee and Peng, 2006); and (3) the design approach (e.g. Salen and Zimmermann, 2003). These approaches do not or only obliquely address the experience of the gamer. We propose a framework for describing the game experience based on two ideas. The first is that the essence of the experience is emotional. We feel that the experience of typical gamers is intense and demanding enough to justify calling it emotional. The second is that a full understanding of the (emotional) experience of gaming requires taking into account a wider temporal context in which a particular gaming session is set. Grasping the emotional character of the experience requires an extremely brief characterization of the concept of emotion. According to theorists such as Frijda (1986), Lazarus (1991) and Scherer (2005), emotions are states of extraordinarily strong feeling and total dedication of resources (from attention to approach or avoidance and other kinds of action dispositions) to dealing with the situation at hand. This qualification seems to apply perfectly to gamers’ experiential states. The degree of emotional involvement in the activity and the amount of sustained effort in the face of seemingly insignificant reward has astonished outsiders, and it may have contributed to the stereotype of the ‘nerd’ gamer. In the chosen framework, emotional experience and feeling organizes cognition, affects and perception in conscious experience and completely dominates awareness of the situation. Feeling also includes self-awareness, and, most importantly, it includes an awareness of a readiness for actions appropriate in the gaming situation. There is a variety of theoretical approaches to emotion. One major strand, the biological account, is inappropriate because it falls short of explaining feeling. Its advantage in establishing areas in the brain involved in the emotions is of little use when we want to give an account of how gamers’ subjective awareness and experience influence their feelings.2 The cognitive appraisal strand of emotion theory and research (Scherer, Schorr and Johnstone, 2001) is particularly suited for our purpose. How gaming feels requires spelling 2 To illustrate the point, Weber, Ritterfeld and Mathiak (2006) recently demonstrated the occurrence of specific emotional brain states associated with playing aggressive video games, using fMRIs of players. A parallel was found with fMR imagery obtained from subjects involved in real aggression and aggressive thought. The authors aptly warned that one couldn’t conclude that the experiences are the same in either case, because gamers are aware of the pretence-play characteristics of the activity.

540

Product Experience

out the essentials of the meaning structures that underlie emotions in gaming. This is exactly what appraisal theory and research methodology purports to do. Research taking a similar perspective to the experience of gaming is scarce. Garris, Ahlers and Driskell (2002) report less than ten studies that have as one of their aims to assess (core) emotions typical for the game experience. Important pleas for an emotionfirst approach may also be found in contributions by Grodal (2000); Vorderer, Hartmann and Klimmt (2003); and Ravaja et al. (2006). The second idea underlying our framework for describing the gaming experience is that the consideration of any gaming experience as a self-sufficient entertainment-episode has severe limitations. We assume that the typical experience of gaming is reserved for persons who have developed considerable knowledge of one or more games. The complexity of multiple controls requires dire practice before eye–hand coordination and fine motor skills become tacit knowledge (Jansz, 2005). Given the requirement of practice, the experience should be studied not only as a snap-shot occurrence, but also as one that makes sense to the player as part of a connected series. Equally important, repeated gaming is associated with increasing intrinsic motivation for engaging in the activity, which in turn alters any ongoing experience. In presenting the framework, we will describe the role of goals as a core feature associated with major affordances in gaming. Then, we present two important emotional states identified in gaming, flow and presence, followed by the other major emotions in gaming. We conclude by discussing the influence of gaming on the self and effects of expertise and motivation. Experience and goals Although most people fancy spending much of their leisure time in freedom, idly relaxing and not pressed by any need to deliver a product or a service to anyone, entertainment is often characterized by activity and by activity with clear goals. The (entertainment) experience, afforded by playing a game, involves knowing what you are expected to deliver (goals). This knowledge is necessary for enjoyment. The ongoing appraisal process, at the basis of emotional experiences, centres on goals to a high degree. Playing games entails having in mind both a final goal (for example, defeating the End Boss) and subordinate ones that arise out of the particular situation (for example, completing a level in order to get closer to the End Boss and acquire weaponry). Experiencing outcomes consists of feelings of pleasure or pain based on direct matching with a goal state. The pleasure is manifold; it stems from reaching a preferred state in the virtual game world, having control of the game (Grodal, 2000), and the gratification of more distal motives, notably improving one’s self-image (see section on ‘Gaming and the self’, p. 547). For instance, ‘killing’ a nasty adversary means both eliminating a threat and getting access to places and resources in the game world, but it also implies that you have mastered a challenge posed by the game. This may invoke the triumphant feeling of male prowess. Gaming is, as Apter would have it, a non-serious activity, carried out in a paratelic motivational state (Apter, 1991). In this state, people are intrinsically motivated. They perform the activity for its own sake, and they tend to prefer higher levels of difficulty and arousal than in situations where they are doing work as part of a job or taking part in serious conversation. The latter activities are associated with a telic state, in which higher levels of difficulty and arousal are experienced as unpleasant, and are generally avoided. In a paratelic state, people have no difficulty in focusing attention exclusively on the activity they are involved in. Gaming absorbs attention, and high arousal levels related to difficult goals are appreciated and sought. Surprising turns and fresh targets can be found pleasant. The sheer depth of goal hierarchies, or winding roads towards goals in many of the most

The game experience

541

popular games that may strike the novice as appalling, pose challenges to typical gamers that they experience as exciting. Descriptions of games from the player’s perspective may illustrate this point (for instance, see VanDeventer and White (2002) on Super Mario; Nutt and Railton (2003), on The Sims). Typically, players have invested a lot of trial and error, patient exploration and vehement duelling before an obstacle has been overcome; and most of the time only to find out that a new obstacle arises. Being faced with another, even more ambitious goal would create a sinking feeling in the everyday life, but less or not so in gaming. Goals seem to be opportunities for skillful activity. One of the players quoted by Nutt and Railton (2003) sees the opportunities of The Sims as an ultimate kind of creativity in which players can set their own goals. A lack of goals would mean the game’s absolute failure to typical players. Tan and Dev (2000, pp. 298–299) quoted a typical gamer participating in a study testing an application requiring free exploration in a hypermedia space who after a few minutes of free roaming exclaimed: ‘Come on, action! What are you supposed to look for in this game?!’, and quit the session before long. Depending on the particular game, it may be more or less difficult to identify, choose, or set goals that mediate between the current game state and the desired final outcomes. In most adventure games, it is not possible to look ahead beyond the level one is in at any present moment, and the immediate goal is dictated by the game’s design. In contrast, in Grand Theft Auto San Andreas, players have the opportunity to ignore some missions while completing others. Simulation games (for example, SimCity; The Sims) allow players maximum freedom in wandering about the game world, and there are hardly any restrictions on goals to be set by the players themselves. It may be expected that factors like final-outcome relevance, concreteness, and urgency of goals, as well as the degree to which they were imposed on rather than set by the player, determine the experience of gaming situations to an important degree.3 As already observed, gender is an important factor in the enjoyment of restrictions in goal setting and the appreciation of ensuing challenges. If awareness of goals forms the backbone of the experience, which is what we posit here, then we need to be more specific on the structure of experience episodes in which they play a part. The basis for the role of goals is that they motivate the player to attempt to realize them. In the simplest interactive play, the goal is to bring about a simple change in the environment. For instance, infants between 6 and 11 months prefer to play with toys such as a chime hanging by a string in front of them. They tend to repeat reaching for and touching for quite a sustained period. We may assume that some motivational tension or desire arises when the toy’s effects have come to a halt. While the toy remains within action range, it acts as an impetus to repeat the behavior. Each motivational cycle is closed with a pleasurable outcome: The jingling of the bell (Friedlander and Kessler [1965], in Berlyne 1969, p. 826). The motivational state itself is experienced as an urge to act or as a desire. Motivational cycles can be regarded as the building blocks of the gaming experience, particularly with respect to effectance motivation, i.e. the motivation to experience success in mastering tasks (Klimmt and Hartmann, 2006). (See also Garris, Ahlers and Driskell [2002] for a comparable account of game cycles.) These motivational cycles coincide with gaming episodes initiated by an event that calls for an activity with a positive outcome, experienced as success, or a negative outcome, experienced as failure. The outcome is delivered as feedback by the game system. Because a gamer is always involved in one game cycle or another, the basic affordances of challenge and control are permanently salient. 3 For a fundamental treatment of these and other characteristics of goals and the self-regulation of behavior, see Carver and Scheier (1998).

542

Product Experience

FIGURE 23.2 Goal hierarchy in a game. Players proceed from bottom to top, that is from lower and intermediate to top-level goals. Attempts to reach a goal plus the final outcome (success or failure in reaching the goal) constitutes a game cycle. Two alternative routes, R 1 and R3 lead to the final goal. Route 2 constitutes a dead end. Motivational tension increases with proximity to the final goal and past success – failure rate. Failures can be more serious if they are unexpected (e.g. a success coupled to a dead end). The closer players get to the final goal and the more failures in previous attempts, the higher tension they will experience. Enjoyment upon reaching a goal is a reverse function of motivational tension.

More complex games have goals arranged in hierarchies. Figure 23.2 shows a goal hierarchy as we will often find in interactive video games such as adventures and shooters. Motivational tension in attempting to reach goals is a function of the value of the particular goal. The final goal is the most valuable. Proximity to the end goal is the second factor, and invested effort is the third. The final assignment in Figure 23.2 brings the highest tension because of exclusive access, proximity to the final goal, and the amount of invested effort prior to the current state of the game. There is usually another factor that intensifies tension, namely, the risk of fatal errors. In many game genres, there is the permanent danger of losing one’s life due to traps, attacks of hidden enemies, or fatal mistakes such as stepping onto a mine. Avoiding the end of the game through a defeat may be called a negative goal (cf. Carver and Scheier, 1998). A related negative goal is avoiding resource depletion (e.g. time, currency, tokens). As convincingly proposed by LeBlanc (2003), the well-designed game has running markers of resource depletion that indicate the progressive inevitability of an outcome and associated urgency of action. Games become more complex and more challenging as goal hierarchies increase in depth and breadth, and as goal states involve more encompassing changes of the game situation. In addition, more than one goal may be active at a time, and thus the number of alternative outcomes constituting success and failure is expanded. Complexity also entails that outcomes are compared to counterfactuals (e.g. when a success or failure anticipated with certainty does not occur, and an exactly opposite outcome was expected), intensifying feeling. Complex goals are arranged in a sequence or a hierarchy, and, consequently, their experience has a temporal and a causal aspect. The player can experience not only success or failure, but ‘progress’ or ‘stagnation’ as well, giving rise to feelings of euphoria or frustration. In addition, reaching any immediate goal may have different values for

The game experience

543

advancing to the final goal. The joy in finalizing a level is greater than any given immediate goal such as escaping from a pit, obtaining a resource, or dealing with an enemy. The most complex games present players with considerable uncertainty about the entire route to the final goal. Initially, no estimation whatsoever can be made of how much effort is required to get anywhere, let alone to reach the final goal. It may also be uncertain whether a strategy that seems to be successful at first will not turn out to be wrong in the long run. Multiplayer games add the complexity of interacting goals such as when players A and B strive for the same goal, but only one can reach it or, both can reach it through collaboration. Finally, one major structural characteristic of games seems to be related closely to complexity at all levels. Narrative games are more complex than non-narrative ones. Games with more than minimally elaborated story-worlds allow for an overall higher degree of dynamics than games without such a world. The emotional experiences in playing non-narrative games may be less intense (Schneider et al., 2004). The major emotions in gaming are associated with goals. Before we discuss these emotions, we have to review the two most current conceptualizations of the game experience. Goals and flow Probably the most widespread approach to understanding the game experience is emphasized in a state of mind known as flow (Sherry, 2004). Csikszentmihalyi (Csikszentmihalyi and Csikszentmihalyi, 1988) coined this term in his study of the unusual total engagement exhibited by persons carrying out creative work such as composing music, or in active leisure activities such as reading stories or playing chess. Persons in flow forget about themselves and become totally immersed in the activity. A paratelic attitude is necessary, that is, interest in the activity should not be at the cost of one’s (working) agenda. Moreover, there should be a delicate balance between one’s capacities and the requirements of the activity. In other words, the challenge must be suited to the player. This balance is experienced as a complete harmony or fusion between person and activity in which one loses his or her ego. Attention, ideation, and action seem to spring from oneself in an inevitable way, free of error, doubt, and rumination. Flow is rare in daily life, according to Csikszentmihalyi. Some ‘autotelic’ persons, however, have a predisposition towards it. These individuals can easily put aside constraints imposed on their curiosity by regular circumstances. Reaching states of flow in gaming is facilitated primarily by fantasy, keeping away daily worries, and, secondarily, by clarity of goals and by rules that act to preserve focus in the activity. As we have seen, the particular way that goals are organized into series and hierarchies would seem to be most relevant for inducing flow. Digital games are generally designed so as to gradually increase difficulty, exactly in pace with the player’s developing knowledge and control of the game. As illustrated in questionnaire research by Vorderer, Hartmann and Klimmt (2003), games that pose the right amount of challenge for the player are liked best. In their research, they presented regular players of Tomb Raider with descriptions of situations from this game varying as to (1) urgency to act; and (2) number of means to act. It was found that a combination of highest urgency and largest choice of means was found to be the most attractive scenario. Goals and presence Another dedicated term that has been used in describing the added value of gaming to the contemporary array of entertainment experiences is presence. This term refers to a state in which the virtuality of interaction with a technical system goes unnoticed (Lee, 2004). Fantasy and sensory stimulation appear to be the affordances involved in presence. Virtual experiences can be mediated experiences of real objects, events, and persons,

544

Product Experience

or experiences of artificial objects, etc. Presence is related to a larger number of conceptualizations of the psychological state characteristic of users of ‘older’ media like books, television, and film. These conceptualizations are captured by notions such as transportation, which is the experience of being carried into another world by a narrative (Gerrig, 1993; Green, Brock and Kaufman, 2000), and the illusion of being in a fictional world as an invisible witness (Tan, 1996). There is a relevant association between this state of mind and the use of the imagination (Valkenburg and Peter, 2006; Singer and Singer, 2005). However, if in ‘lean back’ media entertainment, the user occupies a position of a ‘controlled spectator’ (Tan, 1996) or passive ‘eavesdropper’ (Gerrig, 1993), video-gaming has added an interactive form of presence to the user’s action repertoire, enriching the array of mediated illusions with a strong sense of activity and control. Having the awareness of being in a virtual space, all in itself, would seem to lose its initial attractiveness to typical gamers quite rapidly.4 As said, having goals adds a sense of direction, challenge and motivation to the experience. The nature of goals in games seems to be critical for the feeling of presence in a way that is not yet fully understood. On the one hand, it has been proposed that there must be some match between virtual goals of the protagonist in the game-world and the player’s goals in the real-world. As Juul (2005) noted, a game cannot have the goal of throwing the protagonist under a train. On the other hand, there is the observation by Ravaja et al. (2006) of positive player emotions as a response to the negative fate of a protagonist. In this observation of a game, players had to control a monkey and experienced positive affect when it reached a goal (e.g. finding a banana). However, a fall of the monkey into the depths also elicited positive affect. This implies a split between virtual goals of the game’s protagonist and those of the gamer. Presence in gaming seems to involve a sort of transport of the self of the player into the game world, but the exact match between player and character goals is somewhat mysterious. We will return to this aspect of presence in the section on ‘Gaming and the self’, p. 547. Adding virtual space qualities to games does not automatically result in a more gripping sensory experience, as illustrated in research by Tamborini and his colleagues (2004). They compared two different versions of Duke Nukem 3D. Participants either played the game on a standard PC, or on a Virtual Reality system. Contrary to expectations, the standard game conditions produced the highest level of (tele)presence (Tamborini et al., 2004). Interest and other emotions in gaming We propose that interest is the emotion that dominates the gamer’s experience from one moment to another, that is, as an ongoing process. Interest is a basic emotion (at least according to some emotion theorists (including Frijda, 1986; Izard, 1977, 1992; Panksepp, 1982; and Tomkins, 1984)). According to Izard (1977), interest is the modal state of consciousness and the most basic of basic emotions, to be more precise, it is ‘the most prevalent motivational condition for the day-to-day functioning of human beings’ (Izard, 1977, p. 211). Interest, an emotion discussed at length by Darwin (1965) as the set of ‘intellectual emotions’, has been somewhat neglected in emotion literature (Hidi and Baird, 1986), but it presently enjoys a reawakening (Ainley, Hidi and Berndorff, 2002; Ellsworth, 2003; Oatley and Johnson-Laird, 1987; Silvia, 2005; Tan, 2000). Meanwhile, 4 Liking for challenge and control affordances may be stronger for males than for females, who may relatively favor fantasy affordances, but direct evidence of a probable gender difference are not available. In Tan and Dev (2000) a small-scale study is reported showing that women found linear author-controlled navigation through a pictorial narrative more exciting than men, who showed more excitement when they had control of the navigation.

The game experience

545

Darwin’s terminology has been replaced by the somewhat more contemporary label ‘epistemic’ or ‘epistemological’ emotions (Keltner and Shiota, 2003), including curiosity and interest. In the following paragraphs, we concentrate on the appraisal of interest and its relation with experience, antecedents, and consequents. Izard described the experience of interest as follows: At the experiential level interest … is the feeling of being engaged, caught-up, fascinated, and curious. There is a feeling of wanting to investigate, become involved, or extend or expand the self by incorporating new information and having new experiences with the person or object that has stimulated the interest. In intense interest or excitement, the person feels animated and enlivened. It is this enlivenment that guarantees the association between interest and cognitive or motor activity. Even when relatively immobile the interested or excited person has the feeling that he is ‘alive and active’ (1977, p. 216).

Izard’s definition seems to fit the activity of advanced interactive video-gaming almost perfectly. It stresses the feeling of pleasurable energy and the motivation for sustained activity that is characteristic of the typical gamer’s experience while playing. Interest captures important features of flow in gaming, namely: (1) the skilled motor performance that seems to take no conscious effort as if it were automated; and (2) that it involves the expansion of self into a new ‘reality’. Moreover, Izard’s definition has a theoretical advantage over the flow (and presence) approach, in that it positions the experience explicitly within the widely studied context of the emotions, thus rendering it accessible to empirical study.5 Interest merges well with other emotions in gaming that are manifold. Jansz (2005) argued that digital games elicit all sorts of emotions. Van Reekum et al. (2004) reported altered levels of pride, joy, surprise, and anger in players of games. Anger may be the bestknown example, because games involving virtual aggression and violence conspicuously dominate the market and public discourse. Disgust, pride, and joy also occur frequently. Izard and Ackerman (2000) recently emphasized the potential of interest to subsume and interact with other emotions. Tan (1996) described how virtually all emotions can be experienced in film viewing, and many emotions can occur in parallel; all are supported by interest as a ‘backbone’ affect. The emotion integrating function of interest can be generalized to other entertainment forms than film. Our proposal is that interest is the emotion that is omnipresent in gaming, coloring all other emotions. The gamer’s anger is different from the same person’s anger in daily life or even in a virtual world. This is, in part, because of the game’s encompassing experience with a definite orientation towards a goal that is distinct from the aim of the anger. The game is also controlled by interest, an urge to continue exploring and playing.6 The emotional experiences afforded by the game features may consist of a larger number of ‘emotion palettes’, each depending on the particular game genre; all palettes have interest as a primary emotion. Furthermore, interest may tie together various components of the experience that originate from extremely varied theoretical viewpoints into one encompassing framework. For instance, if interest is associated with an urge to explore, then a plausible theoretical possibility is that the experience of interest merges with that of presence 5 Flow and presence have been referred to as states of mind in this paper, because it is not clear from their definitions whether they are emotions. 6 There seem to be two sources of anger. One is frustration in reaching a goal. Here anger is moderated or partly transformed by interest. Another is being harmed by an antagonist in the fantasy world of the game. Here interest also modifies the anger, allowing it to be enjoyed for its own sake. The gamer ‘loves to hate’ the opponent, without losing the awareness of the make-believe nature of the antagonism.

546

Product Experience

giving way to a sensation of an alternative reality open or even inviting to be discovered. Interest is the emotion that we would like to replace the notion of ‘motivational tension’ mentioned above, removing negative overtones associated with threat. Goals are the affordance structures that elicit interest, and interest includes a motivation to investigate and accept challenges. Game cycles are, in fact, interest episodes. The second emotion that is typical for the gaming experience is enjoyment. Although operating in tandem with interest, the two emotions have to be distinguished (see Silvia, 2005; Tan, 1996). The correlation between time spent on a free choice activity, a measure of interest, and self-reported enjoyment has been found to be much less than unity; it is approximately φ.40 (Harackiewicz, 1979). Enjoyment is a pleasurable emotion involving an appreciation of the entire situation of gaming and its experience. Achieving goals is a major source of pleasure, but not the only one as we shall see. Interest contributes to enjoyment, and it may even be the major determinant of the pleasure one has in gaming. The appraisal of interest involves a sense of promise, expected reward (Tan, 1996), or a judgment that a situation has a certain difficulty which can be met (Silvia, 2005). In other words, interest is based on expectation of reward and mastery; it is an anticipatory emotion. Alternatively, enjoyment occurs upon actually receiving reward; it is a consummatory emotion. Within the other emotions in gaming, three sets may be distinguished that differ as to the focal object appraised. First, as already pointed out, players have emotions relating to progress versus blockage in arriving at valued goal states and obtaining preferred outcomes. Frustration and relief are obvious examples. We may call these emotions task-related. The emotions identified by Van Reekum constitute cases in point. A second class of emotions are responses to (presence in) a virtual world. Awe, a sense of beauty; empathic emotions, such as sympathy and admiration for; and pity upon game characters are examples. These emotions constitute responses to an imagined world. In a third class of emotion, players have feelings in response to the game as an artifact made by a designer/producer. (For a related distinction between emotions in response to the fictional world, on the one hand, and ones in response to the artifact of feature films, on the other, see Tan, 1996.) In summary, aspects of the game artifact that may be objects of player emotion only require a quick glance at game reviews in various media (see also Ivory, 2006). Some of the production values which games offer to regular players include graphic realism, realism of character movement, and control offered to players in altering appearances. A(rtificial) I(ntelligence), a common denominator for the effect of player actions on game states, constitute another major production value. AI’s potential for having the player meet with adequate and surprising opposition (offered by virtual antagonists) in realizing goals deserves special attention. Players often ascribe responsibility and credit to the producers for positive and negative experiences they have with a game. Finally, interest combines with social emotions. According to Izard and Ackerman (2000), interest prepares people to engage in social interaction. This may be an action tendency that matches with games’ affordances. The importance of social affordances was underlined by the results of several studies showing that gamers play digital games because they are motivated to engage in social interaction with other players (Jansz and Martens, 2005; Jansz and Tanis, 2007; Lucas and Sherry, 2004; Yates and Littleton, 1999). More specifically, Colwell et al. found that playing together intensified male friendships (Colwell, Grady and Rhaiti, 1995). Gamers may also like to develop (para-)social interactions with game characters and other virtual players. The interactions with virtual individuals are as much facilitated by interest as interactions with ‘real’ people. In both cases, the social affordances will bring along emotions associated with friendship and shared goal-directed activity.

The game experience

547

In concluding the discussion of interest and other emotions, we will delve into the issue of aggression as part of the gaming experience. Effects research has singled out arousal as an affective state resulting from playing games and, in particular, violent ones. Arousal is considered a condition for subsequent aggressive thoughts and behavior (Anderson and Bushman, 2001; Schneider et al., 2004). Arousal is a term that lacks precise definition, and its relations with interest are complex for that reason. Most researchers agree that it stands for heightened physiological activity of parts of the central nervous system involved in the regulation of vigilance and general activation. Arousal has a negative feeling tone when coping with difficulties in daily life and is associated with threat and anger. In contrast, when arousal is found in the context of entertainment, it can be labeled as either positive or negative. According to Berlyne (1969), an ‘arousal jag’ in situations of play does not always require resolution, but can be experienced as pleasurable in itself. In addition, it has been found that many gamers, both male and female alike, enjoyed playing because of the arousal it generated (Lucas and Sherry, 2004). And recently, Ravaja et al. (2006) found in psycho-physiological research that both positive and negative events in Super Monkey Ball 2 were associated with positively valenced arousal (as mentioned earlier). Negative labeling may occur when conflict related to goals is experienced. For instance, approach– avoidance conflicts arise when attractive goal states are envisioned in combination with heightened risk of sudden death, and the conflict will aggravate when the goal is nearer. We hypothesize that in a typical gamer’s experience, unpleasant arousal can be part of a high interest experience, but that this arousal is at least in part labeled as pleasant excitement when the player has success in realizing goals or is confident about succeeding in the end. An underlying theoretical rationale for this hypothesis is that anger and fear are moderated when they occur in situations that elicit interest. Interest is not merely a state of heightened attention, but it is associated with curiosity and exploration. Interest is related to playful states and behavior. It involves a mood and a drive to engage in problem solving, possibly in a creative, novel manner. The adaptive function of interest is to counterbalance the fear of going out to explore the world. If people did not have a motivation for exploration, they would not take the risks that are inevitably associated with attempts to discover unknown resources in the physical and social world (e.g. Kaplan, 1992). In a similar vein, interest may moderate responses to frustration and transform these into an appraisal of challenge. Nonetheless, empirical research has demonstrated effects of gaming on aggression, and it would seem that emotions such as anger do persist to some significant degree. It may be that fantasy mediates between experience of angry negative arousal and aggressive effects. The fantasy of power afforded by violent games may be more appealing than mastering challenge. This could result in seeking control over situations and people outside the game, assuming the shape of contempt – already a dominant emotional disposition in adolescent males (Harris, 2000) – aggressive thoughts, and behavior. Gaming and the self The challenge required for interest is higher to the degree that the self is involved. Many scholars, including Huizinga (1950) and Goffman (1971) expressed their surprise about the high degree of involvement that participants in traditional games exhibit. In many cases, it does not seem to matter that ‘it is only a game’. Excess ego-involvement may result in the breakdown of friendships and even divorce of married couples, as anecdotes would have it. Higher levels of it keep boredom away and boost interest, enjoyment, and positive social emotions, as it would appear. With this aside, success in gaming (i.e. realizing goals) not only brings along immediate enjoyment, but it also affects the self to some degree. Self-efficacy beliefs (i.e. those in one’s capability to control their fate and their

548

Product Experience

own functioning) are positively affected by success and negatively by failure. According to Bandura (1997), the beliefs involved act as the most central and pervasive mechanism of personal agency. If this is so, playing games and being good at it may enhance self-esteem, which, in turn, explains the appeal it has to some and the avoidance exhibited by others. Earlier research showed a negative correlation between self-esteem and playing digital games in adolescence (Colwell and Payne, 2000). Funk and Buchman (1996) differentiated between male and female players and reported lower self-esteem in girls who engaged in fighting games. This does not mean, of course, that playing digital games causes low selfesteem. It may very well be the other way around; adolescents with low self-esteem like to play quite often. In his discussion of the positive effect of gaming, Griffiths concluded that ‘beating personal high scores raised self-esteem in the individual’ (2005, p. 166), a point also made by other researchers (Fritz, 1997; Tennyson and Breuer, 2002). Success in gaming, then, may increase self-esteem and provoke pride, while failure may decrease self-esteem and elicit feelings of shame and inferiority. The experience of some games is also colored by competition. Self-involvement may be higher in a competitive game, as the player measures success and failure by the performance of others. Vorderer, Hartmann and Klimmt (2003) report significant correlations between the frequency of playing and the motivation to compete in fighting genres, including First Person Shooters and Action Adventures such as Mafia. They found, however, no significant correlation with simulation games (Comanche) and sports games (FIFA Soccer). Correlations of frequency with self-efficacy beliefs followed the same pattern, indicating the common involvement of the self. An interesting hypothesis would be that playing a game individually, without any adversaries, calls forth the feeling of competition because the game itself (or its apparently intentional design) is experienced as a proxy or surrogate of an opponent. Successes are equivalent to outsmarting the machine or the game makers, and losses are perceived as their victory at your cost. Such an experience is in line with the conception of games as communicative entertainment media. Finally, presence and flow are labels for an experience entailing loss of self in games. In most games, players fulfill rule-based roles or fantasy roles and identification with this role can be very strong (Garris, Ahlers and Driskell, 2002). More generally, the experience of being and acting in a world entirely different from everyday reality is among the most enjoyed features of games (Lucas and Sherry, 2004). In interactive video games, the experience is embodied; that is, it is as if the player’s body has an extension in the virtual world. (Newman, 2002). The sensory dimension to presence in gaming is unusually strong, as the player has visual and kinaesthetic experiences that can only be interpreted as being and acting in that world. The game world is less experienced as a simulation or representation of another world, than as a separate reality. Similarly, the self is not so much represented by a game character, but rather immediately operative in the game world. Many games require real and not imagined motor reflexes! The fantasy afforded by games, then, includes the embodied self. The strong sensory and motor experiences are another element contributing to pleasure. Good examples include ‘Whammos’ (i.e. explosions and collisions) and fast movements perpendicular to the screen, which feel as if they resonate or originate in the gamer’s body.7 Game software seems to be specialized in conjuring up such effects. Flow is the experience of loss of self due to immersion in the task at hand (Elliott and Harackiewicz, 1994). An unusually strong focus of attention and seemingly effortless harmony of actions with affordances mitigate loss of self in the activity of gaming.

7

The features mentioned are derived from a description of current action films by David Bordwell (2006, p. 104–114).

The game experience

549

The fusion of the self with controlled activity in flow, and with the virtual world in an embodied fantasy role, form a powerful combination that can make for an altered state of awareness reported by typical gamers. The experience may be qualified as a mixture of presence and flow. Accumulation of experiences: Expertise Any single gaming experience derives its qualities from the interplay between affordances and challenges that a given game presents with particular capacities, motives, and styles of the player. Repeated gaming alters player characteristics and, consequently, the experience. Playing games results in a basic expertise of the game. A video game requires the player to learn, among other things, the scenery, (sub)plots, possible tactical moves of the game, and the skills needed to operate the interface. People learn easily from gaming because they are drawn into the game spontaneously. Positive learning effects have been well documented in empirical research, both with respect to children (Subrahmanyam et al., 2001), and adult players (Green and Bavelier, 2003). The most complete overview to date of what people may learn from digital gaming is by Lieberman (2006). Documented learning includes an increase in interest as a trait, emotional responses, learning motivation, creative problem-solving, engagement, flow, and perceptual and motor skills. Some games act as an exercise in social skills. It should be noted that in Lieberman’s review, learning involves transfer to situations other than gaming. In contrast, in this chapter, we are particularly interested in the growth of specific expertise in gaming as a function of playing games. This is because aspects of the experience may alter with expertise. There is not much direct evidence as of yet about the growth of gaming expertise as a result of playing digital games. One not so recent empirical study by Holbrook et al. (1984) showed improvement of performance over gaming sessions. Participants played an Atari-game, in which they had to land a spacecraft on the moon without crashing. Performance in one session was positively related to that of the next session, and negatively to experienced difficulties and arousal. VanDeventer and White (2002) demonstrated, in a controlled study, that children with experience in gaming exhibited genuine expert behavior when playing Super Mario. This study used external criteria developed in a previous study by Chi, Glaser and Farr (1988). Expertise involves seven behaviors: Excelling in one’s own domain; perceiving large meaningful patterns; fast and error-free problem solving; superior short-term and long-term memory; spending a great deal of time analyzing problems qualitatively; and strong self-monitoring skills. In addition, Chi et al. found significant differences in the behaviors between players of different experience levels defined by game scores and self-reports of skill. Two studies addressed divided visual attention. Playing a digital game requires that players keep track simultaneously of many stimuli at different locations in their visual field. This can only be realized if they are able to shift their attention quickly from one set of information to another. The first study found that expert players of digital games are better at dividing their visual attention than novices. The participants had to locate a target on the computer screen as quickly as they could. When the target appeared in an unexpected location, expert players responded faster than novices (Subrahmanyam et al., 2001). The second study tested visual attention and its spatial distribution. One group lacking earlier experience with games was asked to play Medal of Honor, a rather complex 3D war game that requires divided visual attention. Participants were trained for one hour per day for ten consecutive days. A control group played Tetris, a less complex puzzle game that does not require divided visual attention. After ten days, Medal of Honor gamers showed a significant increase in their capacity for divided visual attention. This effect did not occur among the Tetris players (Green and Bavelier, 2003). If it seems extremely plausible that

550

Product Experience

repeated gaming will increase skills, then what does an increase of expertise mean for the gaming experience? It means that players have more confidence and can persist in playing for a long time. They recognize game situations and, more generally, readily know what to do, as affordances are more salient. As a result, thresholds for experiencing flow and emotions typical for various kinds of games are lowered. The typical gamer’s heightened experience of interest, enjoyment, and control, then, is based on skills acquired in hundreds of sessions and thousands of hours of practice. The flipside of the expertise coin may be that expert gamers do not easily content themselves with successes that anyone could achieve. Expertise also frees the player from a number of attentional, perceptual, motor, and cognitive burdens, which facilitates reflection not only on strategy, but also on one’s use of the game’s affordances and even the experience itself. Expert gamers have the opportunity to take from the game what matters to them. They shape the game’s contribution to their personal projects, and, ultimately, to their self-image and self-presentation. Accumulation of experiences: Intrinsic motivation and social identity Repeated gaming not only affects skills, but it also results in changes in affect. Players with low affinity with the activity will drop out. Typical gamers, those who persisted in gaming, have an interest that has become an emotional disposition, which prolongs the emotion across the confines of separate sessions. Silvia (2001) elaborated the relations between the two affects. Applying his views to gaming, every game experience may add to interest in gaming as a permanent disposition. The regulation of gaming activities alters from extrinsic, driven by external rewards, such as praise and acceptance by a community of peers, to intrinsic. Deci proposed that an intrinsic motivation acts as a regulator of experience ‘… by focusing on three psychological needs – the needs for competence, relatedness, and self-determination (i.e. autonomy) …’ (1992, p. 170). According to Deci, intrinsic motivations are part of human nature, and they are, in the end, necessary to explain why people engage in pastimes such as mountaineering (and, we will add, more pedestrian forms such as gaming). Inherent to it, is a drive to autonomy and growth from learning and doing an activity for its own sake. Intrinsic motives border on social ones. Jansz (2005) pointed out that games are used as a laboratory for experimenting with emotions. Experimenting may assume the shape of trying out affect-based personal identities that are negatively valued by society. For many gamers, sharing their knowledge and experience is an important incentive for gaming (Jansz and Martens, 2005; Jansz and Tanis, 2007). This interaction contributes to the development of a gamer identity, which is subsequently shared with others in the gaming community (Durkin, 2006). Being a gamer is as much a social identity as being a (passionate) fan of a football team (Ellemers, Spears and Doosje, 2002). It allows gamers to identify with their own group and to distinguish themselves apart from other social groups, in particular, from people who do not play digital games. The satisfaction of these social needs inspires a continuous desire for learning and personal growth, including social learning, as it did in the case of intrinsic motivation. The experience of typical gamers, then, differs from that of incidental gamers because the satisfaction they find in the activity is based on enjoying the activity for its own sake and for the sake of the self or one’s social identity. It is intrinsic motivation and self-image that explain the remarkable effort that typical gamers invest in a particular game and in gaming. As previously illustrated, it also explains why frustration and nonreward are not always, and not only, experienced as negative events by typical gamers. Instead, such events may be seen as an occasion to engage in enjoyable action, bolstering the self.

The game experience

551

We note that expertise, intrinsic motivation, and social identity co-develop across gaming sessions. As expertise grows, players can monitor their own behavior and reasoning; they reflect on the difference between optimal and less than optimal responses, courses of action, strategies, and so on. They can compare their performance with that of other, imaginary or real, players. In such reflections, they may notice advances they made in time and enjoy their own progress. Increasing mastery of perceptual and motor skills and an improving record or rating may boost the ego and sustain their identity as an expert or passionate gamer. This may also raise their status among their peers in and outside the gaming community. The joint perspectives of growing expertise, intrinsic motivation, and identity development in repeated gaming allow one to see every single gaming experience as one in a series or as a project in which the person is working on his or her mastery of and affinity with all aspects of the game.

3.3. Conclusion and methodological outlook We believe that our framework will be useful in research supporting evaluation and development of digital games as products. This is because it is explicitly based on the conception of games as entertainment products. Good games stir emotions to a degree that can only astound the uninitiated as theoreticians of play and games, such as Huizinga (1950) and Goffman (1972), have noted. In line with this, we propose that a truly emotional energy drives gamers. This framework also derives usefulness for product research from the idea that digital games are not disposable entertainment products which return ephemeral fun at low cost, but rather commodities that people attach to and invest in. There is a good use for good games. This framework grants a position to self-selection in the experience of gaming, as well as to self-regulated learning by doing. It also includes the social identity of the gamer as it is developed in communal interaction with other gamers. Applying this framework to game development, in particular, should allow for insights in game system features and player experience. The concept of affordance (Gibson, 1979) bridges the gap between the two. Our account of the relations between affordances and experience is incomplete at present. Further research is needed to complete this framework. We propose to focus on two steps. First, we suggest the anchoring of affordances in game system mechanics and dynamics. Affordances need to be perceived by players, but they should also be descriptively grounded in game characteristics. Secondly, we suggest a detailed specification of game genres and player types. Detailed insight is required into attributes of specific genres (and titles) and the possible relation with specific groups of players. The need for further elaboration of this framework does not prevent it from being used as it is. A major consideration in choosing particular methods is the temporal dynamics of the emotion processes. As an answer to the issue of dynamic responses we propose event sampling, rather than temporal sampling, because games typically have a structure allowing for the former option. Units of analysis correspond to embedded game cycles. In every episode, affordances have to be identified. Crucial events can be distinguished in the episode, giving rise to immediate goals, available means, and outcomes in terms of goals that are conditional on player action. Qualitative walk-throughs with test players will further this analysis. Emotion analysis is the next step in the description. Affordances can be translated into appraisal profiles for the various emotions. To give an example, the relations between challenge and control set by a particular event (given a particular goal) can be specified as determinants of interest. As another example, consider the factors mentioned in discussing motivational tension in relation to goals and goal structure (explained in Section 3.2). These are elements of the ongoing appraisal process of interest, in which

552

Product Experience

the history of activities and outcomes is included, and various appraisals may differ as a function of choices that the player has made. The appraisals, together with the emotional response, can be established empirically by observing test players. Instruments for measuring emotion are available in formats that can be shaped to the particular game and player. Free format verbalizations of emotion can be used, as well as standardized verbal scales (see for example Scherer, 2005). Additional interviewing is required because, in gaming, the distinction between positive and negative affect may be complicated by the intervention of intrinsic motivation, rendering putatively negative experiences into positive ones. In order to establish the hedonic tone of experience with more certainty, more direct measures are needed than verbalizations with interviews; for example, the coding of facial expression. The time scale of some crucial emotions, such as joy, also necessitates direct measurement. Such phasic emotions come, peak, and go within short time spans. For example, a moment of control and triumph may quickly give way to another difficulty. Ravajas et al. (2006) offer an example of the use of facial myography and psycho-physiological recording that is helpful in analyzing short-lived positive and negative affect in gaming. Experience prediction and tests can be done in the process, or even as part of the development of a game. Tests provide designers with feedback about intended effects. Programming a trace in the game system with annotations for certain states and events about constituted affordances and expected appraisals, and emotional responses, is not a difficult task, and can be of great help in event sampling. Authoring systems (software tool environments) for creating games are needed that allow for user testing and system adaptations without breaking up an ongoing session. In this way, the gamer’s experience is directly optimized, which results in a shortened feedback loop from developer to user and back. Discovering how the use of a game can be optimized, that is, how players can get the most out of it, requires analyzing the development of expertise and intrinsic motivation for that game, as well as the interaction with affordances and emotions. Like the experience of a single session or game cycle, this is an ongoing process that can be captured by taking ‘snapshots’ along the way. Here, time sampling seems to be indicated rather than event sampling. In this case, measures of performance and motivation have to be registered. Instruments for measuring motivation are available (e.g. SIMS, Guay, Vallerand and Blanchard, 2000), and those for expertise can be devised from available examples (e.g. Chi, Glaser and Farr, 1988). Both seem appropriate for repeated use over longer periods. Finally, the transition of incidental players into typical gamers (typical gamers being persons who have a completely intrinsic interest in gaming and a distinct social identity) is a development that also must be studied in a longitudinal fashion, but, in this case, using qualitative, ethnographic methods. The typical gamer is involved in a project composed of a series of single gaming experiences. It will be worth while for game developers and scholars alike to know what role these separate experiences in a particular game play in the development of long-term affective investments.

REFERENCES Ainley, M., Hidi, S. and Berndorff, D. (2002). Interest, learning, and the psychological processes that mediate their relationship. Journal of Educational Psychology, 94(3), 545–561. Anderson, C. A. (2004). An update on the effects of playing violent video games. Journal of Adolescence, 27, 113–122.

The game experience

553

Anderson, C. A. and Bushman, B. J. (2001). Effects of violent games on aggressive behavior, aggressive cognition, aggressive affect, physiological arousal, and prosocial behavior: A meta-analytic review of the scientific literature. Psychological Science, 12, 353–359. Apter, M. J. A. (1991). A structural phenomenology of play. In: J. H. Kerr and M. J. A. Apter (Eds.) Adult play: A reversal theory approach, pp.13–29. Amsterdam: Swets and Zeitlinger. Bandura, A. (1997). Self-efficacy: the exercise of control. New York: Freeman. Berlyne, D. E. (1969). Laughter, humor and play. In: G. Lindzey and E. Aronson. The handbook of social psychology, Vol III (2nd Ed.), pp. 795–852. Reading: Addison-Wesley. Bordwell, D. (2006). The way Hollywood tells it. Story and style in modern movies. Berkeley: University of California Press. Carver, C. S. and Scheier, M. F. (1998). On the self-regulation of behavior. Cambridge: Cambridge University Press. Chi, M., Glaser, R. and Farr, M. (Eds.) (1988). The nature of expertise. Hillsdale, NJ: Lawrence Erlbaum. Colwell, J., Grady, C. and Rhaiti, S. (1995). Computer games, self-esteem and gratification of needs in adolescents. Journal of Community and Applied Social Psychology, 5, 195–206. Collwell, J. and Payne, J. (2000). Negative correlates of computer game play in adolescents. British Journal of Psychology, 91, 295–310. Csikszentmihalyi, M. and Csikszentmihalyi, I. S. (1988). Optimal experience: Psychological studies of flow in consciousness. Cambridge: Cambridge University Press. Darwin, C. (1965). The expression of the emotions in man and animals. Chicago: University of Chicago Press. Orig. work publ. 1872. Deci, E. L. (1992). On the nature and function of motivation theories. Psychological Science, 3(3), 167–171. DFC Intelligence (2006). The online game market. Retrieved 14 September 2006 from http://www.dfcint.com/ Durkin, K. (2006). Game playing and adolescents’ development. In: P. Vorderer and J. Bryant (Eds.) Playing video games, pp. 415–428. Mahwah (NJ): Lawrence Erlbaum. Ellemers, N., Spears, R. and Doosje, B. (2002). Self and social identity. Annual Review of Psychology, 53, 161–186. Elliott, A. J. and Harackiewicz, J. M. (1994). Goal setting, achievement orientation, and intrinsic motivation: A mediational analysis. Journal of Personality and Social Psychology, 66, 968–980. Ellsworth, P. C. (2003). Confusion, concentration, and other emotions of interest: Commentary on Rozin and Cohen (2003). Emotion, 3(1), 81–85. Entertainment and Leisure Software Publishers Association (ELSPA) (2006). Record sales for game market (Press release 9 Jan 2006). Retrieved August, 21, 2006 from http://www.elspa.com/ Entertainment Software Association (ESA) (2006). 2006 Sales, demographics and usage data. Retrieved 21 August, 2006 from http://www.theesa.com/ Fischer, A. H. (1993). Sex differences in emotionality. Feminism and Psychology, 3, 303–318. Frijda, N. H. (1986). The emotions. Cambridge: Cambridge University Press. Fritz, J. (1997). Macht, Herrschaft und Kontrolle im Computerspiel. In: J. Fritz and W. Fehr (Herausgeb.) Handbuch Medien: Computerspiele, pp. 183–197. Bonn: Bundeszentrale für Politische Bildung. Funk, J. and Buchman, D. (1996). Playing violent video and computer games and adolescent self-concept. Journal of Communication, 46, 19–32. Garris, R., Ahlers, R. and Driskell, J. E. (2002). Games, motivation, and learning: A research and practice model. Simulation and Gaming, 33, 441–467. Gerrig, R. J. (1993). Experiencing narrative worlds: On the psychological activities of reading. New Haven: Yale University Press. Gibson, J. J. (1979). The ecological approach to visual perception. Boston, MA: Houghton Mifflin. Goffman, E. (1972). Role distance. In: Encounters, pp. 73–134. Harmondsworth: Penguin. Orig. work publ. 1961. Graner Ray, S. (2004). Gender inclusive game design. Boston, MA: Charles River Media. Green, C. S. and Bavelier, D. (2003). Action video game modifies visual selective attention. Nature, 423, 534–537. Green, M. C., Brock, T. C. and Kaufman, G. F. (2004). Understanding media enjoyment: the role of transportation into narrative worlds. Communication Theory 14(4), 311–327. Griffiths, M. (2005). The therapeutic value of video games. In: J. Raessens and J. Goldstein (Eds.) Handbook of computer game studies, pp. 161–171. Cambridge, MA: The MIT Press. Griffiths, M. D., Davies M. N. O. and Chappell D. (2004). Online computer gaming: a comparison of adolescent and adult gamers. Journal of Adolescence, 27, 87–96. Grodal, T. (2000). Video games and the pleasures of control. In: D. Zillmann and P. Vorderer (Eds.) Media entertainment. The psychology of its appeal, pp. 197–212. Mahwah, NJ: Lawrence Erlbaum. Grodal, T. (2003). Stories for eye, ear, and muscles. Video games, media, and embodied experiences. In: M. J. P. Wolf and B. Perron (Eds.) The video game theory reader, pp. 129–155. London: Routledge.

554

Product Experience

Guay, F., Vallerand, R. J. and Blanchard, C. (2000). On the assessment of situational intrinsic and extrinsic motivation: The Situational Motivation Scale (SIMS). Motivation and Emotion, 24, 175–213. Harackiewicz, J. M. (1979). The effects of reward contingency and performance feedback on intrinsic motivation. Journal of Personality and Social Psychology, 37, 1352–1363. Harris, P. (2000). Understanding emotion. In: M. Lewis and J. M. Haviland-Jones (Eds.) Handbook of Emotions (2nd Ed.), pp. 281–291. New York: Guilford Press. Hidi S. and Baird, W. (1986). Interestingness: A neglected variable in discourse processing. Cognitive Science, 10, 179–194. Holbrook, M. B., Chestnut, R. W., Oliva, T. A. and Greenleaf, E. A. (1984). Play as a consumption experience: the roles of emotions, performance, and personality in the enjoyment of games. The Journal of Consumer Research, 11(2), 728–739. Huizinga, J. (1950). Homo ludens. Boston: The Beacon Press. Orig. work publ. 1938. Hunicke, R., LeBlanc, M. and Zubek, R. (2004). MDA: A formal approach to game design and game research. Proceedings of the Challenges in Game AI Workshop, Nineteenth National Conference on Artificial Intelligence. Retrieved 27 June 2006 from http://www.cs.northwestern.edu/⬃hunicke/MDA.pdf Ivory, J. D. (2006). Still a man’s game: gender representation in online reviews of video games. Mass Communication and Society, 9(1), 103–114. Izard, C. E. (1977). Human emotions. New York: Plenum Press. Izard, C. E. (1992). Basic emotions, relations among emotions, and emotion–cognition relations. Psychological Review, 99, 561–565. Izard, C. E. and Ackerman, B. P. (2000). Motivational, organizational, and regulatory functions of discrete emotions. In: M. Lewis and J. M. Haviland-Jones (Eds.) Handbook of emotions (2nd Ed.), pp. 253–264. New York: Guilford Press. Jansz, J. (2000). Masculine identity and restrictive emotionality. In: A. H. Fischer (Ed.) Gender and emotion. Social psychological perspectives, pp. 166–186. Cambridge: Cambridge University Press. Jansz, J. (2005). The emotional appeal of violent video games for adolescent males. Communication Theory, 15, 219–241. Jansz, J. and Martens, L. (2005). Gaming at a LAN-event: The social context of playing video games. New Media and Society, 7, 333–355. Jansz, J. and Tanis, M. (2007). The appeal of playing online first person shooter games (FPSG). Cyberpsychology and Behavior, 10, 133–136. Juul, J. (2005). Half-real. Video games between real rules and fictional worlds. Cambridge, MA: The MIT Press. Kaplan, S. (1992). Environmental preference in a knowledge-seeking, knowledge using organism. In: J. H. Barkow, L. Cosmides and J. Tooby (Eds.) The adapted mind, pp. 581–598. New York: Oxford University Press. Keltner, D. and Shiota, M. N. (2003). New displays and new emotions: A commentary on Rozin and Cohen (2003). Emotion, 3, 86–91. Kiousis, S. (2002). Interactivity: a concept explication. New Media and Society, 4(3), 355–383. Klimmt, C. and Hartmann, T. (2006). Effectance, self-efficacy, and the motivation to play video games. In: P. Vorderer and J. Bryant (Eds.) Playing video games. Motives, responses, and consequences, pp. 133–145. Mahwah, NJ: Lawrence Erlbaum. Klug, G. C. and Schell, J. (2006). Why people play games. An industry perspective. In: P. Vorderer and J. Bryant (Eds.) Playing video games. Motives, responses, and consequences, pp. 91–100. Mahwah, NJ: Lawrence Erlbaum. Lazarus, R. S. (1991). Emotion and adaptation. New York: Oxford University Press. LeBlanc, M. (2003). Tools for creating game dynamics. In: K. Salen and E. Zimmerman. The game design reader. A rules of play anthology, pp. 438–459. Boston, MA: The MIT Press. Lee, K. M. (2004). Presence explicated. Communication Theory, 14, 27–50. Lee, K. M. and Peng, W. (2006). What do we know about social and psychological effects of computer games? A comprehensive review of the current literature. In: P. Vorderer and J. Bryant (Eds.) Playing video games, pp. 325–345. Mahwah (NJ): Lawrence Erlbaum. Lieberman, D. A. (2006). What can we learn from playing interactive games? In: P. Vorderer and J. Bryant (Eds.) Playing video games, pp. 379–397. Mahwah (NJ): Lawrence Erlbaum. Linderoth, J., Lindström, B. and Alexandersson, M. (2004). Learning with computer games. In: J. Goldstein, D. Buckingham and G. Brougère (Eds.) Toys, games, and media, pp. 157–176. Mahwah, NJ: Lawrence Erlbaum. Lucas, K. and Sherry, J. L. (2004). Sex differences in video game play: A communication-based explanation. Communication Research, 31, 499–523. Morlock, H., Yando, T. and Nigolean, K. (1985). Motivation of video game players. Psychological Reports, 57, 247–250.

The game experience

555

Newman, J. (2002). In search of the videogame player: The lives of Mario. New Media and Society, 4(3), 405–422. Nikken, P. (2003). Computerspellen in het gezin [Computer games in families]. Hilversum: NICAM. Nutt, D. and Railton, D. (2003). The Sims: Real life as genre. Information, Communication and Society, 6, 577–92. NVPI (2006). NVPI Marktinformatie Interactief 2005 [NVPI Marketinformation interactive 2005] Retrieved 21 August, 2006 from http://www.nvpi.nl/. Oatley, K. and Johnson-Laird, P. N. (1987). Towards a cognitive theory of the emotions. Cognition and Emotion, 1, 29–50 Panksepp, J. (1982). Toward a general psychobiological theory of emotions. The Behavioral and Brain Sciences, 5, 407–467. Poole, S. (2000). Trigger happy. The inner life of videogames. London: Fourth Estate. Prensky, M. (2005). Computer games and learning: Digital game based learning. In: J. Raessens and J. Goldstein (Eds.) Handbook of computer game studies, pp. 97–122. Cambridge, MA: The MIT Press. Raney, A. A., Smith, J. K. and Baker, K. (2006). Adolescents and the appeal of video games. In: P. Vorderer and J. Bryant (Eds.) Playing video games, pp. 165–179. Mahwah (NJ): Lawrence Erlbaum. Ravaja, N., Saari, T., Salminen, M., Laarni, J. and Kallinen, K. (2006). Phasic emotional reactions to video game events: A psychophysiological investigation. Media Psychology, 8, 343–367. Roberts, D. F., Foehr, U. G., Rideout, V. J. (2005). Generation M: Media in the lives of 8–18 year-olds. Menlo Park: Kaiser Family Foundation. Salen, K. and E. Zimmerman (2003). Rules of play. Game design fundamentals. Cambridge, MA: The MIT Press. Salen, K. and E. Zimmerman (2005). Game design and meaningful play. In: J. Raessens and J. Goldstein (Eds.) Handbook of computer game studies, pp. 59–80. Cambridge, MA: The MIT Press. Scherer, K. R. (2005). What are emotions? And how can they be measured? Social Science Information, 44, 695–729. Scherer, K. R., Schorr, A. and Johnstone, T. (Eds.) (2001). Appraisal processes in emotion: Theory, methods, research. New York and Oxford: Oxford University Press. Schneider, E. S., Lang, A., Shin, M. and Bradley, S. D. (2004). Death with a story. How story impacts emotional, motivational, and physiological responses to first-person shooter video games. Human Communication Research, 30, 361–375. Sellers, M. (2006). Designing the experience of interactive play. In: P. Vorderer and J. Bryant (Eds.) Playing video games. Motives, responses, and consequences, pp. 9–22. Mahwah, NJ: Lawrence Erlbaum. Sherry, J. L. (2001). The effects of violent video games on aggression. A meta-analysis. Human Communication Research, 27, p. 409–432. Sherry, J. L. (2004). Flow and media enjoyment. Communication theory, 14(4), 328–347. Sherry, J. L., Lucas, K., Greenberg, B. S. and Lachlan, K. (2006). Videogame uses and gratifications as predicators of use and game preference. In: P. Vorderer and J. Bryant (Eds.) Playing video games, pp. 213–224. Mahwah (NJ): Lawrence Erlbaum. Shields, S. A. (2000). Thinking about gender, thinking about theory. In: A. H. Fischer (Ed.) Gender and emotion. Social psychological perspectives, pp. 3–23. Cambridge: Cambridge University Press. Silvia, P. (2001). Interest and interests: the psychology of constructive capriciousness. Review of General Psychology, 5(3), 270–290. Silvia, P. J. (2005). What is interesting? Exploring the appraisal structure of interest. Emotion, 5, 89–102. Singer, D. G and Singer, J. L. (2005). Imagination and play in the electronic age. Cambridge, MA: Harvard University Press. Smith, B. P. (2006). The (computer) games people play. In: P. Vorderer and J. Bryant (Eds.) Playing video games, pp. 43–56. Mahwah, NJ: Lawrence Erlbaum. Subrahmanyam, K., Greenfield, P., Kraut, R. and Gross, E. (2001). The impact of computer use on children’s and adolescents’ development. Applied Developmental Psychology, 22, 7–30. Tamborini, R., Eastin, M. S., Skalski, P., Lachlan, K., Fediuk, T. A. and Brady, R. (2004). Violent virtual video games and hostile thoughts. Journal of Broadcasting and Electronic Media, 48, 335–357. Tan, E. S. (1996). Emotion and the structure of narrative film. Mahwah (NJ): Lawrence Erlbaum. Tan, E. S. (2000). Emotion, art and the humanities. In: M. Lewis and J. M. Haviland-Jones (Eds.) Handbook of Emotions (2nd. Ed.), pp. 116–136. New York: Guilford Press. Tan, E. S. and Dev, S. (2001). Bypassing the author. In: G. Steen and D. H. Schram (Eds.) The psychology and sociology of literature, pp. 289–313. Amsterdam: John Benjamins. Tennyson, R. D. and Breuer, K. (2002). Improving problem solving and creativity through the use of complexdynamic simulations. Computers in Human Behavior, 18, 650–668.

556

Product Experience

Tomkins, S. S. (1984). Affect theory. In: K. R. Scherer and P. Ekman (Eds.) Approaches to emotion, pp. 163–195. Hillsdale, NJ: Lawrence Erlbaum. Valkenburg, P. M. and Peter, J. (2006). Fantasy and imagination. In: J. Bryant and P. Vorderer (Eds.) Psychology of entertainment, pp. 115–118. Mahwah (NJ): Lawrence Erlbaum. VanDeventer, S. S. and White, J. A. (2002). Expert behavior in children’s video game play. Simulation and Gaming, 33(1), 28–28. Van Reekum, C. M. and Johnstone, T., Banse, R., Etter, A., Wehrle, T. and Scherer, K. (2004). Psychophysiological responses to appraisal dimensions in a computer game. Cognition and Emotion, 18(5), 663–688. Vorderer, P., Hartmann, T. and Klimmt, C. (2003). Explaining the enjoyment of playing video games: The role of competition. In: D. Marinelli (Ed.) Proceedings of the 2nd International Conference on Entertainment Computing (ICEC 2003), Pittsburgh, pp. 1–8. New York: ACM. Yates, S. J. and Littleton, K. (1999).Understanding computer game cultures: A situated approach. Information, communication and society, 2(4), 566–583. Weber, R., Ritterfeld, U. and Mathiak, K. (2006). Does playing violent video games induce aggression? Empirical evidence of a functional magnetic resonance imaging study. Media Psychology, 8(1), 39–60. White, R. W. (1959). Motivation reconsidered: The concept of competence. Psychological Review, 66, 297–331. Williams, D. (2003). The video game lightning rod. Information, Communication and Society, 6(4), 523–550. Williams, D. (2006). A brief social history of game play. In: P. Vorderer and J. Bryant (Eds.) Playing video games, pp. 197–212. Mahwah (NJ): Lawrence Erlbaum.

24

EXPERIENCING FOOD PRODUCTS WITHIN A PHYSICAL AND SOCIAL CONTEXT HERBERT L. MEISELMAN US Army Natick Soldier Center

1. INTRODUCTION Imagine yourself flying on an airplane on business or pleasure as lunchtime approaches. What do you expect for lunch – a great fresh salad or a great fresh sandwich? Do you expect to enjoy your lunch? Or imagine yourself at your destination on your next holiday at lunchtime – where will you be? What will you eat? Will you enjoy it? I like to go to warm climates for a winter vacation away from cold New England, and I picture myself eating out of doors in a café with great atmosphere and the sun on my face. Product experience is about products, but it is also about places, that is, where we eat.

1.1. Overview The goal of this chapter is to broaden the definition of product experience from definitions which view product experience as the interaction of a person and a product. In fact both experience and substantial research now show that product experience is the interaction of a person and a product in a particular context or situation. Contextual research has received far less attention than either product research or person research, probably because both of these classes of variables are the main foci of major stakeholders. The huge product industry is the main stakeholder for product research; the huge health industry is the main stakeholder for person research. Technologies which support product-oriented research and health-oriented research ensure that these technologies remain the foci of food research and development activities. Product Experience Copyright © 2008 Elsevier Ltd.

559

560

Product Experience

However, research clearly shows that the same product is experienced differently in different environments. In this chapter I will review the evidence for this. But our own experience confirms this as well; occasionally in fancy restaurants we question whether the great experience is the food itself or the ambiance. And on the other end of the spectrum, we wonder whether we could ever enjoy airline food or hospital food. Below I will review research which shows that the experience of the same food in different settings is indeed different. This fundamental fact, that the same product is experienced differently in different environments, means that we must consider environment when designing and testing foods. And we must also consider environment when designing products for food service, because within the service setting we are truly bringing together the product and the person with the environment. Food products are often developed and tested in a laboratory devoid of ambiance. And people often sample food products in the laboratory. But food service needs a real environment, a naturally occurring environment. Are the results of the laboratory applicable to food service? Can one predict food service success from laboratory studies? And can one predict home use success from the laboratory? The home is perhaps the most difficult environment to study, because it exists largely outside of our observation and control. Any attempt to control it obviously changes the natural setting in the home. I will try to address these questions in this chapter. In addition, many researchers have focused on one or several dimensions of eating, for example, the sensory attributes of foods. The sensory topic alone has been the subject of numerous textbooks, journal articles, conferences, and even an entire division of the Institute of Food Technology in the United States. In focusing on sensory issues and other issues, researchers have ignored the meal context, in which foods are consumed in combination with other foods and with other meal variables such as the social ones. Further, when researchers seek to serve a meal to subjects in a laboratory setting, they have often served a single food or a combination of foods without appropriate consideration of what foods normally go together in that culture. This has resulted in unrealistic meals served to subjects. Any grouping of foods does not necessarily constitute a meal. As you will see below, meal context is an important part of the overall eating environment. Research shows that food products are consistently liked and consumed more in some environments than in others. This poses serious practical problems for food developers and marketers. Can the same version of a product be targeted to home use and institutional use, to luxury food service use and supermarket use? These questions also present issues for those interested in product acceptance, and those interested in product consumption. Marketing and sales must consider where the product is to be used; and health care professionals must consider the environment as well. The recommendation leading from this three factor approach to foods, encompassing the product, the individual, and the environment, is that product testing should incorporate environmental/context concerns for better test validity. Contextual concerns should be factored into testing to address these issues, and to better predict how products will be experienced in different locations. And more experience with context will continue to identify which environmental/contextual variables are critical in these environmental effects. Identifying these variables will help us to improve product design and testing.

1.2. Background There are very few references to environmental, contextual or situational controls of eating before the 1980s. Researchers at the US Army Natick Laboratory began to study the context in which soldiers eat in the 1980s, and first reported their work

Experiencing food products within a physical and social context

561

(Meiselman, Hirsch and Popper, 1988) at the 1988 conference on Food Acceptability at Reading, England (Thomson, 1988). They discussed the lack of agreement between product liking (acceptability) and product consumption in their data – products which were most liked were not necessarily most consumed. They were just beginning to become aware that environmental variables were controlling intake, as they do under certain conditions. At the same Reading meeting Schutz (1988) introduced the concept of appropriateness, which tries to include situational concerns in food behavioral research. Following that, Meiselman (1992a,b) argued for greater use of natural contexts in research, suggesting that we ‘… refocus human eating research towards greater use of real meals, served to real people (not subjects), in real eating situations’ (p. 54). Rozin and Tuorila (1993) discussed context at the first Pangborn Sensory Science Symposium, and presented an organizational scheme for contextual variables. Their scheme had two dimensions: A temporal dimension distinguishing simultaneous as opposed to consecutive context; and a reference unit dimension distinguishing levels of eating moving from the simpler bite, to dish, and finally to meal pattern. Rozin and Tuorila note that, ‘Whereas in most sensory studies, the sip or bite is the unit of reference, in natural settings, the meal seems to be the particularly appropriate unit…’. I will return to the importance of meal context below. The temporal dimension of context was also used by Bell and Meiselman (1995) in their review of contextual variables. Meiselman (1996) discussed context within the three-part organizational scheme of the food, the person and the environment. More recently, Wansink (2004) and Stroebele and de Castro (2004) have reviewed work relating eating to the environment.

1.3. Terminology Terminology to deal with environmental variables has not been clearly defined because of the relative lack of research and discussion in this area. The words environment, context, situation, setting, and location are often used interchangeably. Probably the two clearest terms are location and setting because they usually refer to a physical location, such as home, restaurant or market. The terms environment, situation, and context often refer to a much broader range of variables pertaining to a location. Thus any location, such as home, will have a wide group of variables that characterize it. Those variables might include physical features (size, color, noise level, etc.), social features (number of people present, ages, genders, relationships, etc.), economic features (food cost, etc.), and other more subtle variables such as mood, decoration, etc. Thus, authors refer to contextual variables, environmental variables or situational variables without clear definition of what is or is not included in these categories. This makes the task more difficult for both the reader and the researcher. The task is further complicated by our lack of knowledge about which environmental factors contribute to variance in consumer response. Research is beginning to shed some light on this, and will be reported below. The variables include meal context and food choice, that is, whether a food is served as part of a meal, and whether the consumer is offered a choice at the time of consumption. The factors of meal context and providing a choice also characterize most laboratory research, and raise the question of whether laboratory methods might be inadequate. The research reported and reviewed in this chapter will involve several different research designs. Some of the research has been carried out under natural conditions. Surprisingly, one has to define what constitutes natural conditions, because authors claim to have natural conditions when they do not. Natural conditions mean that the testing situation and the testing product(s) exist in nature. When testing conditions are modified

562

Product Experience

to be ‘more natural’ they might be termed ‘naturalistic’ or even ‘more natural’, but the term natural conditions is reserved for what is actually natural. This excludes all laboratory research – no one ever eats in a laboratory as part of their normal day (other than the researcher eating a sandwich while working!). Naturalistic settings are settings designed to imitate or produce the effect or appearance of nature. When a laboratory is modified to make it ‘like a restaurant’ or ‘like home’, it is a naturalistic setting. Natural environments can include the home, restaurants, cafeterias, picnic locations, and institutions such as hospitals and schools. In hospitals patients eat in their rooms, as well as in other locations such as cafeterias; and in schools children eat in the classrooms and in other locations such as cafeterias. It is surprising that more research is not carried out in institutional cafeterias because many researchers have access to them. And a surprising amount of food is consumed at work sites, school sites, hospital sites, and other institutional settings. Wherever research is conducted, the researcher is usually interested in two classes of information: How much the food is liked (and why); and how much is consumed (and why). These issues of liking and intake cover most of the research in this chapter. Liking and intake are the two main facets of product experience, aside from the shopping experience itself. The reader might want to look at models of the whole food process. Marshall’s food provisioning process (Marshall, 1995) is based on earlier cross-cultural research. The model contains five sequential stages: Acquisition; preparation; cooking; eating; and disposal. Each of these stages involves product factors (what to eat), economic factors (what to spend), social factors (with whom to prepare and eat), and environmental factors (where to do each stage of the model). An important message here is that product experience is different at each stage of the provisioning process, and we should not only concentrate on the eating stage. Product liking is usually considered in product development because it is thought to be highly related to purchase. But the emphasis on product liking ignores the major role of habit in product choice. And habit also might have a major role in product experience. Do products which are highly habitual, such as coffee or breakfast cereal, have the same product experience as equally complex products which are rarely consumed? Liking is also believed to correlate highly with product consumption (de Graaf et al., 2005; Lahteenmaki and Tuorila, 1995), but the degree of that correlation varies widely.

2. PRODUCTS ARE EXPERIENCED DIFFERENTLY IN DIFFERENT CONTEXTS It is more convenient to believe that an apple is an apple is an apple, as a poet once said. But most products are experienced differently based on context. We know from studies on taste perception that things taste different when they follow different tastes, and things even taste different when they follow water itself. But these contextual effects also work on much more complex perceptions such as food acceptability. These context effects become so ingrained in our thinking that they contribute to the development of stereotypes – airline food is bad, and fancy restaurant food is good. Cardello et al. (1996) asked people how good they expect the food to be in a number of different locations in a questionnaire study. He observed consistent differences in expected liking in the following order: Home → restaurant → fast food → school → military → airline → hospital. Note that home and restaurant occupy one end of the distribution and institutional settings like school, military, airline, and hospital occupy the other end.

Experiencing food products within a physical and social context

563

Recently Boutrolle and colleagues have studied the home environment, comparing it to a more traditional central location test. Home use testing (HUT) and central location testing (CLT) differ in a number of important aspects, as noted by Boutrolle. There is most likely a difference in desire to eat, choice or no choice, amount of product consumed, number of contacts with the sample, an appropriate time of consumption, a comfortable and social environment, foods served alone rather than in combination, and monadic rather than sequential presentation. Boutrolle et al. (2005) reviewed earlier studies on home testing, noting that while many found that laboratory tests and home tests produced different results, only two prior studies showed that product rankings differed in home use and CLT tests. Boutrolle et al. (2005) recruited people on the street for a 15-minute CLT, and compared this to home panels which used products for six successive days. They confirmed that home use produced higher ratings than CLT, and that product ranking was the same in both. The studies of King reported below confirmed this lack of ranking differences in most location changes. Boutrolle et al. argued that since CLT changes are more robust with changes in panel size, that robustness should be considered as an advantage of CLTs when considering where to conduct sensory/consumer testing. More recently, Boutrolle et al. (2007) compared CLT and HUT ratings of three different products. They confirmed that HUT gives higher ratings, but they noted product differences. While acknowledging a confounding of people and places in HUTs and CLTs, and raising the question of whether the two locations actually have different types of respondents, they placed most of the HUT-CLT difference with improved appropriateness in the HUT, which might include the following: (1) people choosing when to eat; (2) eating when actually hungry or thirsty in HUT; (3) prolonged product contact in HUT; (4) difference in the type and degree of involvement, with the question of whether CLTs stimulate a more analytical and critical thinking; (5) the usual context of consumption in HUT, with the authors noting that only 60% of HUT samples were consumed during meals. Boutrolle et al. conclude that when natural context is simpler, a CLT test might give a less biased result; however, when context is complex, HUT would better reflect that consumption situation. The authors call for more contextual research to understand these effects better. Meiselman and colleagues have also studied the effect of food service style on product experience. Meiselman et al. (2000) studied product experience in university restaurant settings where real customers pay for their food, and compared this with the institutional experience of a university cafeteria. They also served the same foods within a sensory laboratory. They found lower ratings in institutional settings as compared with restaurant settings, with laboratory ratings falling between (Table 24.1). Edwards et al. (2003) greatly expanded the number of different food service outlets, comparing acceptability of the same prepared food in ten different locations. They again observed that the acceptability of food in institutional settings is lower than in restaurant settings (Table 24.2). In all of these studies, there was about a one point scale point difference between the highest and lowest ratings of the same product. The authors attributed this difference to differences in expectations which consumers have for products in these different locations (Cardello, 1994; Cardello et al., 1996). Customers expect products in fine restaurants to be better, and expectation theory shows that actual product ratings move in the direction of expectations. A product which is expected to be better rates better than the same product with no expectation. Edwards et al. (2003) also observed an age effect, with hedonic rating increasing with age from teenage until middle age (46–65), when it declines slightly for older consumers (Table 24.3). This higher criticality of younger consumers has been noted in other studies, and raises the question of how people of different ages experience products. Is the experience the same, but judged against a different standard, or is the experience itself different?

564

Product Experience

TABLE 24.1 Acceptance ratings of a main dish (pasta and chicken) served in different settings MEAN FOOD ACCEPTANCE RATINGS FROM THREE SETTINGS Laboratory/Dining Halls/Training Restaurant Fettuccine alfredo with chicken Flavor

Texture

Color

Overall

N

Mean

s.d.

Mean

s.d.

Mean

s.d.

Mean

s.d.

89 113 18

5.80 5.18 6.78

1.23 1.63 0.55

5.15 5.13 6.28

1.33 1.60 1.45

5.65 5.39 6.33

1.17 1.47 1.03

5.79 5.28 6.67

1.03 1.61 0.69

Effects

F

df

P

Setting Attribute

9.56 5.67

2217 3651

0.000 0.001

Ratings were done on the seven point hedonic scale (Meiselman et al., 2000).

TABLE 24.2 Acceptance ratings of foods served in ten different settings Overall acceptability Location/situation

Mean

Army camp University staff refectory Private boarding school Freshers’ week buffet Private party Elderly residential home Student refectory Elderly day care centre THR patrons 4-star restaurant

6.63 6.64 6.66 6.69 6.99 7.05 7.09 7.09 7.58 7.63

N a a a a ab ab ab ab b b

43 36 88 83 77 43 33 33 19 32

Ratings were done on the nine-point hedonic scale, and sample sizes are shown. Significant differences are indicated by differences in letters a, b, c (Edwards et al., 2003).

King and colleagues also studied the acceptability of the same foods served in different food-service settings. King et al. (2004) studied the same pizza meal in both the laboratory and in a real restaurant, concluding that testing the same foods in an actual restaurant produces higher ratings. Moving from the laboratory to the real restaurant with the same products significantly increased liking (on a nine-point scale) from 6.7 to 7.4 averaged across all foods. Some products increased a lot (tea, 5.9–7.3), while others showed little or no increase (pizza, 7.2–7.4). King et al. (2007) confirmed these results in a second series of studies involving a national chain restaurant in the US. Once again restaurant settings produced higher ratings, with a difference of 4.1–5.0 measured on a six-point scale of liking. And once again, some products showed large effects, while one (lasagna) showed no effect. Based on these studies, Meiselman, King and Hottenstein

565

Experiencing food products within a physical and social context

TABLE 24.3 The effect of age on acceptance ratings of the same foods served in ten different settings Overall acceptability Age group

Mean

13–18 19–25 26–35 36–45 46–65 65

6.58 6.77 6.79 7.02 7.34 7.07

N a a ab a b ab

36 195 47 53 62 85

Ratings are shown in increasing order of acceptance score. Ratings were done on the nine-point hedonic scale, and sample sizes are shown. Significant differences are indicated by differences in letters a, b, c (Edwards et al., 2003).

(2004) concluded that laboratory testing underestimates true product acceptability and recommended adding 0.5 to 1.0 scale points to laboratory product ratings in order to estimate product experience in the real world. Until we have more data from more food types we cannot conclude that this effect is general; it is possible that well liked and popular foods, like pizza and lasagna in the US, are either less susceptible to these effects, or that pasta products perform differently for other reasons.

3. WHICH CONTEXTUAL VARIABLES CONTRIBUTE TO THE PRODUCT EXPERIENCE? As research on environmental or contextual variables has increased from the 1980s to the present time, there have been a number of reviews (Meiselman, 1996; Bell and Meiselman, 1995; Wansink, 2004; Stroebele and de Castro, 2004). There is still no comprehensive and integrated view of which environmental variables are critical, and how they operate to produce product experience. However, the field is evolving from consideration of single variables to more integrated studies, which are yielding progress in the direction of a more comprehensive view. Many environmental variables are best studied in natural eating situations, because these variables are part of the eating situation, and relatively unrelated to the food itself. These include effort to obtain food, eating duration, convenience, the physical environment, the presence of other people, and whether there is a choice of food. Interestingly, some of the early environmental studies of eating, mainly from the Stunkard group at the University of Pennsylvania (Stunkard and Kaplan, 1977; Coll et al., 1979; Myers et al., 1980), examined variables such as effort to obtain food. These studies were conducted in the context of over- or under-eating as related to weight loss.

3.1. Effort to obtain food Effort to obtain food is one of the most important environmental variables. In the early studies from the Stunkard group, Myers et al. (1980) examined the effect of product placement or ‘accessibility’ in a cafeteria service line. Products placed with easier access, and therefore less effort, were selected more often. The US Army studies reported by

566

Product Experience

Natick researchers in several publications (Marriott, 1995; Hirsch et al., 2005) mention the critical role of effort. The situations in which these studies were conducted, cafeterias and military field feeding, might exaggerate the role of effort in day-to-day household eating. But the importance of effort in human eating is consistent with its importance in animal eating, in which efficiency is a main driver. Meiselman and colleagues (Meiselman et al., 1994) reported a study in which effort was manipulated in a student cafeteria. Food choice, acceptability, and intake were measured. The studies took place in a student refectory or cafeteria where students ate daily, and paid for their food. To manipulate effort, one food item in each of two studies was moved from its usual location to a new location. In order to obtain the test food, the student had to obtain and pay for the meal in one meal line, and then go to the new line to obtain the test food. Both studies began with a baseline period in which regular eating was measured. In the first study, using chocolate confection, the effort manipulation lasted one week (Table 24.4a), and in the second study, using potato chips or crisps, the effort condition lasted three weeks, followed by a recovery phase in which the chips were returned to their former location (Table 24.4b). Increased effort reduced selection of the test foods to virtually zero (Tables 24.4a,b). This was a strong message that environmental variables can have very large effects. Research in product variables or psychological/physiological variables of the eater often have much smaller effects, yet they continue to receive more attention than environmental variables. The acceptability of the test foods did not vary with the effort manipulation, showing that choice, acceptance, and intake are not always correlated. We have observed this same lack of correspondence between choice, acceptance, and intake in many other studies, as noted below. TABLE 24.4a The effect of effort on consumption rates of university students in a student cafeteria over a one-week period

Condition Baseline Effort

Main dishes 0.471 0.38 N.S

Chocolate

Total dessert fruit accessory foods

0.392 0.031

0.353 0.549

p 0.001

p 0.10

(Meiselman et al., 1994.)

TABLE 24.4b The effect of effort on consumption rates of university students in a student cafeteria over a three-week period and a three-week recovery with original conditions restored Condition

Main meal

Starch items

Bread

Crisps

Baseline Effort Recovery Difference between the three periods Contrast between baseline and manipulation

0.385 0.408 0.398 N.S N.S

0.274 0.462 0.398 p 0.01 p 0.05

0.000 0.005 0.006 N.S N.S

0.718 0.092 0.322 p 0.001 p 0.001

(Meiselman et al., 1994.)

567

Experiencing food products within a physical and social context

The second study might be the only study in which recovery from an effort condition was measured. When the chips/crisps were returned to their original location for a three week recovery period, the choice of chips did not fully recover to its baseline level (Table 24.4b). Under the effort condition the chip selection rate decreased from 0.718 to 0.092, but only increased back to 0.322 in recovery. Thus, a three week recovery did not produce even 50% recovery of the former behavioral pattern. We do not know whether we had introduced a very long-term change. The reduction in chip selection was associated with an increase in selection of other starch products; thus these substitutions do not seem to be random. This study result raises the important general question whether we are conducting enough long-term research to see whether consumer trends are shortterm or longer-term. One of the current trends in sensory and consumer food research is greater interest in long-term testing.

3.2. Eating duration Recent research suggests that eating duration might be a critical variable in eating, but it has not been the subject of much research. Holm (2002) reported estimated meal durations from the Nordic study of 1200 consumers in each of four countries. The most frequent response, in ten-minute intervals of response, for all meals in all countries was 10–20 minutes, with 21–30 minutes the second most frequent for three countries. The most infrequent response in all countries was 31–40 minutes. Thus meals are shorter than many people think. Short meals also characterize eating in the Arab Gulf States (Iddison, 2002). Because of the importance placed on the presence of other people at meals, studies have documented eating durations in restaurants in the United States and correlated eating duration with the number of people present. Sommer and Steele (1997) observed eating in both American coffee shops and restaurants, and reported increased duration at the table for groups rather than individuals, and for those reading rather than non-reading. Being in a group added approximately 10 minutes to a meal, and reading added approximately 10 minutes. Bell and Pliner (2003) observed eating duration and the number of people at tables in three types of eating establishments in the US and found moderate correlations between the two measures in all restaurant types. They also documented that people eat much longer in worksite cafeterias and moderately priced restaurants than in fast foods restaurants (Table 24.5). Pliner and colleagues have also studied the impact of duration on social facilitation of eating, the phenomenon of eating more in the presence of other people. These important results are presented below under Socialization. Eating duration might be critical for both consumer enjoyment and consumer intake. Research has examined not only eating time, but also waiting time in restaurants. Edwards (1984) distinguished waiting times before the meal (pre-process), during the meal (inprocess) and after the meal (post-process). Waiting for food can affect food acceptability. Waiting produced a monotonic decline in preference for almost all foods tested. TABLE 24.5 Observational data that people eat longer when eating with others in three different food service settings

Fast food restaurant: Worksite cafeteria: Moderately priced restaurant: (Bell and Pliner, 2003.)

Group size: 1

Group size: 5ⴙ

10.7 min 12.6 min 27.6 min

21.9 min 44 min 58.5 min

568

Product Experience

3.3. Choice Choice of foods is a part of the natural eating experience. Almost all natural dining environments present the diner with a number of choices. These choices extend to the question of whether the diner should eat at all, should have a beverage, or should have a full meal or a snack. Even at home, diners can (politely) refuse certain foods and select others. The variable of choice is one of the least studied variables and it is potentially one of the most important because choice varies tremendously across eating situations. The infant has very little choice, almost none, whereas the diner in a fine restaurant has many choices. One of the main issues of the laboratory research situation is that the laboratory provides almost no choice. Subjects in laboratory studies are expected to sample the products provided to them for test. Research protocols rarely ask the subject which sample they want or whether they want to skip samples. While human use regulations require that subjects can terminate a study, in fact most subjects feel pressured to conform to what is expected of them, especially because many experimental subjects are compensated. Academic subjects are usually performing for course credit and the pressure of serving as a subject for a professor. In a demonstration of the role of choice on eating, Kramer et al. (2001) compared the traditional laboratory approach to a more natural research model with more choice. In the traditional approach, the same daily lunch meal was served for a week, and led to reduced acceptance scores and reduced intake, but not when the meal varied (Meiselman et al., 2000). This is a demonstration of the laboratory effects of monotony (reduced acceptance and intake) and variety (increased acceptance and intake). However, Kramer et al. (2001) found that this monotony effect did not occur when consumers had choice (Kramer et al., 2001). They observed that soldiers who select the same meal every day, actually rate that product higher, and this effect held for both all foods and main dishes (Tables 24.6a,b). This seems to make common sense, that a person who selects a product frequently likes it more. The laboratory phenomenon shows that when a person is pressured to eat the same thing repeatedly that their liking declines. The monotony phenomenon seen in the laboratory might not exist or rarely exist in the real world. This raises the question of how many other laboratory eating phenomena are laboratory phenomena only. In further demonstrations of the important role of choice, King et al. (2004; 2007) demonstrated the criticality of choice in two successive product development tests. In both studies, providing choice enhanced acceptance scores. In the first study, subjects were tested in a sensory laboratory where additional variables were added to determine their effects on product liking. Giving the subjects a choice of which products to sample had a bigger effect than the physical environment itself, and increased overall liking scores from 6.7 to 7.2 on the nine-point scale. King et al. (2007) replicated this effect by showing an increase in liking scores in a natural restaurant condition which included choice. In this study, subjects were tested in a laboratory, a restaurant without choice and without a real meal context, and in a restaurant with choice and meal context. Scores for the first conditions did not differ, but providing choice in the third condition increased liking scores.

3.4. Convenience While everyone accepts that convenience is one of the major trends in eating in the past decades, with convenience food products and convenience (fast) food service, relatively little research has studied what constitutes convenience to the consumer. Candel (2001) suggested that meal preparation convenience has two key dimensions: Time and effort. Thus convenience relates to two variables we have been discussing, effort and duration.

569

Experiencing food products within a physical and social context

TABLE 24.6a The frequency of main dishes served to soldiers at two field sites are shown, along with acceptance rating at first consumption occurrence and the average acceptance score for all products which were consumed more than one time Site 1

Site 2

# Of times item was consumed

Frequency

Accept rating

Frequency

Accept rating

1 2 3 4 5 6 7

879 305 73 20 2 1 1

7.25

302 100 28 5 0 2 2

6.28

2 or more

402

7.66

137

7.07

(Kramer et al., 2001.)

TABLE 24.6b The frequency of consumption of all foods served to soldiers at two field sites are shown, along with acceptance ratings at first consumption occurrence and the average of all subsequent occurrences Site 1

Site 2

# Of times item was consumed

Frequency

Accept rating

Frequency

Accept rating

1 2 3 4 5 6 7 8 9 10 10

2159 1037 538 263 140 95 60 45 39 35 187

7.41

1085 395 162 72 46 24 19 14 9 0 0

6.80

2 or more

2439

7.70

7.41

7.23

(Kramer et al., 2001.)

Candel proposed a six-item rating scale to measure convenience orientation in food preparation. Jaeger and Meiselman (2004) added that convenience be considered throughout the entire food provisioning process, including acquisition, preparation, eating and cleaning up. They studied female US consumer perceptions of convenience, time and effort using a repertory grid analysis of responses to written scenarios. The scenarios included the elements of food acquisition/shopping, preparation, and cleaning up. They confirmed the importance of time and effort in the perception of convenience, but noted that these two variables were highly interdependent. Results to be presented under eating together provide further weight to duration as an important variable in the eating experience. The importance of convenience was further demonstrated by development of the Food Choice Questionnaire (Steptoe, Pollard and Wardel, 1995) through factor analysis of

570

Product Experience

TABLE 24.7 The convenience factor on the Food Choice Questionnaire ‘It is important to me that the food I eat on a typical day’: • • • • •

Item 1. Is easy to prepare. Item 15. Can be cooked very simply. Item 28. Takes no time to prepare. Item 35. Can be bought in shops close to where I live or work. Item 11. Is easily available in shops and supermarkets. (Steptoe et al., 1995.)

questionnaires from UK respondents. The Food Choice Questionnaire contains 36 items on nine factors. Eertmans et al. (2006) studied the invariance of the Food Choice Questionnaire structure in three western countries, showing good agreement but some differences in factor structure. Sensory appeal, health, convenience, and price were identified by Steptoe et al. as the most important factors. The convenience factor dealt with the purchase and preparation of food, and contained five items (Table 24.7).

3.5. The physical environment Consideration of where people eat is a daunting task because the range of possibilities is so vast. Do most people world-wide eat at home? And if they eat at home, where do they eat within the home? Is it in the kitchen, in a dining room, in front of the television, out of doors? Making assumptions based on western culture can be very misleading. Iddison (2002) reports that in Gulf State homes there was traditionally not a separate room for eating. One of the most basic questions about where we eat is whether most meals are consumed at home. The enormous popularity of street-food vendors in many parts of the world should raise questions about this assumption. Even within western countries there are wide differences. Holm (2002) noted that relatively few Nordic meals are consumed away from home (Table 24.8). At the same time, almost 60% of Americans eat a meal away from home on any given day, and this figure is increasing. British and Swedish consumers eat out at about the same rate (about 22% weekly and about 38% monthly), lower than Americans and higher than other Nordic countries. The percentage of the American food dollar spent away from home is now near 50%. There have been a number of recent studies on the impact of the physical environment on food liking. King et al. (2004; 2007) found that enhancing the physical environment had little reliable effect on food product ratings. In their earlier paper they studied meal acceptability in a standard laboratory test. They then added a series of contextual variables leading to an environment more similar to natural eating. Serving the foods as a meal and providing a choice of which foods to eat had the greatest effects on acceptance as noted above. Adding contextual variables increased acceptance scores for salad and tea. In the second paper, testing within the actual chain restaurant setting did not affect ratings unless the food was served to regular customers who chose their food from the regular menu in a meal setting. Moving from the laboratory to the restaurant by itself did not change product experience. This shows the importance of testing various factors for effects. It also suggests that simulated dining environments might not have the same effects as real dining environments where consumers make real choices.

571

Experiencing food products within a physical and social context

TABLE 24.8 Eating alone. Average number of individual eating events in population groups and proportion of population who had no individual eating events a day Deamark

Norway

Sweden

% with

% with

% with

Mean

s.d.

0

Mean

s.d.

0

Mean

s.d.

0

(N) Total Population

(1187) 1.2

1.3

42

(1178) 1.2

1.2

34

(1244) 1.4

1.3

33

Gender

Men Women sig

1.1 1.2 NS

1.3 1.3

41 43 NS

1.3 1.2 NS

1.2 1.2

32 37 NS

1.3 1.4 NS

1.2 1.4

34 32 NS

Age

15–24 25–34 35–44 45–54 55–64 65 sig

0.9 0.9 0.9 1.1 1.3 1.7 ***

0.9 1.1 1.0 1.3 1.4 1.6

40 44 47 43 39 27 ***

1.2 1.1 1.0 1.3 1.4 1.6 ***

1.1 1.1 1.1 1.1 1.2 1.5

30 36 39 30 33 36 ***

1.3 1.2 1.1 1.2 1.4 1.8 ***

1.1 1.1 1.1 1.3 1.4 1.7

21 37 39 35 33 35 ***

Country

(Holm, 2002.)

Hersleth et al. (2003) also varied test physical environment in a study of eight Chardonnay wines differing in three product characteristics served in a sensory laboratory or reception room, with or without food. The wines were rated on a nine-point scale by 55 consumers/wine users who completed all four sessions. The reception room had groups of eight in an enhanced social setting. Context effects were as large as product effects. The presence of food and the enhanced reception room raised hedonic scores 0.3–0.5 scale points (as did the product factors, one of which reduced liking). The presence of food was a more effective enhancer in the reception room than in the laboratory. In a practical application of the use of environmental effects, Hoyer and de Graaf (2004) found that enhancing the meal environment for elderly diners, including flowers, tablecloths, acoustics, lighting, increased food intake for males (4.9 vs. 4.4 MJ) and females (4.9 vs. 4.6 MJ) as compared to a more basic environment. There has been relatively little published research on specific variables of the physical setting, especially in the all-important home setting. This could be of great importance to the food service industry and for health professionals concerned about weight control. There is a large amount of business folklore relating to the proper environment for all food service, including fast food restaurants, fine dining restaurants, and other eating settings. But very little of it appears to be based on sound data. This is in keeping with the business-orientation of the food service industry, as compared with the technology orientation of the food product industry. Bell and Meiselman (1995) summarized some of the existing information on variables affecting food choice, including variables in the physical environment. They conceptualized the food choice situation as resulting from the interaction of personal antecedents to choice (such as various attitudes and habits) with the eating environment (such as effort to obtain food). Within the food environment they considered cognitive information in the form of food packaging and food menus, and visual and auditory stimulation in the form of lighting, decoration, and music within food service environments. Bell et al. (1994) studied restaurant decor with actual manipulations of restaurant interiors, and demonstrated changes in consumer behavior.

572

Product Experience

Kimes and Robson (2004) analyzed effects of table configuration and table location in restaurants, from data from a single large chain restaurant in Phoenix USA. The restaurant seats 210 people at 65 tables seating two, four, and five people, and is aimed at younger adults and families. Their dependent variables were duration and spending. Spending variables included spending per minute and average check per person. Tables that offered more privacy had both higher eating durations and higher checks. Customers in booths, in which an upholstered bench is on each side of a table, ate longer and spent more per check but not per minute. Customers in banquette seating, in which one side of the table has an upholstered bench and other side has chairs, stayed longest and had lower spending per minute. Interestingly, customers at seven less desirable tables (next to kitchen door, tables for two between larger tables with high traffic and noise) had shorter durations and therefore higher revenues. The authors do not discuss eating duration from the viewpoint of social facilitation of eating (eating more in groups). Nor do they discuss whether higher income or higher spending customers might select or request higher spending table types. Such physical food service research could be combined with behavioral research on consumer attitudes and expectations. Several published studies have examined sound level and the presence of music on eating (Milliman, 1986; North et al., 2003). The results support the hypothesis that fast music promotes faster eating, shorter table time and lower bar purchases, but not food purchases. Slower and classical music promote greater bar purchases. Health-oriented researchers could try to reverse these studies to determine if one can reduce food intake with music.

3.6. Socialization/commensality Eating is a social activity. Most people eat meals with other people, which is the definition of commensality (Sobal, 2000). Eating alone is devalued in many cultures, and it is not clear that eating alone in the laboratory is exempt from the lower status of eating alone. The phenomenon of eating alone should receive more attention, as many believe that more people will be eating alone in the future. Sobal and Nelson (2003) conducted a cross-sectional mail survey of 663 people in one US area. Most respondents ate alone at breakfast, alone or with co-workers at lunch, and with family members at dinner. Unmarried individuals more often ate breakfast and dinner alone and more often ate lunch with friends. Thus, work-oriented society leads people to eat alone during the day and with family in the evening. People maintained commensal relationships mainly with family, which is consistent with data from the large Nordic study (Holm, 2002). Sobal and Nelson found no gender differences, and living alone was not determinative of eating alone, so lone diners are not necessarily those living alone. It was expected that living and eating alone would characterize the elderly, but this was not the case. Sobal and Nelson suggest that commensal eating is healthier because of social facilitation (prevent under-consumption, risking over-consumption), social support (healthy food choices), and social control (healthy food choices). The four-country Nordic study on eating patterns (Table 24.8) provides detailed data on commensal patterns (Holm, 2002) and confirms many of the observations of Sobal and Nelson. The study was based on 1200 surveys each in Norway, Sweden, Denmark, and Finland. About 2/3 of respondents reported eating alone at least once on the day before the survey. The proportion of people eating alone and with family members was about the same, with the latter increasing in the evening. People living alone ate alone three times more often, and older people ate alone more than younger people. The chance of eating a full (proper) lunch or dinner did not vary whether eating alone or with others. Eating with colleagues peaked at midday, during

Experiencing food products within a physical and social context

573

the typical lunch time, as reported by Sobal and Nelson. Eating with friends and others occurred on weekends and was very infrequent, also confirming Sobal and Nelson. The relationship between commensality and food intake has been the subject of a lot of research, most of it within the context of health. It was first reported by de Castro and colleagues in a series of papers based on the food diary method. De Castro and de Castro (1987) trained people for one day on how to fill out a dietary diary and then had them keep detailed records for the next week including with whom they ate. They observed that people eating alone had fewer daily meals (1.6) than people eating with others (2.1) and ate less food per meal (410 Kcal) than people eating with others (591 Kcal). This effect is called social facilitation of eating. In many replications of the same pattern, de Castro and his associates found that the amount consumed increased with the number of people present. This social facilitation of eating effect produced a large number of studies focusing on effects of other variables on intake. Variables which should increase eating also increased the number of people present (de Castro et al., 1990). For example dinner is the biggest meal and has the most people present; restaurant meals typically have more people present and are bigger, and meals with alcohol have more people present and are bigger. Social facilitation of eating was reviewed by Herman et al. (2003). Several authors have suggested that eating duration might be critical in social facilitation of eating effects including Feunekes et al. (1995). Pliner et al. (2006) recently presented the first study which independently varied eating duration and group size. They found that the increased intake was related to eating duration and not to group size. This is an important effect which needs to be replicated in a variety of eating environments; it might present an important mechanism to increase or decrease eating for reasons of health. Social facilitation of eating remains an important phenomenon, but it might work through duration, and food intake might be especially sensitive to changes in eating duration.

3.7. Service Social variables emphasize verbal communication, which is critical in actual service situations. Edwards and Meiselman (2005) tested the impact of positive and negative verbal cues in an actual restaurant setting. Restaurant customers were offered a menu with a choice of five main courses, and the waiter made a comment about a target dish. The comment was either nothing, or a positive or negative message. The positive message (‘To assist in your selection, could I just say that the “target dish” [name] has been particularly popular this week/yesterday/last week’) produced a 41% selection rate, while the negative comment (‘To assist in your selection, could I just say that the “target dish” [name] has not been particularly popular this week/yesterday/last week’) produced half the selection rate at 19%. The no comment condition yielded the same selection rate as the positive comment (40%). There were no significant differences among the acceptance ratings of the products in the different conditions. Thus, once a choice had been made the positive and negative comments had no effect on the experience of the product.

4. DESIGNING PRODUCTS FOR A MEAL CONTEXT Those interested in environmental/contextual effects on food choice, acceptance, and consumption inevitably find that they must address food as meals. The natural context for eating is the meal because most food, as much as 70–80%, is consumed as a part of a meal (Kjaernes, 2002). Having said that, it is surprising that most food research

574

Product Experience

and most consumer research does not take place in a meal setting. The major point of this section of this Chapter is to emphasize the need to conduct food research as meal research. This has at least two outcomes: First, it places foods within their natural context. But it also helps to camouflage the food item of interest within the other foods of the meal. For example, if the dessert is the object of interest, then the meal puts emphasis on the main dish, the vegetables, bread, and other meal components, as well as the dessert. Food products designed in isolation raise the risk that the meal experience will result in a different food experience than the food sampled by itself. Research bears this out, as you will see below. The decision to consider meals in research is a major one, because meals are highly complex events comprising many dimensions including at least dietetic, culinary, sensory, social, nutritional, anthropological, psychological, cultural, health, temporal, economic, and others. In fact the more one looks into what is involved in meals, the more complex this unit of eating is likely to appear. Several recent books help to organize this material (Meiselman, 2000; Walker, 2002). There are two other good sources of research on meals in English, both within specific cultures. There is a tradition in England of studying meals, beginning with Mary Douglas in the 1970s (Douglas, 1976; Douglas and Nicod, 1974), and carrying on to Anne Murcott in the 1980s. Other British investigators have built on these traditions, for example Marshall (1995; 2000). Douglas presented a framework for studying meals within the social context, which remains a major influence to this day. Douglas emphasized that meals are highly structured events following a series of rules about where, when, and in what sequence foods could be served. Investigators who arbitrarily design laboratory meals often violate these rules. A second source of material is the major study of eating and meals in the four Nordic countries of Denmark, Finland, Norway and Sweden. These countries participated in a survey of 1200 people in each of the four countries. Many papers from this research were published in the local language of those countries, but a book in English summarizes the work (Kjaernes, 2002). And what constitutes a meal is constantly evolving and changing. This is difficult for most students, laypeople, and researchers to grasp. What constitutes a meal changes at least once or twice a century, and probably more often, in our rapidly evolving cultures. Well into the 1900s most western countries ate three hot meals per day (working people often went home to eat); now some countries eat one hot meal while other countries continue with three. In the nineteenth century, many working people ate up to five or more times per day a combination of meals and large snacks. And different classes of society ate with different frequency, the upper classes eating less often. Even within the Nordic countries the meal pattern of which meals are hot and which meal is the major meal varies from country to country. What constitutes a meal has changed as well, as will be noted below. These changing trends in eating are very important to understanding health issues in eating or product development needs of the future. The meal most changed from its traditional form is probably breakfast, and as Sobal and Nelson (2003) have pointed out: ‘American breakfast is the most anomalous meal, more often small, short, skipped, and involving special foods’. Historically, breakfast was not always part of the daily eating routine (Albala, 2002). This is an important practical issue for those seeking to develop breakfast foods for the future – will breakfast exist, or will it continue to decline into its old status as a meal for laborers, children, and the elderly? The mean number of eating events in the Nordic survey is 3.9 eating events per day, with slightly less (3.7) in Denmark. Almost everyone (over 90%) eats at least three times

575

Experiencing food products within a physical and social context

per day. Denmark and Norway eat more cold meals, and Sweden and Finland eat more hot meals. Is the meal pattern becoming more irregular? Are we becoming grazers? The Nordic study does not conclude that grazing typifies modern Nordic eating, either based on meal pattern (Gronow and Jaaskelainen, 2002) or based on what is consumed at those meals (Makela, 2002). In a study of meal food combinations, Marshall and Bell (2003) asked students in both Scotland and Australia to provide hedonic ratings, frequency of use ratings, and appropriateness ratings for 51 common food names, and to then construct snack, lunch, and dinner meals from the food names for 11 different physical locations. Through cluster analysis they identified six different meal types: Main meal; light meal; fast food; snack; camping trip; and seafood snack. They emphasized that some foods are associated with specific meal types (hamburgers in fast food), and many food items belong in different meal types and in different locations (pasta). Fast food items fell into a separate cluster. The main meals for both lunch and dinner were similar to the British ‘proper meal’ with a meat, vegetable, and starch. Light meals contained the types of main dishes (pizza, pasta, sausage) not usually associated with main meals. Pizza fit into fast food, light meal, and snack, depending on country and meal group. The effect of location on food choices was more important at lunch than dinner. Another way of looking at food patterns within meals is to examine what foods are consumed at which meals. The Nordic study presents detailed data on what people eat at meal-times. Every meal pattern contains a main dish (‘center’) by definition, and almost all (95%) meals contain a beverage. Other meal components range from

TABLE 24.9

Hot meal combinations on weekdays. Percent of all hot meals

Denmark

Finland

Norway

Sweden

Combination Weekdays

Sundays Weekdays

Sundays Weekdays Sundays

Weekdays

Sundays

C CS CV CT CB CSV* CVT CST CSB CVB* CTB CSVT* CVTB* CSTB CSVB* CSVTB* N (meals)

11 15 7 6 6 17 3 13 2 3 1 13 2 0 1 0 239

12 8 6 7 10 19 2 5 6 2 2 8 1 1 8 2 260

8 14 6 8 7 15 2 9 1 1 4 15 2 2 2 4 1240

6 11 5 7 8 17 1 11 0 3 2 19 2 2 2 7 199

10 15 8 8 5 18 2 15 1 2 1 14 1 0 0 0 700

13 12 9 7 15 13 1 4 3 5 3 5 2 2 5 3 1388

15 11 6 8 6 27 2 7 0 1 1 13 1 0 0 0 982

9 7 8 8 4 29 1 9 0 1 3 19 0 0 0 0 224

C center (meat, fish, vegetables, other); S staple (potatoes, rice, pasta, pulses); B bread. V vegetables (as a side dish); T trimmings (sauces, preserves, condiments). * proper meals. Every meal has a main dish (by definition), and most meals have a beverage (Makela, 2002).

576

Product Experience

10–65% present in meals (Table 24.9). Hot meal patterns vary considerably, with the pattern of main dish/starch/vegetable with or without sauces being the most common. This is similar to the British proper meal studied by Douglas and others. Considering all meals, most meals have two or three components, with one and four component meals about equal in frequency. People have not only studied what we eat at meals, but how those meal components contribute to the overall meal experience. The earlier work on food acceptability within meals produced a clear pattern of data, showing that the main dish within a meal contributes the largest portion to overall meal acceptability (Rogozenski and Moskowitz, 1982; Turner and Collison, 1988). This was assessed by asking respondents to rate individual food item acceptability and also overall meal acceptability. Respondents did not actually see or consume any food. Regression analysis on the individual foods and overall meals produced the equations relating overall liking to meal component liking. Instead of modeling questionnaire meals, Hedderley and Meiselman (1995) modeled actual University cafeteria meals (n 309) freely selected by University students in the UK. They found that the main dish accounted for varying amounts of overall meal acceptability depending on the type of meal and the type of main dish. Traditional meals with a main dish (n 175) were the most frequent and the main dish accounted for 60% of overall meal acceptance. Sandwich meals (n 82) and pizza meals (n 52) were less frequent, and the main dish accounted for relatively more of the overall meal’s acceptability, 70% and 90% respectively. For a pizza meal, the meal’s acceptability is largely determined by the pizza itself. Those using pizza as a food in research should realize that it might function differently as a meal component than other main dishes. What will meals be like in the future? Are meals converging globally? In my experience, no question excites a food lecture audience more than this question. People are passionate whether change lies ahead for foods and for eating, and that globalization will occur for meals as it has in technology. Others are just as passionate on the need for traditionalism in foods and eating. The 2004 meeting of the European Sensory Network in Florence, Italy focused on regional products (‘typical products’), emphasizing both the cultural and economic aspects of the issue (Monteleone and Bertuccioli, 2006). The fact that some products have become global (for example, hamburger, cola, and pizza) does not mean that many others will follow. Perhaps a small number of foods will become global foods, while many foods will stay regional. This is an important issue for product developers, food service managers, and health officials. The food choice field has yet to undertake major studies of the trend toward globalization in food choices.

5. HOW TO TEST PRODUCTS TO REFLECT CONSUMER PRODUCT EXPERIENCE Most research on food choice can be conducted under laboratory control, in a naturalistic setting, or in a completely natural environment. While there is an increase in the amount of food choice research on the effects of the environment, the vast majority of research continues to be conducted under controlled conditions such as exist in the laboratory (Meiselman, 1992). The choice of environment in which to conduct research is perhaps more critical when the subject of the research is the environment itself. Studies have begun to determine which variables differentiate laboratory and natural settings and which of those variables determine any differences in outcomes between laboratory and natural or naturalistic studies. This is clearly a long-range issue to determine these critical variables, but early studies reported above and summarized below suggest that meal context and having a choice about what to eat might be two of the critical variables.

Experiencing food products within a physical and social context

577

In the future, with increased knowledge of what are the critical variables, modifications of laboratory methods might be possible to improve prediction, and modifications of natural environments might be possible to increase the number of study designs. Variables which contribute to environmental effects can be studied in both controlled settings such as the laboratory and in natural settings. The choice of setting is based on both practical and methodological considerations. Some have suggested that natural settings are more expensive and more difficult to manage. This is not always the case; in most natural studies the environment already exists and therefore nothing has to be designed and built. A test product (such as a new food) or a test procedure (such as price change) can simply be added into the existing environment and the effect noted. The dependent variable or measure might be the number of food portions purchased or consumed, or the waste of the test food left on the plates. Studies in natural settings have the advantage of higher external validity. Natural studies can be inexpensive to conduct, but they usually lack the control which is present in the laboratory. The lack of trained research personnel in natural settings and the need to perform the main job of the natural setting, such as feeding people in a restaurant, preclude complete control. This lack of control can be disturbing to laboratory-trained researchers, who value control over realism. Both the advantage and the downside of natural settings is the lack of control. It is the lack of control in nature which we are seeking in natural settings, but the lack of control also means that things happen which we cannot control. A study might continue for one week, and during that week the weather might change, the air conditioning might break, and the serving personnel might change. Do natural studies or laboratory studies produce higher external validity, the extent to which results relate to the real world? Schutz (1988) presented a model of external validity of eating research containing the variables of type of subjects, type of stimuli, and measurement procedure. More recently, Van Trijp and Schifferstein (1995) also focus on type of respondents, type of stimuli, and scaling procedures as determinants of external validity. But they also consider test circumstances, including the context or location of research. Most current research on foods takes place at low levels of external validity, using expert or recruited laboratory subjects, simple substances and foods, hedonic/preference judgments, and laboratory settings. Van Trijp and Schifferstein point out that the traditional model of food sensory research emphasizes product attributes and sensory characteristics, producing higher internal validity but lower external validity. In this type of work, the focus is on the product and the sponsor of the research is often a food company or food department in a university. Natural subjects (consumers) and more natural stimuli produce more externally valid results. In this type of work, the focus is on the consumer. Concerning the measurement procedure, Schutz suggests the measurement of use intentions, such as appropriateness, rather than affective/liking measures. One advantage of doing research in more natural locations is that the natural subjects and natural stimuli often come with the location – natural eating locations usually have real customers and real food, thus solving all three challenges. In other words, when one uses a natural location, one does not have to recruit subjects because the location already has customers. And one does not design a setting, because it is already there. In many ways, natural research is more economical and efficient. Some people might argue that the lack of control in natural settings means that only observational studies are appropriate as opposed to manipulated studies. This is not the case. Both types of research can be conducted in natural settings, observation of an eating environment with no interference other than the presence of the observer (for example, Sommer and Steele, 1997; Bell and Pliner, 2003), or manipulated variables

578

Product Experience

with clear independent and dependent variables and hypotheses of outcomes (for example, Meiselman et al., 1994). Since manipulated natural studies might involve manipulation of the environment, it can be debated whether these changes eliminate the claim of the studies as natural. But that claim stems from the fact that these are natural eating locations in which people eat regularly. Those favoring laboratory research have not adequately addressed the non-representative, recruited samples which are used, and on which generalizations to the whole population are attempted. This seems especially true in academic settings where student samples are the norm. While natural settings are suitable for both observational and manipulated studies, some studies are better conducted in the laboratory. When the task is purely sensory, such as formulating a blend of ingredients, or selecting a substitute for an ingredient, or describing a product in agreed upon terms, then a laboratory with its controls is appropriate for measuring the sensory experience of a product. Whenever one seeks to measure liking or actual consumption/intake, then one should at least consider a natural setting for this consumer experience. It is an empirical question whether controlled research settings or natural settings better predict the real world. At this point, one can certainly conclude that overall product experience is affected by where this experience takes place, and that proper inclusion of environmental variables is a requirement for state of the art product experience testing.

REFERENCES Albala, K. (2002). Hunting for breakfast in medieval and early modern Europe. In: H. Walker (Ed.) The meal, pp. 113–122. Totnes: Prospect Books. Bell, R. and Marshall, D. W. (2003). The construct of food involvement in behavioral research: scale development and validation. Appetite, 40, 235–244. Bell, R. and Meiselman, H. L. (1995). The role of eating environment environments in determining food choice. In: D. Marshall (Ed.) Food Choice and the Consumer, pp. 292–310. Glasgow: Blackie Academic and Professional. Bell, R. and Pliner, P. L. (2003). Time to eat: the relationship between the number of people eating and meal duration in three lunch settings. Appetite, 41, 215–218. Bell, R., Meiselman, H. L., Pierson, B. and Reeve, W. (1994). The effects of adding an Italian theme to a restaurant on the perceived ethnicity, acceptability and selection of foods by British consumers. Appetite, 22, 11–24. Boutrolle, I., Arranz, D., Rogeaux, M. and Delarue, J. (2005). Comparing central location test and home use test results: application of a new criterion. Food Quality and Preference, 16, 704–713. Boutrolle, I., Delarue, J., Arranz, D., Rogeaux, M. and Koster, E. P. (2007). Central Location Test vs. Home Use Test: contrasting results depending on product type. Food Quality and Preference, 18, 490–499. Candel, M. J. J. M. (2001). Consumers’ convenience orientation towards meal preparation: conceptualization and measurement. Appetite, 36, 15–28. Cardello, A. V. (1994). Consumer expectations and their role in food acceptance. In: H. J. MacFie and D. M. H. Thomson (Eds.) Measurement of food preferences, pp. 253–297. London: Blackie Academic. Cardello, A. V., Bell, R. and Kramer, F. M. (1996). Attitudes of consumers toward military and other institutional foods. Food Quality and Preference, 7, 7–20. Coll, M., Myers, A. and Stunkard, A. J. (1979). Obesity and food choices in public places. Archives of General Psychiatry, 36, 795–797. de Castro, J. M. and deCastro, E. S. (1987). Spontaneous meal patterns of humans: influence of the presence of other people. American Journal of Clinical Nutrition, 50, 237–247. de Castro, J. M., Brewer, E. M., Elmore, D. K. and Orozoco, S. (1990). Social facilitation of the spontaneous meal size of humans occurs regardless of time, place, alcohol and snacks. Appetite, 15, 89–101. de Graaf, C., Kramer, M. F., Meiselman, H. L., et al. (2005). Food acceptability in field studies with US army men and women: Relationship with food intake and food choice after repeated exposures. Appetite, 44, 23–31. Douglas, M. (1976). Culture and food. Russell Sage Foundation Annual Report 1976–77, pp. 51–58. Reprinted in M. Freilich (Ed.) (1983). The pleasures of anthropology, pp. 74–101. New York: Mentor Books.

Experiencing food products within a physical and social context

579

Douglas, M. and Nicod, M. (1974). Taking the biscuit: The structure of British meals. New Society, 19, 744–747. Edwards, J. S. A. (1984). The effects of queuing on food preferences. International Journal of Hospitality Management, 3(2), 83–85. Edwards, J. S. A. and Meiselman, H. L. (2005). The influence of positive and negative cues on restaurant choice and food acceptance. International Journal of Contemporary Hospitality Management, 17, 332–344. Edwards, J. S. A., Meiselman, H. L., Edwards, A. and Lesher, L. (2003). The influence of eating location on the acceptability of identically prepared foods. Food Quality and Preference, 14, 647–652. Eertmans, A., Victoir, A., Notelaers, G., Vansant, G. and Van den Bergh, O. (2006). The Food Choice Questionnaire: Factorial invariant over western urban populations? Food Quality and Preference, 17, 344–352. Feunekes, G. I., de Graaf, J. C. and Van Staveren, W. A. (1995). Social facilitation of food intake is mediated by meal duration. Physiology and Behavior, 58(3), 551–558. Gronow, J. and Jaaskelainen, A. (2002). The daily rhythm of eating. In: U. Kjaernes (Ed.) Eating patterns: A day in the lives of Nordic peoples, pp. 91–124. Report No. 7-2001. Lysaker Norway: National Institute for Consumer Research, Hedderley, D. and Meiselman, H. L. (1995). Modeling meal acceptability in a free choice environment. Food Quality and Preference, 6, 15–26. Herman, C. P., Roth, D. and Polivy, J. (2003). Effects of the presence of others on food intake: a normative interpretation. Psychological Bulletin, 129, 873–886. Hersleth, M., Mevik, B.-H., Naes, T. and Guinard, J.-X. (2003). Effect of contextual variables on liking for wine – use of robust design methodology. Food Quality and Preference, 14, 615–622. Hirsch, E., Kramer, F. M. and Meiselman, H. L. (2005). Effects of food attributes and feeding environment on acceptance, consumption and body weight: lessons learned in a twenty year program of military ration research (Part 2). Appetite, 44, 33–45. Holm, L. (2002). The social context of eating. In: U. Kjaernes (Ed.) Eating patterns: A day in the lives of Nordic peoples, pp. 159–198. Report No. 7–2001. Lysaker Norway: National Institute for Consumer Research. Hoyer, S. and de Graaf, C. (2004). Workshop Summary: How do age-related changes in sensory physiology influence food liking and food intake. Influence of context and environment. Food Quality and Preference, 15, 910–911. Iddison, P. (2002). Perpetual picnics – The meal in the UAE. In: H. Walker (Ed.) The meal, pp. 113–122. Totnes, Devon: Prospect Books, Jaeger, S. R. and Meiselman, H. L. (2004). Perceptions of meal convenience: the case of at-home evening meals. Appetite, 42(3), 317–325. Kimes, S. E. and Robson, S. K. A. (2004). The impact of restaurant table characteristics on meal duration and spending. Cornell Hotel and Restaurant Administration Quarterly, Nov, 332–346. King, S. C., Weber, A. J., Meiselman, H. L., Lv, N. (2004). The effect of meal situation, social interaction, physical environment and choice on food acceptability. Food Quality and Preference, 15, 645–654. King, S. C., Meiselman, H. L., Hottenstein, A. W., Work, T. M. and Cronk, V. (2007). The effect of contextual variables on food acceptability: A confirmatory study. Food Quality and Preference, 18, 58–65. Kjaernes, U. (Ed.) (2002). Eating patterns: A day in the lives of Nordic peoples. Report No. 7-2001. Lysaker, Norway: National Institute. Kramer, F. M., Lesher, L. L. and Meiselman, H. L. (2001). Monotony and choice: repeated serving of the same item to soldiers under field conditions. Appetite, 36, 239–240. Lahteenmaki, L. and Tuorila, H. (1995). Consistency of liking and appropriateness ratings and their relation to consumption in a product test of ice cream. Appetite, 25, 189–198. Makela, J. (2002). The meal forma. In: U. Kjaernes (Ed.) Eating patterns: A day in the lives of Nordic peoples, pp. 125–158. Report No. 7-2001. Lysaker, Norway: National Institute for Consumer Research. Marriott, B. M. (Ed.) (1995). Not eating enough. Washington: National Academy Press. Marshall D. W. (Ed.) (1995). Food choice and the consumer. Glasgow: Blackie Academic and Professional. Marshall, D. W. (2000). British meals and food choice. In: H. L. Meiselman (Ed.) Dimensions of the meal, pp. 202–220. Gaithersburg: Aspen. Marshall, D. and Bell, R. (2003). Meal construction: exploring the relationship between eating occasion and location. Food Quality and Preference, 14, 53–64. Meiselman, H. L. (1992a). Methodology and theory in human eating research. Appetite, 19, 49–55. Meiselman, H. L. (1992b). Obstacles to studying real people eating real meals in real situations: Reply to Commentaries. Appetite, 19, 84–86. Meiselman, H. L. (1996). The contextual basis for food acceptance, food choice, and food consumption: The food, the situation and the individual. In: H. L. Meiselman and H. J. H. MacFie (Eds.) Food choice acceptance and consumption, pp. 239–263. Glasgow: Blackie Academic and Professional.

580

Product Experience

Meiselman, H. L. (Ed.) (2000). Dimensions of the meal. Gaithersburg: Aspen. Meiselman, H. L. (2004). Pangborn workshop summary. What to eat: A multi-discipline view of meals. Food Quality and Preference, 15, 901–905. Meiselman, H. L., de Graaf, C. and Lesher, L. (2000). The effects of variety and monotony on food acceptance and intake at a mid-day meal. Physiology and Behavior, 70, 119–125. Meiselman, H. L., Hedderley, D., Staddon, S. L., Pierson, B. J. and Symonds, C. R. (1994). Effect of effort on meal selection and meal acceptability in a student cafeteria. Appetite, 23, 43–55. Meiselman, H. L., Hirsch, E. S. and Popper, R. D. (1988). Sensory hedonic and situational factors in food acceptance. In: D. M. H. Thomson (Ed.) Food acceptability, pp. 77–88. London: Elsevier Applied Science. Meiselman, H. L., King, S. and Hottenstein, A. W. (2004). Laboratory product testing produces an underestimation of true product acceptability. Paper presented at A Sense of Identity, Florence, Italy. Meiselman, H. L., Johnson, J. L., Reeve, W. and Crouch, J. E. (2000). Demonstrations of the influence of the eating environment on food acceptance. Appetite, 35, 231–237. Milliman, R. E. (1986). The influence of background music on the behavior of restaurant patrons. Journal of Consumer Research, 13, 286–289. Monteleone, E. and Bertuccioli, M. (2006). (Eds.) The First European Conference on Sensory Science of Food and Beverages. Food Quality and Preference, 17(1, 2), 1–144. Myers, A., Stunkard, A. and Coll, M. (1980). Food accessibility and food choice. Archives of General Psychiatry, 37, 1133–1135. North, A. C., Shilcock, A. and Hargreaves, D. J. (2003). The effect of musical style on restaurant customers’ spending. Environment and Behavior, 35(5), 712–718. Pliner, P., Bell, R., Hirsch, E. S. and Kinchla, M. (2006). Meal duration mediates the effect of social facilitation on eating in humans. Appetite, 46, 189–198. Rogozenski, J. G. Jr and Moskowitz, H. R. (1982). A system for the preference evaluation of cyclic menus. Journal of Food Service Systems, 2, 139–161. Rotenberg, R. (1981). The impact of industrialization on meal patterns in Vienna, Austria. Ecology of Food and Nutrition, 11, 25–35. Rozin, P. and Tuorila, H. (1993). Simultaneous and temporal contextual influences on food acceptance. Food Quality and Preference, 4, 11–20. Schutz, H. G. (1988). Beyond preference: Appropriateness as a measure of contextual acceptance of foods. In: D. M. H. Thomson (Ed.) Food acceptability, pp. 115–134. London: Elsevier Applied Science. Sobal, J. (2000). Sociability and meals: Facilitation, commensality and interaction. In: H. L. Meiselman (Ed.) Dimensions of the meal, pp. 119–133. Gaithersburg: Aspen. Sobal, J. and Nelson, M. K. (2003). Commensal eating patterns: A community study. Appetite, 41, 181–190. Sommer, R. and Steele, J. (1997). Social effects on duration in restaurants. Appetite, 29, 25–30. Steptoe, A., Pollard, T. M. and Wardel, J. (1995). Development of a measure of the motives underlying the selection of food: the Food Choice Questionnaire. Appetite, 25, 267–284. Stroebele, N. and de Castro, J. M. (2004). Effect of ambience on food intake and food choice. Nutrition, 20, 821–838. Stunkard, A. J. and Kaplan, D. (1977). Eating in public places: a review of reports of the direct observation of eating behavior. International Journal of Obesity 1, 89–101. Stunkard, A. J. and Messick, S. (1985). The three factor eating questionnaire to measure dietary restraint, disinhibition and hunger. Journal of Psychosomatic Research, 29, 71–84. Thomson, D. M. H. (Ed.) (1988). Food acceptability. London: Elsevier Applied Science. Turner, M. and Collison, R. (1988). Consumer acceptance of meals and meal components. Food Quality and Preference, 1, 21–24. Van Trijp, H. C. M. and Schifferstein, H. N. J. (1995). Sensory analysis in marketing practice: Comparison and integration. Journal of Sensory Studies, 10, 127–147. Walker, H. (Ed.) (2002). The meal. Totnes, Devon: Prospect Books. Wansink, B. (2004). Environmental factors that increase the food intake and consumption volume of unknowing consumers. Annual Reviews in Nutrition, 24, 455–479.

25

THE MEDIATING EFFECTS OF THE APPEARANCE OF NONDURABLE CONSUMER GOODS AND THEIR PACKAGING ON CONSUMER BEHAVIOR LAWRENCE L. GARBER, JR. Elon University, Elon, NC

EVA M. HYATT Appalachian State University, Boone, NC

ÜNAL Ö. BOYA Appalachian State University, Boone, NC

Vision has primacy in our sensory world (Schifferstein, 2006) such that information to our brains mediated by the visual sense comes to have a particularly powerful impact on, for example, consumers’ experience of nondurables. In a retail frame, this means that visual information and processes play a key role in the impact of nondurable products and packages on the consumer at the point of purchase. Moreover, the nature of typical food and convenience store layouts, and how consumers are caused to move through and shop them, elevates the impact of visuals by dictating to consumers the physical spaces they must cross, the paths they must follow, and the spaces they must occupy in order to browse product categories and consider brands for purchase. For example, consumers moving down long store aisles first see category facings from a distance and at an angle, and therefore begin processing the grosser elements of appearance well before they can process finer details or read text (the latter also being visual processes). These first visual impressions may therefore condition the product or package information that can only be processed later, and may predispose the consumer towards certain ultimate actions such as consideration and choice (Garber, Burke and Jones, 2000). Product Experience Copyright © 2008 Elsevier Ltd.

581

582

Product Experience

Even though product or package appearance has long been understood to mediate such consumer information processes as attention, comprehension, and consideration, as well as actions such as purchase, consumption, and disposal of nondurable goods, surprisingly little research in marketing or related disciplines has been done to understand how appearance, or those various visual elements that go to make up total appearance, has its effect on consumer behavior. Such limited research on a clearly important factor in consumer behavior has two reasons. First, marketers are by and large untrained visually, even though. They are called upon to make product and package selections for appearance surprisingly often. Standard business curricula simply do not typically include courses in art and design. This gap is becoming all the more glaring as new electronic media move us toward an evermore visual world and marketplace. Secondly, artists and designers, being nonscientists, have formed and tend to rely on various rules of thumb relating to visual responses that, though they may largely be correct, have not been scientifically tested; nor have their behavioral underpinnings been investigated. In this chapter, we review the literature on product and package appearance, looking specifically at the effects of the main visual elements that comprise appearance – color, shape, and size (Treisman and Gelade, 1980; also see elsewhere in book where research on proportion is discussed). Then we discuss those many important aspects of the effects of product and package appearance, which forms a rich field for future visual researchers. And, given that much research interest may have been retarded by the fact that many marketing researchers simply do not know how to approach experimental visual problems, we propose a visual research methodology that may have general application to many visual problems.

1. APPEARANCE AS A CARRIER OF BRAND EQUITY Appearance and its several visual components (color, size, shape, and motion) are what consumers must perceive and integrate in order to recognize and interpret a commercial

FIGURE 25.1 Coke’s visual package conventions.

The mediating effects of the appearance of nondurable consumer goods

583

object, e.g. a product, package and/or store display, in their visual fields (Treisman and Gelade, 1980; Treisman, 1991). Appearance as a mediator of the purchase process is designed by marketers to be a visual stimulus, or set of visual stimuli, that are vivid, affect-laden and memorable (Cheskin, 1957), giving it great power as a promotional tool (Francis, 1977), and therefore as a contributor to brand equity. For example, The CocaCola Company has worked very hard to establish visual conventions around the world that most all consumers associate with this cola brand, including Coke red, the signature curvilinear bottle, the wave graphic, and the familiar cursive font, as can be seen in Figure 25.1. Not only do these various visual elements, singly and in combination, identify the Coke brand, but they also convey meaning to the brand, in the form of positive associations, including symbolic meanings, metaphors and stories, such as what is connoted when we watch polar bears on television drinking Coke in the snow at Christmas time.

1.1. Four roles for package appearance By extension, brand equity theory suggests there are at least four roles for appearance as a carrier of brand equity in the store (Garber, Burke and Jones, 2000). As shown in Figure 25.2, these would include: (1) identifying the category to which the product belongs (e.g. white paper bags for flour); (2) identifying the brand (e.g. the barbellshaped Listerine bottle); (3) conferring meaning to the brand or reinforcing or heightening existing meanings and associations (e.g. the festive, carnival-like yellow and red of McDonald’s that connotes fun particularly to a child’s sensibilities, or the small size of caviar jars or expensive perfumes, meant to convey preciousness); and (4) providing novelty or contrast to make the brand more distinctive in and of itself, or more eye-catching and salient with respect to its competitors (e.g. Obitz soft drinks, with gelatin balls floating in liquid, which provide a novel mouth feel as well as visual experience!). Any given product or package design must fulfill all four functions simultaneously. The package represents the product to all consumer segments, serving the needs of disparate groups and serving disparate purposes across all stages of the choice process. But some of these functions may be in conflict with others. For example, a package shape, size or color that is highly associated with a category may do a good job of denoting a new product as a category member, but may not allow it to stand out sufficiently to get notice or purchase consideration. When designing a new package, a manufacturer can borrow on the visual conventions established by existing brands in the category that connote typicality and familiarity. For example, a new dishwashing liquid may use the color green, similar to Palmolive, to communicate gentleness. This approach has the virtue of reassuring the shopper by fulfilling expectations of what a brand in the category should look like, thus providing a measure of legitimacy and credibility (Dichter, 1975). Consistent with this, Loken and Ward (1990) report that consumers prefer products which tend to match their expectations. Conversely, package novelty and contrast refer to the package’s ability to stand out visually from its surroundings, and to draw attention to itself through its novel appearance, as discussed further in Section 2.2.

1.2. The role of color Considerable empirical research into color response has been done in the last 100 years (for reviews, see Ball, 1965; Whitfield and Wiltshire, 1990). Primary findings show a general preference for short wavelength colors (blue, green), which people find quiet and serene, over long wavelength colors (red, orange), which people find arousing and hot. Other research shows some response differences between groups – primarily in degree of

584

Product Experience

FIGURE 25.2 Four roles for package appearance.

response, but with similar overall patterns (Adams and Osgood, 1973; Lee and Barnes, 1990; Aaronson, 1970; Golden, 1974; Bjerstedt, 1960; Fisher, 1974). For example, Coca-Cola red may be perceived as such by Coke loyals and Pepsi loyals alike (indicating a homogeneity of perception across these groups), but may evoke a very different feeling, stronger for Coke loyals than Pepsi loyals (indicating a heterogeneity of preferences across groups). However, these findings on color are too broad and simple to be of much value in a marketing context. For example, people may say they prefer blue to red, all things being equal, but this does not explain the successful use of red by organizations and brands like Coca Cola, McDonald’s, Campbell’s, Colgate, the Cincinnati Reds, Kentucky Fried Chicken, Harvard, Marlboro, and Big Red chewing gum. Color preference clearly depends upon the type of object or product on which the color is placed

The mediating effects of the appearance of nondurable consumer goods

585

(Holmes and Buchanan, 1984), and also by circumstances and situation of the viewer. For example, concerning the latter, Kearney (1966) found that cooler hues (e.g. blues and greens) were preferred in warmer ambient temperature conditions, and warmer hues (e.g. reds and oranges) were preferred in cooler ambient temperature conditions. The short answer to this apparent contradiction is, of course, that it is often constructive marketing practice to divert and arouse the consumer. Not everything can be blue.

2. THE RELATIVITY OF VISUAL PHENOMENA As much as appearance is a powerful and salient promotional tool, it is also a complex, multidimensional phenomenon not well understood in general, much less for application to marketing purposes, making consumer response to appearance notoriously hard to predict (Sharpe, 1975). Appearance mediates consumer behavior in response to advertising, at the point of purchase, and during product use. Marketers intuitively understand that appearance should enhance the appeal of and satisfaction with products, but little theory exists. Classical studies on visual perception are not useful to marketers, because they provide few guidelines to assist the marketing manager with the design selection problem. (Bruce and Green, 1990; Crick, 1994; Hilbert, 1987; Sacks, 1995; Swirnoff, 1989).

2.1. The relativity of color Complicating matters is the fact that appearance, and in particular its visual components, are highly interactive, relative phenomena dependent for their effects on the entire visual field in which they are perceived. We use color to illustrate these points, drawn largely from a discussion by Garber and Hyatt (2000). Land (1977), for instance, demonstrated that color determination depends, ‘not … solely on the wavelengths entering the eye from that patch, but also on the wavelengths entering from the other regions of the visual field’ (Crick, 1994, p. 53). In particular, color has been shown to depend for its effect on an interaction with adjacent colors (Albers, 1963; Cheskin, 1957; Swirnoff, 1989). For example, red is made to look redder when it is surrounded by green, its complement, as when a red Lava Soap pack sits next to a green pack of Irish Spring. And red appears less salient when surrounded by red, its analog, as in a ketchup shelf facing in a grocery store display. Moreover, color’s effect is highly interactive with the other visual features of which an object is composed, all of which must be integrated before the total effect can be recognized (Crick, 1994; Davidoff, 1991; Treisman, 1991; Bruce and Green, 1990; Marr, 1982; Treisman and Gelade, 1980). An example would be the failed Crystal Pepsi, whose transparency caused the bottle form to appear lighter in weight and created different flavor expectations, whereas regular Pepsi, with its opaque dark color, appears heavier and denser than its erstwhile counterpart (Garber, Burke and Jones, 2000). Indeed, there are those who argue that color cannot be perceived and understood independently of form (Collinson, 1992, p. 145). Color meaning is also relative to context. For example, Hine (1996, p. 221) reports that a 1987 study showed Americans to believe that red means love, safety, danger, strength, and warmth, but when asked to think about products, they state that it means Coca-Cola. In addition, there are cultural dimensions to color and its meaning. Hine (1996) describes the cultural dimension as visual conventions that have built up over time in respective societies. The usual example of differences in the symbolic meaning of color across cultures is that black is the color of death in Western societies, while it

586

Product Experience

is white in many Asian countries. And, in Japan, brighter colors are reserved for packages representing foreign products whose people they consider to be forthright in nature, and the more subtle soft gray hues are reserved for their own products. The meaning of color is also highly situational, changing over time, as in fads and fashion (Sharpe, 1975; Danger, 1969), and depends upon the subject category in whose context it is considered (Bruce and Green, 1990, p. 190; Marr and Nishihara, 1978). And finally, color, along with visual perception in general, is known to interact with the other senses, in that visual color sensation may make an impression in another sense altogether (Ball, 1965; Nelson and Hitchon, 1995; Sharpe, 1975; Schifferstein and Tanudjaja, 2004). Therefore, the effect that a color has on a person may be involuntarily physically experienced as temperature (red is hot, blue is cool), weight (dark colors are heavy, light colors are light), sound (loud, soft) or smell (fresh).

2.2. The relativity of visual novelty In visual marketing terms, novelty may be embodied in a surprising change to the familiar appearance of an existing product or package, or in the unconventional appearance of an altogether new product. It is a function of both a package’s distinctiveness relative to the other brands on the store shelf (Veryzer and Hutchinson, 1998) – as shown in Figure 25.3, where Palmolive Lemon-Lime dishwashing liquid appears darker and greener in the company of lemon liquids, above, and lighter and ‘lemonier’ in the company of mild green liquids, below, – and its departure from consumer expectations based on past shopping and consumption experiences. Novelty and contrast are defined in relative rather than absolute terms. The novelty of a package relative to consumers’ expectations and its contrast relative to the competitive context will increase the likelihood that the package will evoke an involuntary attentional response (Kahneman, 1973). There is also evidence in the empirical aesthetics literature (Berlyne, 1974), the attention literature (Kahneman, 1973), and the psychology of visual perception literature (Bruce and Green, 1990), that a positive relationship between novelty and preference exists (Hekkert, Snelder and van Wieringen, 2003). Schema theory suggests that consumers prefer moderate levels of incongruity (Mandler, 1982; Meyers-Levy and Tybout, 1989), which can be created by new or different packaging. For example, Heinz Ketchup was introduced several years ago in a green form, packaged in an eye-catching green bottle, nominally to appeal to the novelty-seeking tendencies of children, but was far too unpalatably incongruous to appeal to the more conventional sensibilities of adults. Garber, Burke and Jones (2000) demonstrate that novel color draws the consumer’s eye precisely because it is unexpected. However, they also show that that novel color, as eye-catching as it may be, will not affect purchase consideration unless it is also consistent in meaning with the product category, connoting an effective product that is not just attractive, but will work. For example, in the US, baking flour is typically packaged in white paper bags. Therefore, a paper bag that is a color other than white would represent a departure from the color conventions for that flour category, and would likely be perceived as novel to the consumer who is familiar with that category. The more distinctive that color is relative to white, the more novel it will appear to the familiar consumer, the more likely it will stand out to the familiar consumer and that it will draw the consumer’s eye. However, as Garber, Burke and Jones (2000) show, the greater attention that novel color garners will not be translated into greater likelihood of purchase consideration unless that color also evokes a meaning that is consistent with favorable product performance for that category. Therefore, the black or purple flour bags shown in Figure 25.4 will increase consideration

The mediating effects of the appearance of nondurable consumer goods

587

FIGURE 25.3 Contrast effects. These two photos show respective lineups of dishwashing liquids. The center bottle is the same in both, Palmolive Lemon-Lime dishwashing detergent. Its yellow-green color contrasts with the green liquids that surround it in the top photo, and contrasts as well with yellow liquids that surround it in the bottom photo. It is also interesting to note that the Palmolive Lemon-Lime bottle appears darker in the top photo than it does in the bottom photo. The lighting conditions were the same in each case. What causes this contrast effect are the respective colors that surround it.

and subsequent purchase, despite its attention-getting potential, only if consumers who notice it also conclude that the flour it represents will also be good for baking. Another important finding for novelty is that its effect on the consumer may be mediated by differential attention. Differential attention refers to the notion that novel stimuli draw the eye away from more mundane stimuli, such as an unusually shaped package drawing the eye away from the more conventionally shaped package. Folkes and Matta (2004) find that novel appearance causes packages to appear larger in volume than packages whose appearance is more commonplace, and attribute this effect on volume perception to the fact that the consumer attends to the novel stimulus more actively, and works harder to comprehend it. Further research is needed to confirm the mediating effect of

588

Product Experience

FIGURE 25.4 Consistency of meaning within novelty. The black and purple flour bags above are both examples of altered packages that depart sharply from the white flour bag conventions established for this category, and to which most competitor brands in the United Stated adhere. As a result, these bags appear to be sharply contrasting, and bring attention to this particular flour brand. The question is, having caught the eye, will these bags be interpreted as representing a flour that will bake well? This latter quality, an example of how package appearance conveys meaning of the brand it represents, is a crucial next step in bringing the consumer to the point of considering a brand for purchase.

differential attention on volume perception, as well as explore other effects that it may have on the consumer. Folkes and Matta (2004) demonstrate these effects via novel package shape. Will other visual elements such as novel color or size also evoke similar results?

3. COLOR EXPERIMENTATION IN MARKETING The persuasive effects of color are vastly under-researched in commerce; surprising given color’s powerful role in identifying and distinguishing brands, and its ability to confer symbolic and associative meaning to them, particularly in a world that is becoming ever more graphic in nature. (It should be noted, however, that anecdotal evidence suggests that visual research of a proprietary nature is ongoing, though most of this is out of the public domain.) What little marketing-specific color research there is mostly confirms the long-wavelength, short-wavelength dichotomy described previously. Bellizzi and Hite (1992) and Bellizzi, Crowley and Hasty (1983) test consumer color preferences for retail store designs and find that blue is soothing and preferred, and red is arousing and less well liked. Gorn et al. (1997) decompose color into its constituent elements – hue, chroma, and value – and test their respective effects on arousal, affect, and recall in print ads. They extend the notion that red is exciting by noting that any highly saturated color also tends to be arousing, and that paler colors tend to be relaxing. Several studies compare the effectiveness of color versus black-and-white in print media. Sparkman and Austin (1980) look at print advertising, finding that color ads sell

The mediating effects of the appearance of nondurable consumer goods

589

more than black-and-white ads. Click and Stempel (1976) report that newspaper readers prefer the front pages of newspapers with color. Meyers-Levy and Peracchio (1995) demonstrate that black-and-white ads have greater impact when few cognitive resources are devoted to the processing of a print ad photo, or when too few resources are available for the viewer to process the photo as elaborately as they would like. Schindler (1986) points out that the use of color in an ad can sacrifice contrast, reducing legibility and readability. A serious limitation to this research is that color as a visual stimulus is treated atheoretically as a purely sensory phenomenon, and the cognitive processing of visual stimuli is largely overlooked or ignored. That is why this research as a whole does not present a consistent set of findings (Scott 1994), nor does it really extend our knowledge of what color is or how it works in a communications context (Garber, Hyatt and Starr, 2001). Garber, Burke and Jones (2000) examine the effects of rather large color changes in existing packaging on brand consideration and consumer choice. An important extension to this research would be to consider the effect of rather more subtle color changes. Given that the human eye and brain are acutely aware of rather minute color differences, and that such differences are oftentimes all that separate the colors used by, for example, some leading brand and a competitor seeking to emulate the leader’s appearance (i.e. a me-too color strategy), how dissimilar do colors have to be in order for viewers to perceive the difference? How similar do different colors have to appear in order for viewers to perceive similarities, and associate the respective brands they represent? Other visual questions in marketing to be addressed would include the meaning of color when presented in contexts. Guilford and Smith (1959) studied the meaning of colors in general. For example, it is understood that long wavelength colors such as blues and greens are perceived to be cool, gentle, and serene, whereas short wavelength colors such as reds and oranges are perceived to be hot, exciting, sexy, and dangerous. But colors’ meanings are also known to be very context dependent. For example, red in the context of the cola category means ‘Coke’. But, apart from color’s meaning as a simple identifier, what meaning might a color convey to the object it represents, such as a cola? Do category-specific colors adopted by brands as identifiers convey meanings to the brands they represent in accordance with the generalized meanings attributed to colors by Guilford and Smith (1959), or do colors take on new meanings in specific contexts such as product categories and brands? Do these meanings vary across categories? And, if so, what are they, and what determines them? Also, do colors mean different things to different people? Men versus women? Young versus old? Across more specifically defined target segments? Research is also necessary to examine the effects of each of the visual elements that comprise an integrated package and other unified visual marketing elements, including the main effects of size, shape, graphic elements, motion, and their respective interactions. As much as there is limited research into the main effects of aspects of appearance, research examining interaction effects is particularly lacking. One study that tests such interactions compares the relative effects of one visual element with its interaction with other visual elements, by comparing masked and unmasked conditions (Garber, Hyatt and Boya, 2007). Consumer response to the main effect of a single visual element (in this case, package shape) was found to be the same regardless of whether packages were painted gray, thereby eliminating exposure to all other visual elements, or whether packages were left fully unmasked. This result suggests that the main effects of individual visual elements (e.g. size, shape, color, etc.) may be equally robust regardless of their interaction with other visual elements, lending external validity to all prior main effects tests. However, more work needs to be done, explicitly examining the strength of visual interaction effects of all types. All the visual effects studied so far in marketing have used packages as visual test stimuli. Those effects found to be true for packaging need to be extended to other visual

590

Product Experience

marketing phenomena, including, for example, advertising of various types, store atmospherics and displays, trade show displays, trucks, uniforms, logos, websites, etc. Future research should examine the interaction of these visuals with other marketing mix variables. Most prior visual research in marketing has limited itself to examining the main effects of some visual element while controlling for variations in overall marketing strategy. This has allowed researchers to determine the effects of such visual elements, but not in the context of the marketplace. Further visual research, in which marketing strategies of various types are manipulated along with visual effects, would reveal much concerning strategy formulation surrounding appearance.

4. THE INTERACTION OF COLOR AND FLAVOR Another set of interactions that seems to have piqued the interest of practitioners of late are multisensory effects, though research in the area is in its infancy. An exception is a well developed literature, most of it to be found in food sciences, examining the relationship between food color and flavor. Outside the food sciences, however, the impact of food color has been completely neglected (for reviews of this literature, see Cardello, 1996; Garber, Hyatt and Starr, 2001). Following is a brief review of the food color and flavor research. Many foods today do not display their natural color (Tuorila-Ollikainen, 1982). In general, food producers commonly select, modify, heighten or standardize the colors that we see and come to associate with specific foods. Green for peppermint or brown for cola are examples. The effects of conditioned food color/flavor associations in colorassociated foods become so ingrained that the unexpected pairing of a given food with a novel color renders it unpalatable (Maga, 1974). Recent violations of the previous, such as blue and purple ketchup (Heinz), or pink and blue margarine (Parkay), would seem to contradict prior assertions; however, it is to be noted that these novelly colored food products were being targeting to children whose color/flavor conventions are not well formed, and therefore were not available to be violated. Prior research illustrates the multi-modal character of flavor and the interplay of visual elements with other sensory modes. One study investigates the role that food color plays in conferring identity, meaning, and liking to those foods and beverages that assume many flavor varieties. In a taste test experiment manipulating food color and label information, results indicate that food color affects the consumer’s ability to correctly identify flavor and to form distinct flavor profiles and preferences (Garber, Hyatt and Starr, 2001). Similarly, the effects of color on odor perception have also been studied (Zellner, Bartoli and Eckard, 1991). Food color dominates all other flavor information sources, including labeling and taste. These results support the notion that food color is inextricably linked to expected flavor in the minds of consumers, making the selection of uncharacteristic food color problematical. Several new packaging strategies attempt the disconnection of the food color/expected flavor relationship by denying the consumer the ability to readily categorize the flavor cues that food color and labeling present, so that the consumer is induced into a mode of more elaborate information processing. This opens an opportunity for the presentation of promotional ideas, symbols, meanings, and associations through the medium of novel food color. For example, Gatorade sought to extend their product line, but was limited by the fact that there are a finite number of fruit flavors, all of which can be found in competitor product lines. The Frost series, an example of which is shown in Figure 25.5, solved this problem by replacing the usual flavor references on the package with references to themes

The mediating effects of the appearance of nondurable consumer goods

591

FIGURE 25.5 Novel food color. The color of this ‘Frost’ drink is a cold blue, more readily associated with temperature than any fruit flavor.

of winter, with colors and names that evoked images of cold, ice and snow; images that are not inconsistent with themes normally positively associated with fruit beverages, but which are normally evoked in advertising and not by food color and flavor names. For example, Gatorade’s Frost series includes ‘Glacier Freeze’, which comes in a clear strong blue color not unlike mouthwash, ‘Whitewater Splash’, which comes in a clear strong green, and one called ‘Alpine Snow’, which comes in a semi-translucent white. The consumer is therefore forced to consider and evaluate the Frost line of drinks in an entirely new context. Other valuable future research would include replicating the color/flavor research while manipulating flavor along with food color and label information, so that any flavorspecific effects can be accounted for. Similarly, food category may also be manipulated in order to account for any food-category-specific effects. Future research should examine the specific meanings and associations that consumers have concerning clear or colorless

592

Product Experience

beverages, as well as the circumstances under which consumers may prefer less complete or less explicit product information. Color–flavor interactions are one of the many possible multisensory interactions relevant to marketing and consumer behavior. Most other such interactions have heretofore been neglected, with the likely exception of proprietary research, and deserve future attention given their potential importance to understanding sensory marketing effects. For example, the new field of neuromarketing, which utilizes ever-improving instrumentation to map brain activity related to specific consumer tasks, holds much promise for understanding complicated multisensory effects.

5. THE INTERACTION OF SHAPE AND SIZE APPEARANCE As with color, the full effects of other visual package elements, including size and shape, have not been fully explored. Interestingly, most of the work done so far looks at the effect of one of the visual elements, shape, on the other, size, with the general finding that humans err, at times grossly, in their package volume estimation judgments, and that complex container shape amplifies the error, and causes consumers to alter their estimation strategies. We begin by reviewing the literature on the interaction of these visual elements, though it may fairly be said that the more important effects of size and shape on identification, meaning, and contrast are less well-developed.

5.1. The importance of size to the consumer The appearance of package size exerts an important, if problematical, influence on shoppers in the store. Size appearance is important because, in many package goods categories, there can be favorable connotations to consumers projecting a certain size appearance. For example, appearing small can be advantageous when ‘small’ favorably connotes high quality, elegance or ease of use, as it does with caviar, perfume, and cellular telephones. Conversely, appearing large can be advantageous when ‘large’ favorably connotes economy, value or long-lastingness, as it does with breakfast cereals and economy shampoos, such as Suave and Capri (the latter referred to in the trade as ‘tanker’ brands).

5.2. The importance of size appearance to the marketer A complicating factor for the package designer or the manager that must create or select new packaging is that consumption-related utilitarian aspects place practical limits on how large or how small a package can actually be made to be, and, in turn, how large or how small it can or should be made to appear. For example, a big bottle or box that is attractive on the shelf can pall when it has to be lugged to the car, does not fit in the pantry, pours with difficulty, or spoils before it is used up. Conversely, a tiny box may lose its charm when its contents fail to live up to the quality or refinement that its precious size suggests, or its contents are used up in the blink of an eye. Though consumers have size preferences, humans commonly and systematically err in their size estimations (Hundleby et al., 1993). It is therefore appearance of size, and not merely actual size, that affects the shopper. Actual size is therefore not as robust a predictor of perceived size as many might presume. Other visual characteristics of a container tend to confound the size estimation process, and can serve to deceive the viewer into believing that a container holds more or less than it actually does (Frayman and Dawson, 1981; Raghubir and Krishna, 1996). Those other visual factors that may confound

The mediating effects of the appearance of nondurable consumer goods

593

package size estimation may include height, shape, head space (whether actual or presumed), color, variegated color, value, container material, contents (whether actual or presumed), weight appearance, label elements including contrasting color and value (as in light or dark), text, logos, characters, photos, the size appearance of surrounding packages, and the viewer’s degree of familiarity with any or all of the above. With respect to volume estimation, there is a sizable body of research also studying haptic effects, with the interesting result that there is an inverse relationship between perceived weight and perceived volume (Krishna, 2006).

5.3. Size/shape research Given that size estimation research has a long history, about a hundred years, and that size consideration plays such an important role in consumer choice among packaged goods, it is surprising that it has been only recently that a few papers have appeared in the marketing literature addressing size estimation in a commercial context. These recent papers constitute a small stream, primarily investigating the effects of volume over-estimation on over-consumption in various consumption contexts. General findings indicate that people tend to consume more when they draw from what appears to them to be a larger rather than a smaller vessel (Raghubir and Krishna, 1999; Wansink, 1996; Wansink and van Ittersum, 2003). Only two papers, Raghubir and Krishna (1999) and Folkes and Matta (2004), have examined volume estimation bias in a packaging context; the former in the context of cylindrical packages, the latter in context of tapered beverage bottles. Both demonstrate the significant main effect of package shape on volume estimation, and each illustrates one of the two main volume estimation strategies used by consumers, one dimensionbased and the other shape-based. A dimensions-based approach, represented by Ragubir and Krishna, and referred to as the ‘height-size illusion’, presumes that people first seek out and estimate the linear dimensions of a container, and then infer size from them. For an example of the height-size illusion, see Figure 25.6. The elongation of the vertical dimension predominates over width and depth as an indicator of volume, and often is the only one used by consumers, who abhor covariance (Frayman and Dawson, 1981; Hundleby et al., 1993; Jenkins, McGahan and Richard, 1994; Krider, Raghubir and Krishna, 2001). A shape-based approach, represented by Folkes and Matta, presumes that humans view shape holistically and infer volume directly from package shape (Bingham, 1993; Jenkins, McGahan and Richard, 1994). This is the ‘familiar size illusion’. Estimation error in this process may derive from the mental categorization that would underlie this process. For example, if a container being evaluated by a given consumer is thought of as mustard-jar-like, and if mustard jars hold a size meaning of, say, four ounces for that given consumer, then the consumer employing the ‘familiar size’ strategy may be inclined to assign a size of four ounces to the jar at hand, even when that jar is significantly smaller or larger than four ounces. A jar that is the same size but a very different though familiar size may fall into another shape category, and therefore may evoke an entirely different size reference, and subsequent size estimate. The ‘familiar size’ strategy therefore suggests that different package shape types may convey distinct size meanings, such that packages of the same size but different shapes may connote different sizes, appearing different in size from each other, as well as from their actual size. What happens with more complex forms, where shape itself may serve to obscure linear dimensions, making them harder to discern as well as to estimate? According to Biederman (1987), complexity in geometric forms is a function of the number and prominence of its separate parts, and the number and prominence of the concavities that a

594

Product Experience

FIGURE 25.6 The height-size illusion. These 7-UP bottles encompass the same volume. The bottle on the right looks larger because it is taller, and has a larger cap.

FIGURE 25.7 A taxonomy of package shape parts.

container displays. Compound complex forms, – containers made up of the conspicuous joining of two or more simple forms, as is the case, for example, with many salad dressing bottles, which commonly exhibit clearly delineated caps and necks, shoulders, bodies, and feet (see Figure 25.7), are commonly employed by package designers seeking unique and uniquely identifiable forms (Garber, Hyatt and Boya, 2007). Biederman (1987), introduces what is now the prevailing theory of object recognition, which gives us clues as to how consumers may go about estimating the volumes of the

The mediating effects of the appearance of nondurable consumer goods

595

FIGURE 25.8 Package meaning.

most complex package containers of all, complex compound forms, where not only linear dimension, but simple shapes themselves, may be obscured. Called recognition-by-components theory (RBC), it provides a means by which we may define, measure and compare volumetric shape complexity. Recognition-by-components theory takes a decompositional approach to perceptual object recognition, and is (p. 115) ‘… conceptualized to be a process in which the image of the input is segmented at regions of deep concavity into an arrangement of simple geometric components, such as blocks, cylinders, wedges, and cones’. RBC theory suggests that the recognition of objects comes from the prior processing of the simple volumes that comprise a compound complex form, which are subsequently integrated into whole objects by the brain. In Biederman’s (1987) vernacular, these simple forms are called geons, and there are 32 of them. However, utility requirements constrain the range of shapes that packages may take, as well as the number and shape of its parts, and their means of joining, thus simplifying the demands for a package-specific shape part taxonomy. Such a taxonomy includes just five such simple package parts – cap, neck, shoulders, body and foot (Garber, Hyatt and Boya, 2007). Packages can then be characterized and categorized according to the presence of different levels of these separate shape parts. Recent research on the shape/size relationship (Garber, Hyatt and Boya, 2007) indicates that the body of the package is the predominant indicator of perceived volume for consumers, suggesting that they infer container volume from their assessment of the most salient dimension of the most salient shape part. Clearly, though, package shape may have many more implications for store behavior than size appearance. In particular, shape may convey many intrinsic meanings, both symbolic and utilitarian, concerning the product it represents. For example, in a recent US promotional campaign by Perrier, the French sparkling water marketer, Perrier substituted like-sounding descriptors such as ‘Sexier’, Sassier’, and ‘Flirtier’ for its own brand name on the package, as seen in Figure 25.8. Perrier gambled that other visual elements of their familiar, long-time, bottle would serve to identify the brand, while using the space normally reserved for brand name to connote and attribute new meanings and associations.

6. A METHOD FOR EMPIRICAL VISUAL RESEARCH Most of the scant empirical research on the effects of visual processes, or the appearance of objects in marketing, addresses specific problems such as the utility of certain package forms (Wansink, 1994), without much ability for generalization; or is broadly conceptual

596

Product Experience

without much ability for concrete application. An example of the latter is Bloch (1995), who offers a conceptual model of product design. The purpose of the model is to bring ‘needed attention to the subject of product design and enable researchers to better investigate design issues’ (p. 17). Unfortunately, this research does not address the design selection problem itself. The method described in the following is intended to enable such experimentation. One reason for the lack of any far-reaching research on the visual elements of packaging may be that interested researchers do not know how to approach the problem empirically. Therefore, we propose a method of visual research taken from Garber (1995) and Garber, Burke and Jones (2000) that may enable such inroads to be made. This method adds experimental power by suggesting a systematic means of inventing, altering, calibrating, and selecting visual elements to obtain true and plausible representative visual types for experimental purposes. Prior visual and point-of-purchase effects research (c.f. Burke et al., 1992) has typically relied on existing products, packages and displays to represent all manipulation levels for its experiments. Two exceptions are Veryzer and Hutchinson (1998), who physically alter photos of durable goods to achieve different levels of visual design elements including unity, harmony, to measure effects of aesthetic quality on consume choice; and Durgee and O’Connor (1994), who use the tissue box creations of an undergraduate design class to test the ability of designers to communicate intended messages to targeted consumers. However, in neither of these cases is there the attempt to create a standard process transferable or applicable to other visual research initiatives in marketing. This method is appropriate for testing the effects of any visual element, or the visual effects of some stimulus object, such as a package, when presented in a relative context. By relative context, we refer to any situation where the target stimulus is embedded in a visual field with a number of distracter objects, as in the case of a product or package on a store shelf, or even an ad jammed into a commercial break with other ads on television. A particular feature of this method is its ability to separate the sensory effects of a visual experience from the effects of prior experience. This confound is a longstanding problem in sensory research, as indicated by Duncker (1939, p. 255), who early on was concerned about the influence of past experience on perceptual properties. The method addresses such confounding, however, by using a combination of similarity scaling and correspondence analysis to disentangle these effects. Following, we provide a general description of this method, illustrated by visual stimuli generated and selected in Garber, Burke and Jones (2000).

6.1. Calibration for novelty of appearance In a point-of-purchase marketing context, the sensory experience that is of interest is novelty. It is the surprising or unexpected visual stimulus that calls for (involuntary) attention; attention being the primary point-of-purchase goal of marketers, and therefore a primary role for package and product appearance. A typical convenience store layout puts competitor brands together into adjacent facings for comparison purposes. In that context, it is the package that departs significantly from the visual norms because it stands out from its more conventional neighbors. In the method we describe, novelty is operationalized as the degree of dissimilarity of some changed package from an actual package for an existing familiar brand in an established product category. The following method allows the researcher to calibrate a set of visual stimuli according to the relative degree of novelty, relative to some base stimulus, that each of them exhibits.

The mediating effects of the appearance of nondurable consumer goods

597

FIGURE 25.9 Perceived similarity of alternative gold medal packages.

In the case of our example, Gold Medal Flour bags, the actual bag, white in color, is our base stimulus. A number of altered bags were created exhibiting a range of different background colors, from beige and pale blue to purple and black, collectively representing a full span of alternatives along all the color dimensions of hue, value, and chroma (Gorn et al., 1997). (The more stimuli that are generated and tested, the more likely that final results will be robust. Since novelty is a relative stimulus, and evaluations come by means of comparison, the more comparisons that a subject makes, the more robust the final calibrations.) Each of the bag alternatives are paired with all other alternatives, on display cards, so that all pairwise combinations including those incorporating the actual bag are represented. The subject task is to evaluate each of these stimulus pairs one at a time and rate the relative degree of similarity or dissimilarity, by appearance, within pairs. These data are then analyzed using the KYST multidimensional scaling algorithm (Kruskal, Young and Seery, 1973) as implemented in PC-MDS 5.1, or some other multidimensional scaling software from which low-dimension perceptual maps are generated, such as the map shown in Figure 25.9. The maps represent package alternatives as points in a common, perceptual space, where the Euclidean distance from the original (‘actual’) package to each of the color-altered packages indicates the dissimilarity or novelty of the new package. New designs that are perceived to be most similar to the base package in both the lowdimensional scaling solutions are classified as ‘very similar’, (e.g. white bag with orange banner, beige bag, orange bag). Candidates that are the furthest away from the original package are classified as ‘very dissimilar’ (e.g. the black and purple bags). Packages that fall between these two extremes are categorized as ‘somewhat dissimilar’.

6.2. Calibration for meaning inferred from appearance Visual cues are not simply direct sensory experience. Visual cues also trigger memory, leading to comprehension. The consumer therefore will draw upon past experience to process packages and products by their appearance. This describes the new role that

598

Product Experience

appearance takes on in the later stages of the decision process. Having come to a consumer’s attention, appearance is used to confer meaning to the product it represents. It may also lead to forming expectations of how the product will perform. In the method described, comprehension is operationalized as the brand performance profiles that consumers would associate with the products that each of the altered stimulus packages would represent and connote by their appearance. Researchers are able to calibrate a set of visual stimuli according to the performance-based meanings that each exhibits. Attributes become input to map the resulting brand and attribute sets onto a common, multidimensional space. The subject task that generates the needed brand/attribute associations is as follows. The subject is presented with a matrix on which columns represent package appearance alternatives, including the base package stimulus, and rows represent a bundle of product performance benefits that consumers would consider when selecting brands for purchase. The top row of the matrix would include images of the brand packages visually representing each alternative being evaluated. The left-hand column would include each benefit attribute represented by a verbal descriptor, typically a sentence or two. The subject is asked to imagine and evaluate the product that each package variation contains, across the listed benefit attributes. The resulting input data is a frequency table indicating the number of subjects who associate a given package appearance alternative with a given benefit attribute. The SIMCA correspondence analysis package (Greenacre, 1993), or some other correspondence analysis package, can be used to generate a map such as that shown in Figure 25.10, representing both the package alternatives and the attribute benefit sets as points in a low-dimensional space. The locations of the respective sets give meaning to specific locations within that space. From this, the researcher may infer and interpret, and ultimately compare, the individual performance profiles of each package alternative. The comparisons of interest are between each of the package alternatives and the actual, or base, package. Those packages that are placed physically close to the base stimulus may be interpreted in having a similar perceived performance profile, and therefore be similar in product performance meaning to the actual package; and, by extension, consistent in meaning with the category as a whole. Those packages that fall at a distance from the base package on any or several dimensions may be interpreted as having a profile that is inconsistent with the category. For example, the actual Gold Medal bag is seen as being ‘fresh quality’, ‘good value’, ‘naturally pure’, and ‘good tasting’. New packages with similar benefit profiles (like the beige bag) are classified as having meaning that is consistent with the category. New designs with very different benefit profiles (such as the black bag, which was seen as being ‘inexpensive’) are classified as being inconsistent in meaning. Having used this pretest process to generate and calibrate as large a number of package appearance alternatives as possible independently for both novelty and meaning, the researcher is now able to select stimuli for the respective levels of novelty and consistency of meaning that each represents. Subsequent experiments that test the respective effects of novel appearance and consistency of meaning that a particular appearance connotes, on whatever set of dependent variables are of interest, are then able to be conducted. For example, Garber, Burke and Jones (2000) employed a computerized three-dimensional grocery simulation store developed by Burke (Burke, 1996; Burke et al., 1992) that uses 3D computer graphics to recreate the appearance of a grocery shelf on a 20-inch touchscreen monitor, as shown in Figure 25.11. Shoppers pan down the aisles of the store using a 3D trackball, ‘pick up’ products by touching their images on the screen, and rotate packages and magnify labels for closer inspection. To purchase a product, the consumer touches an image of a shopping cart and the package would fly into the basket. This

The mediating effects of the appearance of nondurable consumer goods

599

FIGURE 25.10 Attribute associations for alternative gold medal flour packages.

FIGURE 25.11 Burke’s virtual reality grocery store.

simulation offers several advantages over existing methodologies. It provides the realism and visual clutter of an in-store experiment while delivering the control and process tracing measures of laboratory research. The computer unobtrusively records the amount of time consumers spent shopping in each category, the items they pick up, the amount of time taken to examine individual packages and labels, as well as the quantity of items

600

Product Experience

purchased. Consumer behavior in the virtual shopping simulation has been found to closely mirror behavior in the physical store (see Burke, 1996; Burke et al., 1992).

7. CONCLUSION In summary, this chapter reviews the extant research on the effects of product and package appearance. It is organized in terms of the basic visual elements that comprise overall appearance, size, shape, and color (movement being an additional visual element that has to our knowledge received no treatment in the product and package literature). Research into these effects has received spotty attention at best, though it is generally understood that appearance is a strong mediator of attention, consideration and choice. A primary reason for this lack of attention may be because these visual elements, and appearance in general, are both hard to conceptualize in concrete terms and to test in empirical terms. A primary reason for both these inabilities is that these visual elements are strongly interacting, both with each other and their environment, making them hard to think about, because they are so hard to isolate, and their effects hard to test, because there are so many confounds to account for experimentally. We therefore also propose a method for dealing with these issues. We hope that this chapter will provide an impetus and a starting point for those who desire to do visual research of such an ‘in-context’ nature, and fill in the many gaps in a critical area for which there has been far too little research.

REFERENCES Aaronson, B. S. (1970). Some affective stereotypes of color. International Journal of Symbology, 2, 15–27. Adams, F. M. and Osgood, C. E. (1973). A cross-cultural study of the affective meanings of color. Journal of Cross Cultural Psychology, 4, 135–156. Albers, J. (1963). Interaction of color. New Haven, CT: Yale University Press. Ball, V. K. (1965). The aesthetics of color: A review of fifty years of experimentation. Journal of Aesthetics and Art Criticism, 23, 441–452. Bellizzi, J. A. and Hite, R. E. (1992). Environmental color, consumer feelings and purchase likelihood. Psychology and Marketing, 9, 347–363. Bellizzi, J. A., Crowley, A. E. and Hasty, R. W. (1983). The effects of color in store design. Journal of Retailing, 59, 21–45. Berlyne, E. E. (1974). Studies in the new experimental aesthetics. New York, NY: John Wiley. Biederman, I. (1987). Recognition-by-components: A theory of human image understanding. Psychological Review, 94 (April), 115–147. Bingham, G. P. (1993). Perceiving the size of trees: Form as information about scale. Journal of Experimental Psychology: Human Perception and Performance, 16(6), 1139–1161. Bjerstedt, A. (1960). Warm–cool color preferences as potential personality indicators: Preliminary note. Perceptual and Motor Skills, 10, 31–34. Bloch, P. H. (1995). Seeking the ideal form: Product design and consumer response. Journal of Marketing, 59 (July), 16–29. Bruce, V. and Green, P. R. (1990). Visual perception: Physiology, psychology and ecology. Hillsdale, NJ: Lawrence Erlbaum. Burke, R. R. (1996). Virtual shopping: Breakthrough in marketing research. Harvard Business Review, 74 (March–April), 120–131. Burke, R. R., Harlam, B. A., Kahn, B. E. and Lodish, L. (1992). Comparing dynamic consumer choice in real and computer-simulated environments. Journal of Consumer Research, 19 (June), 71–82. Cardello, A. V. (1996). The role of the human senses in food acceptance. In: H. L. Meiselman and H. J. H. McFie (Eds.) Food Choice, Acceptance and Consumption, pp. 1–82. New York, NY: Blackie Academic and Professional. Cheskin, L. (1957). How to predict what people will buy. New York, NY: Liveright.

The mediating effects of the appearance of nondurable consumer goods

601

Click, J. W. and Stempel, G. H. III (1976). Reader response to front pages with four-color halftones. Journalism Quarterly, 53, 736–738. Collinson, D. (1992). Aesthetic experience. In: O. Hanfling (Ed.) Philosophical Aesthetics. Cambridge: The Open University. Crick, F. (1994). The astonishing hypothesis: The scientific search for the soul. New York, NY: Simon and Schuster. Danger, E. P. (1969). How to use color to sell. Boston, MA: Cahners. Davidoff, J. (1991). Cognition through color. Boston, MA: MIT Press. Dichter, E. (1975). Packaging: The sixth sense? A guide to identifying consumer Motivation. Boston, MA: Cahners Books. Duncker, K. (1939). The influence of past experience upon perceptual properties. American Journal of Psychology, 52, 255–265. Durgee, J. and O’Connor, G. C. (1994). Perceiving what package designs express: A multisensory exploratory study using creative writing measurement techniques. Unpublished manuscript. Fisher, J. D. (1974). Situation-specific variables as determinants of perceived environmental aesthetic quality and perceived crowdedness. Journal of Research and Personality, 8, 177–188. Folkes, V. and Matta, S. (2004). The effect of package shape on consumers’ judgments of product volume: Attention as a mental contaminant. Journal of Consumer Research, 31 (September), 390–401. Francis, F. J. (1977). Colour and appearances as dominating sensory properties of foods. In: G. Birch, J. Brennan and K. Parker (Eds.) Sensory Properties of Foods, pp. 27–43. London: Applied Science. Frayman, B. J. and Dawson, W. E. (1981). The effect of object shape and mode of presentation on judgments of apparent volume. Perception and Psychophysics, 29(1), 56–62. Garber, L. L., Jr. (1995). The role of package appearance in consumer choice. Unpublished Ph.D. Dissertation, University of North Carolina at Chapel Hill. Garber, L. L. Jr., Burke, R. R. and Jones, J. M. (2000). The role of package appearance in consumer purchase consideration and choice. Boston, MA: Marketing Science Institute Working Paper Series. Garber, L. L., Jr. and Hyatt, E. M. (2000). Color as a Tool for Visual Persuasion. In: R. Batra and L. Scott. Visual persuasion. Hillsdale, NJ: Lawrence Erlbaum. Garber, L. L. Jr., Hyatt, E. M. and Boya, Ü. Ö. (2007). The effect of package shape type on perceived volume: an empirical investigation. (Manuscript submitted for publication). Garber, L. L. Jr., Hyatt, E. M. and Starr, R. G. Jr. (2001). Placing food color experimentation into a valid consumer context. Journal of Food Products Marketing, 7(3), 3–24. Golden, C. J. (1974). Sex differences on the performance in the Stroop color and word test. Perpetual and Motor Skills, 39, 1067–1070. Gorn, G. J., Chattopadhyay, A., Yi, T. and Dahl, E. W. (1997). Effects of color as an executional cue: They’re in the shade. Management Science , 43, 1387–1400. Greenacre, M. J. (1993). Correspondence analysis in practice. New York, NY: Academic Press. Guilford, J. P. and Smith, P. C. (1959). A system of color-preferences. American Journal of Psychology, 4, 487–502. Hekkert, P., Snelders, D. and van Wieringen, P. C. W. (2003). Most advanced, yet acceptable: Typicality and novelty as joint predictors of aesthetic preference in industrial design. British Journal of Psychology, 94, 111–124. Hilbert, D. R. (1987). Color and color perception: A study in anthropocentric realism. Palo Alto, CA: Center for the Study of Language and Information. Hine, T. T. (1996). The total package. New York, NY: Little, Brown. Holmes, C. B. and Buchanan, J. (1984). Color preference as a function of the object described. Bulletin of the Psychonomic Society, 22, 423–425. Hundleby, J. D., Musimi, L., Van Kampen, J. and Keating, L. J. (1992). The generality of individual differences in the estimation of size. Journal of Psychology, 127(2), 233–243. Jenkins, C. A., McGahan, J. A. and Richard, F. D. (1994). Effects of a common linguistic misrepresentation on judgments about the relationship between height and weight. Perceptual and Motor Skills, 79, 339–347. Kahneman, D. (1973). Attention and effort. Englewood, NJ: Prentice-Hall. Klink, R. R. (2003). Creating meaningful brands: The relationship between brand name and brand mark. Marketing Letters, 14(3), 143–157. Kearney, G. E. (1966). Hue preferences as a function of ambient temperatures. Australian Journal of Psychology, 18, 271–275. Krider, R. E., Raghubir, P. and Krishna, A. (2001). Pizzas: Π or square? Psychophysical biases in area comparisons. Marketing Science, 20 (Fall), 405–425. Krishna, A. (2006). Interaction of senses: The effect of vision versus touch on the elongation bias. Journal of Consumer Research, 32(4), 557–566. Kruskal, J. B., Young, F. W. and Seery, J. B. (1973). How to use KYST, a very flexible program to do multidimensional scaling and unfolding. Murray Hill, NJ: Bell Laboratories.

602

Product Experience

Land, E. H. (1977). The retinex theory of color vision. Scientific American, 237, 108–128. Lee, S. and Barnes, J. H., Jr. (1990). Using color preferences in magazine advertising. Journal of Advertising Research, 29, 25–30. Loken, B. and Ward, J. (1990). alternative approaches to understanding the determinants of typicality. Journal of Consumer Research, 17 (September), 111–126. Maga, J. A. (1974). Influence of color on taste thresholds. Chemical Senses, Flavor, 1, 115–119. Mandler, G. (1982). The Structure of Value: Accounting for Taste. In: M. S. Clark and S. T. Fiske (Eds.) Affect and cognition: The 17th Annual Carnegie Symposium, pp. 3–36. Hillsdale, NJ: Lawrence Erlbaum Associates. Marr, D. (1982). Vision: A computational investigation into the human representation of processing of visual information. San Francisco CA: W.H. Freeman. Marr, D. and Nishihara, H. K. (1978). Representation and recognition of the spatial organization of threedimensional shapes. Proceedings of the Royal Society of London, Series B, 207, 187–216. Martino, G. and Marks, L. E. (2001). Synesthesia: Strong and weak. Current Directions in Psychological Science, 10(2), 61–65. Meyers-Levy, J. and Peracchio, L. A. (1995). Understanding the effects of color: How the correspondence between available and required resources affects attitudes. Journal of Consumer Research, 22, 121–138. Meyers-Levy, J. and Tybout, A. M. (1989). Schema incongruity as a basis for product evaluation. Journal of Consumer Research, 16 (June), 39–54. Nelson, M. R. and Hitchon, J. C. (1995). Theory of synesthesia applied to persuasion in print advertising headlines. Journalism and Mass Communication Quarterly, 346–360. Raghubir, P. and Krishna, A. (1996). As the crow flies: bias in consumers’ map-based distance judgments. Journal of Consumer Research, 21 (June), 26–39. Raghubir, P. and Krishna, A. (1999). Vital dimensions in volume perception: Can the eye fool the stomach? Journal of Marketing Research, XXXVI (August), 313–326. Sacks, O. (1995). The case of the color blind painter. An Anthropologist on Mars: 7 Paradoxical Tales. New York: Vintage. Schifferstein, H. N. J. (2006). The relative importance of sensory modalities in product usage: a study of selfreports. Acta Psychologica, 121, 41–64. Schifferstein, H. N. J. and Tanudjaja, I. (2004). Visualising fragrances through colours: the mediating role of emotions. Perception, 33, 1249–1266. Schindler, P. S. (1986). Color and Contrast in magazine advertising. Psychology and Marketing, 3, 69–87. Scott, L. (1994). Images in advertising: The need for a theory of visual rhetoric. Journal of Consumer Research, 2, 252–273. Sharpe, D. T. (1975). The psychology of color and design. Chicago, IL: Nelson-Hall. Sparkman, R. R., Jr. and Austin, L. M. (1980). The effect on sales of color in newspaper advertisements. Journal of Advertising, 9, 39–42. Swirnoff, L. (1989). Dimensional color. Boston, MA: Birkhauser. Tuorila-Ollikainen, H. (1982). Pleasantness of colourless and coloured soft drinks and consumer attitudes to artificial food colours. Journal for Intake Research, 3, 369–376. Treisman, A. (1991). Search, similarity, and integration of features between and within dimensions. Journal of Experimental Psychology: Human Perception and Performance, 40A, 201–37. Treisman, A. and Gelade, G. (1980). A feature-integration theory of attention. Cognitive Psychology, 14, 107–141. Veryzer, R. W. and Hutchinson, J. W. (1998). The influence of unity and prototypicality on aesthetic responses to new product designs. Journal of Consumer Research, 24 (March), 374–394. Wansink, B. (1994). How and why package size influences usage volume. Report No. 94–100. Cambridge, MA: Marketing Science Institute Working Paper Series. Wansink, B. (1996). Can package size accelerate usage volume. Journal of Marketing, 60(July), 1–14. Wansink, B. and Van Ittersum, K. (2003). Bottoms up! The influence of elongation on pouring and consumption volume. Journal of Consumer Research, 30 (December), 455–463. Whitfield, T. W. A. and Wiltshire, T. J. (1990). Color psychology: A critical review. Genetic, Social, and General Psychology Monographs, 116(4), 387–411. Zellner, D. A., Bartoli, A. M. and Eckard, R. (1991). Influence of color on odor identification and liking ratings. American Journal of Psychology, 104, 547–561. Zellner, D. A. and Kautz, M. A. (1990). Color affects perceived odor intensity. Journal of Experimental Psychology: Human Perception and Performance, 16, 391–397.

26

OFFICE EXPERIENCES CHRISTINA BODIN DANIELSSON Royal Institute of Technology, Stockholm, Sweden

1. INTRODUCTION It is important to recognize the office environment as an influencing factor in the western world. It exerts a significant impact on everyday life for a lot of people because of the amount of time that is spent at work. While organizational members need a place to work, the requested workspace is usually seen as a provision by the organization, with the organization maintaining clear ownership and control of the environment (Mazumdar, 1992). In this chapter the following are discussed: (1) the individual office employee’s experience of the physical office environment; (2) its influence on the individual employee and in the prolongation on the organization to which the employee belongs. Since most organizations and businesses operate in a physical environment and the physical office environment sets the conditions for the activities performed, its impact should be recognized. The office experience has an impact through its functional, social, and symbolic implications on interaction and cooperation among employees; thus office experiences are fundamental at both an individual and an organizational level. Research on how the work environment influences employees is found in numerous fields such as architecture, organizational and management theory, social and stress medicine, as well as environmental and social psychology. This chapter touches upon all these fields, but its focus is on the interior experience of office environments among employees. The aim is to discuss how the office environments influence employees, with an interdisciplinary approach to the subject. A special focus in this chapter is on the part of my research that investigates the influence on employees’ office experience by different office-types. When comparing the employees’ experiences of different office environments it is, in my opinion, important to use the variety of office types that exists in office design today, instead of only comparing a vaguely defined open plan office to a single room office. According to my research, the architectural and functional features Product Experience Copyright © 2008 Elsevier Ltd.

605

606

Product Experience

that define the existing office types have a great impact on office employees in different respects, such as employees’ health status, job satisfaction, and office experiences (Danielsson, 2005b). Therefore, I will discuss how research results can be used in the professional practice of office design. My research deals with the influence of office types on office employees in different respects. The identified office-types which are investigated are: (1) cell-office (a single person room office); (2) shared-office (2–3 people sharing room). Open plan offices; (3) small open plan office (with 4–9 persons/room); (4) medium open plan office (10–24 persons/room); (5) large open plan office (⬎24 persons/room); (6) flex-office (no personal workstation); and (7) combi-office (team-based office type). The seven identified office types used show unique features with regard to architectural and functional features and are based on the work by Ahlin and Westlander (1991) and Duffy (1999). The framework of organizational theorist Davis (1984), which describes how physical office environments influence employees, is used in the chapter to approach the interdisciplinary field of office research. Davis’s framework is combined with a classification of office experience into two groups: (1) design-specific experiences; and (2) experiences related to general conditions of an office environment that has a general solution to it when it is causing an environmental problem, thus here called general experience. This classification of office experiences is based on the nature of the experience and its components, but mainly on how problems related to these two types of group of experience are solved. Design-specific experiences are dependent on a unique condition in each specific office, determined by its architectural and functional features. It is the context that sets the framework for these experiences. This group of experiences is to a great extent dependent on spatial conditions at a specific location. When there is a problem related to design-specific experiences it is solved by case-specific solutions based on the architect’s/designer’s previous experiences of how to solve this type of problem. Solutions to problems which have their origin in design-specific experiences depend on the skill of the individual architect/designer. This skill is based on his/her former experiences of what has worked before in similar cases, as well as his/her creativity. General experiences, on the other hand, are not connected to a specific design of the office, but to the general conditions in the office. Work environment problems related to general experiences can be handled in the design process by general solutions, so called ‘cook book’ solutions, such as regulations and specified demands described in programs. The solutions to this group of problems depend on the architect’s/designer’s insight into the problem and the knowledge of regulations that will work as guidelines in the design process. These two groups of experiences are highly connected to each other, since they have a mediating effect on each other. For example, dissatisfaction with a general experience such as ventilation and noise often influences the experience and perception of designrelated factors which are design-specific experiences by nature. The classification of office experiences into two groups is useful for two reasons. It emphasizes the fact that there are different solutions to different problems of office experiences based on which classification the experience belongs to. It also emphasizes the interrelation between the different environmental factors that influence each other in the process of evaluation and perception of office environments. With knowledge of the origin of a specific office experience it will be easier to handle the issue in the design process. Davis’s framework, combined with the classification of office experiences, offers access to the subject from two different perspectives, which complement each other: (1) the theoretical and scientific perspective; and (2) the practitioner’s perspective.

Office experiences

607

The combination of the two frameworks provides a way to transfer research into practice, which is important in order to create office environments that support employees, as well as organizations, in the best possible way. When the knowledge of a specific office experience’s origin is put into a framework such as that of Davis, it is easier to grasp and view scientifically.

1.1. The importance of office design for organizations Since office work is mainly carried out in social situations, most of the research conducted in office environments within the field of organization and management theory deals with interpersonal relationships. These theories are united by the idea that the physical office environment is a possible means to achieve higher productivity due to a more satisfied workforce. The pioneer study within this field of research is the Hawthorne Studies (Roethlisberger and Dickson, 1939). The increased performance among employees in the studies was ascribed to the attention the employees received rather than the actual changes in the physical environment. This phenomenon has been called the ‘Hawthorne effect’. It should be said here that the Hawthorne Studies have been criticized by several researchers due to the way the study was conducted, and evidence from these studies themselves that contradicts the researchers’ results (e.g. Bramel and Friend, 1981). Nevertheless they have had a great impact on office research. Another early important work in this particular field is the work by Herzberg and his colleagues conducted in 1959, presented in the book ‘Work Motivation’ (2003). A distinction was made between the factors that led to high job satisfaction and work motivation and factors that lead to dissatisfaction. The former were called motivators, or satisfiers. The latter were called hygiene factors, or dissatisfiers. According to their theory, to achieve an acceptable level of job satisfaction among employees, the hygiene factors have to be provided for by the employer: A good physical environment, for example, will not enhance satisfaction, only reduce dissatisfaction. This theory constitutes the basis for my own standpoint that motivators such as work assignments, performance, and responsibility over your own work, personal development, leadership, and cooperation are the major factors in employee’s satisfaction with work. After these pioneer studies, anyone interested in the behavior of employees could no longer consider behavior isolated from the social and physical context. Instead, the social context, including group influences, social status, informal communication, and norms was now incorporated and embedded with the physical environment (e.g. Becker, 1981; Sundstrom, 1986). The social context should thus always be recognized in studies on employees’ experiences related to work.

2. ENVIRONMENTAL STRESSORS AND COPING STRATEGIES IN OFFICE ENVIRONMENTS The physical environment is an exterior stimulus which influences the individual psychologically as well as physiologically. The reactions caused by the physical environment can be classified as arousal, stress, distraction, or overload and fatigue in accordance with different theories (Sundstrom, 1986). Stress is of specific interest when discussing office employees’ experiences, since office employees run a high risk of ‘mental stress’ due to the nature of office work. Office work is not physically, but rather mentally demanding. The ability to switch off mental stress related to work after coming home from work varies highly between individuals. In addition, the natural breaks between activity

608

Product Experience

and recovery, which are important for the ability to cope with high demands, have to a great extent been lost in today’s office work. These two factors combined have most likely contributed to the present increase in mental health problems among white collar workers. Stress-related disease was in fact the single most common reason for sick-leave among white collar workers in Sweden in the year 2000 (Åsberg et al., 2002). When a physical stimulus generates stress it is called an environmental stressor. Fundamental to all environmental stressors is the fact that they, in a sense, lead to a certain degree of loss of personal control. They are perceived as threats and lead to stress reactions. Examples of environmental stressors are crowding, noise, disorientation, and environmental deprivation. When discussing the office environment and its psychological impact on office employees, it is important to acknowledge that the user of an office environment does not own the environment like the user of a home environment does. This leads to a different experience of office environments compared with home environments. Status has, for example, a great influence on office experiences as opposed to home experiences, since hierarchy is always measured in the physical setting by the different means available to organizational members. Certain environmental stressors that are of specific interest in an office context include: • Environmental deprivation – the experience of environmental deprivation is highly related to the cultural context. It occurs when environmental elements which mean a lot to the employee’s status or membership in the work force are taken away (Mazumdar, 1992). Mazumdar explains this as: ‘Members faced with environmental deprivation conditions experience loss of face and poise, loss of position, embarrassment, and shame. They feel deprived and not in control, which results in stress, anxiety and sometimes panic’ (Mazumdar, 1992, p. 703). Examples of elements that may cause environmental deprivation when lost are special furniture that grants the employee status, such as an exclusively designed desk or the loss of a corner window location in an open plan office. • Disorientation – in an office environment, the experience of disorientation does not mean that the employee will not find his/her way around the office. Disorientation occurs when the architecture of the workplace is hard to ‘read’ due to illogical plan layout or design features. It is considered a stressor, since finding it hard to orientate oneself causes individual frustration and in more serious cases loss of personal control. Disorientation can result in negative physical and psychological effects, such as hyperventilation, high blood pressure, etc. In my research I have found that office employees often describe a non-logical plan layout as something negative that causes irritation in their daily life; to find your way easily around the office appears to be something very important (Danielsson, 2005a). • Crowding – the experience of crowding is a psychological response to the perception of density. It is considered an environmental stressor since it may lead to physiological arousal (e.g. Aiello, Epstein and Karlin, 1975; Evans, 1979). Crowding is a result of: (1) a high density of people in insufficient space; or (2) an individual experiencing more social stimulation or interaction and interference with activities than desired (Stokol, 1972). Individuals that experience interference in privacy often report problems with crowding, thus the issue of privacy is closely related to crowding. Since office experiences, to a great extent, mean sharing office space and work facilities, these two issues are fundamental in the experience. When a physical stimulus is perceived as an environmental stressor it is taken care of by a coping strategy (e. g. Cohen et al., 1986; Lazarus, 1966). Personal control is a fundamental component in different coping strategies and exercised by different means. Control refers to autonomy. Personal control for people in the workplace may include

Office experiences

609

participation in the design process, the ability to control the closest work environment or being able to personalize it (Evans and McCoy, 1998). Concepts that are fundamental when discussing coping strategies are: • Privacy – a term used to describe anything from the need for space – visually and physically, via psychological separation, low population density and control over space – to freedom of activity. Privacy can be used as a coping strategy. It is a means to achieve personal control, thus it is closely connected to the psychological term crowding (Evans, 1979; Stokol, 1976). There are physical aspects in the office environment that determine the perception of privacy and crowding, such as partitions in space for example (Stokol, Smith and Prost, 1975). It can be used to mark privacy between office employees or different work teams in an office. Privacy is connected to status, as well as interaction and communications among individuals. It correlates with satisfaction with the workplace among employees (Sundstrom, Burt and Kamp, 1980). • Personalization – is another coping strategy that also gives a sense of ‘control of a place’, and which is often referred to in office research. Personalization refers to the deliberate adornment, decoration, modification, or rearrangement of an environment by its occupants. It serves many psychological functions at both an individual and group level. At an individual level it is used to express the employee’s personality (Donald, 1994; Scheiberg, 1990) and regulate the social contact (Altman, 1975; Brown, 1987), as well as demarcate a personal territory. Personalization can also be used to enhance attachment to a place (e.g. Goodrich, 1986; Harris, 1991) and to show the owner’s status (Wells, 2000). It is known to correlate positively with satisfaction with the workplace among employees (Sundstrom et al., 1980). At a group level, personalization seems to foster a sense of group identity, and possibly also commitment to an organization (Steele, 1986; Sundstrom, 1986). With regard to health and well-being there is research suggesting that personalization has a mediating effect on stress among employees (Scheiberg, 1990; Wells, 2000). There are many dimensions to personalization when it comes to office environments. Sundstrom suggests that ‘it expresses not only the individual’s self-identity, but also the amount of freedom and control the organization allows the individual to exert over the workspace’ (Sundstrom, 1986, p. 217). • Patterning – is yet another coping strategy used to handle stress caused by disorientation. It refers to the attempts to impose a structure or pattern on a seemingly chaotic and/or random constellation of stimuli or events (Wohlwill, 1973). Patterning is facilitated if orientation is made easy by a logical layout of rooms and by places that are made unique by design as well as by signs (Carpman and Grant, 1993, 2002; Weisman, 1981). Wener and Kaminoff (1983) have in their research showed that the presence of signs significantly reduces perceived crowding, discomfort, anger and confusion, which all are stress responses.

3. A FRAMEWORK TO UNDERSTAND OFFICE EXPERIENCES Organizational theorist Davis (1984) analyzes the physical environment in the office from a framework that is divided into the following three categories: (1) physical structure; (2) physical stimuli; and (3) symbolic artifacts. Through Davis’s division of the physical office environment the relation between the employees and the organization is emphasized, since it clearly shows the different means by which the physical environment extents its influence on the former. He suggests that these variables have a pervasive effect on managerial behavior. In this chapter, Davis’s framework is used as a

610

Product Experience

starting point for a discussion concerning the perception of the physical office environment and its influence on an individual level, as well as on an organizational level. The three categories are useful when approaching employees’ office experience of individual environmental factors and psychological concepts in relation to environmental factors that influence behavior and attitudes. Davis’s model (Figure 26.1) is also used in this chapter to describe the result of a study that I and a colleague have done on the office type’s influence on office employees with regard to their level of satisfaction with the physical work environment (Bodin Danielsson and Bodin, in press).

3.1. Physical structure The physical structure is defined by Davis as ‘the architectural design and physical placement of furnishings in a building that influence or regulate social interaction’ (Davis, 1984, p. 272). These structural arrangements may be permanent or subject to modifications and change. The research that deals with the physical structure of office environments has mainly been devoted to three areas: (1) building design and physical location; (2) furniture comfort, placement, and seating arrangement; (3) open plan versus closed office design. Office research has mainly investigated how individual environmental factors in office environment and different physical conditions in office environment affect the employees’ productivity and job satisfaction (e.g. Becker and Steele, 1995; Beehr, 1995; Brill, Margulis and Konar, 1985; Lu, 1999; Siegrist, 2003; Sundstrom et al., 1994). These studies have often been conducted to compare the employees’ attitude in cell-offices to those in open plan offices (e.g. Oldham and Brass, 1979; Sundstrom, Herbert and Brown, 1982; Sundstrom et al., 1994). These studies have been performed without any clear definitions of the design of the open plan offices, which is unfortunate since there are a variety of office types with an open plan layout. Consequently I am critical of the results, since clearly defined spatial and functional features are important when comparing different office environments to each other. The spatial and functional features are among the most important and influential factors in office environments (Danielsson, 2005b), as well as in other environments (e.g. Canter, 1976; Proshansky, Ittelson and Rivlin, 1976). When discussing experiences of office employees in open plan offices versus celloffices it is thus important to recognize the fact that there are different office types that have open plan layouts. There are great internal differences with regard to their features which might not appear at first, quick view; differences that appear to play a decisive part in employees’ office experiences. In an ongoing research project on office environment’s influence on employees I used the definition of the office type as a starting point for the employees’ office experiences (Danielsson, 2005b). The present study was conducted in 26 different companies in the

PHYSICAL STRUCTURE

PHYSICAL STIMULI

ORGANIZATION MEMBERS

SYMBOLIC ARTIFACTS

FIGURE 26.1 Davis’s model of the physical setting variables that influence behavior in organizations. (Source: Davis, 1984, p. 272).

Office experiences

611

Stockholm area with approximately 500 office employees in total. The seven identified office types used in the study show unique features with regard to architectural and functional features. The results show that these features together, to a great extent, determine office employees’ experiences of different environmental factors, as well as their psychological responses in relation to these. The defined office types are based on work by Ahlin and Westlander’s (1991) and Duffy(1999). The office types used are: 1. The cell-office, a single person room office. The plan layout is characterized by rooms along the facade of the building, connected by corridors. Every room has access to a window. Most office equipment is in the individual room. The office work is often highly concentrated and independent. 2. The shared-room office, defined by 2–3 people sharing a room. The shared-room offices are either a result of a team-based work organization that emphasizes interaction within projects or a consequence of a lack of space. In the latter case, the people nevertheless tend to have similar work assignments. The employee has no personal window. Most office equipment is outside the room, though the teambased shared-room offices sometimes have their own equipment within the room. Open plan offices These office-types are defined by employees sharing a common workspace. There are no walls between workstations and the employees have no access to individual windows. The work is often routine-processed with low levels of interaction between employees. The purpose of these office types is to be flexible to organizational changes and to handle these without any reconstruction. For reduction of noise and some privacy there are often screens between workstations. The open plan office exists in different varieties, depending on the number of people sharing workspace. The three different categories of open plan offices have the same architectural and functional features. It is the number of people sharing workspace that differentiates them. The three different categories of open plan offices used are: 3. Small open plan office, with 4–9 persons/room. Considered a good size for group identity (Svedberg, 1992). 4. Medium open plan office, with 10–24 persons/room. The most common size of open plan office in Sweden (Christiansson and Eiserman, 1998). 5. Large open plan office, with more than 24 persons/room. Not very common in Sweden. In addition there are office-types with more flexible characteristics: 6. The flex-office, defined by employees not having any personal workstation. It is often an open plan layout, but not necessarily. It is the most flexible office type; not only is the plan flexible, but also the employees. There is good access to back-up spaces for teamwork, concentrated work, meetings, etc. A good IT system is necessary since the choice of workstation is free and all work is dependent on access to the common computer system. The flex-offices are dimensioned for ⬍70% of the workforce to be in office, because much work is carried out outside of the office or employees are absent due to illness, etc. The work is in its character highly independent. 7. The combi-office, nowadays an office-type with no strict spatial definition; instead it is the teamwork and the sharing of common facilities that defines it. There is good access to back-up spaces for teamwork, concentrated work, meetings, etc. Over 20% of the work of employees takes place within the office at places other

612

Product Experience

than one’s own workstation, in the back-up spaces, on an ‘as-needed basis’. The work is characterized by both independent work and interactive teamwork. Office experiences related to office types Scientific knowledge of different office experiences can improve the design process and may help the involved parties in the design process to avoid mistakes. Thus it is important to find ways to transfer the scientific knowledge of office experiences into the design process. Guidelines may be a useful tool to accomplish this goal. Guidelines can at their best demonstrate how research-based design can be put into practice, and can work as theory in practice. A guideline for office design, however, needs to keep in mind the distinction between the two different groups of office experiences: Design-specific experiences; and general experiences. The two types of experience differ in their approach to office design and to specific problems. In addition, the two groups of experience provide different conditions for research feedback to the design process. Design-specific experiences are dependent on a unique condition in each specific office, determined by its architectural and functional features. It is the context that sets the framework for these experiences. They are to a great extent dependent on spatial conditions at a specific location. The experience of light is, for example, mainly a design-specific experience, since it to a great extent is determined by the office’s architecture – its location in the building, the placement of windows and lamps, etc. General experiences, on the other hand are not connected to a specific design of the office, but to the general conditions in the office. The experience of noise is a general experience; for example, when there are more people sharing a workspace there will be more noise. Thus the knowledge that there is a higher degree of disturbance by noise in an open plan office compared with in a cell-office is based on general experience proved over time. The two groups of experiences are highly connected to each other, since they have a mediating effect on each other. For example, dissatisfaction with a general experience such as ventilation and noise often influences the experience and perception of design-related factors. Design-specific experiences are of specific interest when discussing office types in relation to employees’ office experiences. This is due to the fact that architectural and functional features together define the different office types. It is however important to view the existing office types as prototypes since ‘intermediate versions’ always exist. The results of our study (Bodin Danielsson and Bodin, in press) show that there are great differences between employees in the different office types with regard to their satisfaction with the physical office environment and their psychological reaction to the physical environment. The differences between employees’ experiences in the different office types persisted in a lot of cases after adjustment for important background factors such as age, gender, job rank, and line of business. Environmental stressors and coping strategies that are highly connected to design-specific experiences include, for example, the possibility to personalize the office and the perception of privacy. These effects are to a great extent determined by the architectural context, the way the office work functions, as well as the way the office space is shared among employees. Work environment problems related to general experiences can be handled in the design process by general solutions, regulations, and specified demands. Design-specific experiences, on the other hand, are determined by the conditions at a specific place and must be handled by case-specific solutions. The successfulness of a solution to a designspecific problem depends on how well the designer or architect has understood the context, and how well he or she has adopted the solution to the specific problem. Knowledge of how to deal with design-specific experiences is transferred to different parties in the design process by good examples and instructive cases to similar problems. The knowledge of

613

Office experiences

how to deal with problems related to general experiences and design-specific experiences are part of the same tradition. However, the conditions and the way in which problems are handled differ. Being aware of these ‘two sides of the coin’ of office experiences is necessary for developing a research-based design for office environments.

A

B

FIGURE 26.2 (A) The cell-office, a single person room office.

A

FIGURE 26.3 Open plan offices. (A) Small open plan office, with 4–9 persons/room.

(B) The shared-room office, 2–3 persons (Bodin Danielsson and Bodin, in press) sharing an office.

B

(B) Left, Medium open plan office, with 10–24 persons/room and Right, Large open plan office, with more than 24 persons/ room. (Bodin Danielsson and Bodin, in press)

FIGURE 26.4 Flex-office and Combi-office. Office-types with more flexible characteristics. Both office types are defined by good access to back-up spaces for teamwork, concentrated work, meetings, etc. (Bodin Danielsson and Bodin, in press).

614

Product Experience

Office type in relation to design-related factors Office employees’ experiences of design-related factors are in the current study investigated according to the following categories: (1) workstation design; (2) workspace design; and (3) office design (Bodin Danielsson and Bodin, in press), which can be related to the physical structures in accordance with Davis’s model (Davis, 1984). These experiences are design-specific experiences since design-related factors are highly determined by the architectural and functional features at the specific office. The location of a workstation in relation to common areas such as lunch and break areas, as well as location of the copy machines will, for example, determine the degree of interaction and spontaneous meetings among colleagues. The architectural design of the workstation, the workspace and the office as a whole will also determine if natural meeting places, so called ‘activity nodes’ (Bechtel, 1976) will appear in an office. When the experience of design-related factors in the office is investigated all together in relation to office-type, the overall results show that cell-office employees are the happiest of all office employees (Bodin Danielsson and Bodin, in press). When we look closer at each category of design-related factors a varied picture appears between employees in office-types that at first do not appear to be that very different to each other in their characteristics, such as the different open plan offices for example. Design-related factors turn out to be a category which employees are dissatisfied with to a great extent. The overall analysis of design-related factors shows that the relative risks for complaints are significant among employees in all office types except the cell-office. The highest relative risks for complaints are found among employees in medium open plan offices, followed by those in large open plan offices (Bodin Danielsson and Bodin, in press). Workstation design In terms of the category workstation design the cell-office employees are conspicuously more satisfied in all regards. The least satisfied group consists of employees in medium open plan offices, followed by those in large open plan offices and flex-offices. The experience of not having enough space for work material is commonly shared among employees in medium and large open plan offices, as well as those in flex-offices. Surprisingly, the employees in small open plan offices, an office type that only differs from medium and large open plan offices by the number of people sharing workspace, are more satisfied with the space for working material at their workstations. A hypothesis is that the great satisfaction of general comfort and ergonomics at the workstation among employees in small open plan offices has a mediating effect on the whole workstation experience. Employees in small open plan offices are just as satisfied with this aspect of the workstation design as those in cell-offices. Most dissatisfaction with the general comfort and ergonomics at the workstation is found among employees in medium and large open plan offices, as well as in flex-offices. The dissatisfaction with the ergonomics at the workstation among employees in flex-offices employees can most likely be explained by the fact that there are no personal workstations in this office-type, thus there is less possibility for individual adjustments at the workstations. A result that, on the other hand, is surprising is the fact that as many as 30% of the employees in medium open plan offices experience that the workstation design does not support the work to be carried out at the workstation. This is alarming since the aim of the workstation at the bottom line is to support work. Workspace design When it comes to office employees’ experience of workspace design employees in celloffices are somewhat more satisfied than employees in other office types. Least satisfied

Office experiences

615

with the workspace design are employees working in medium open plan offices (Bodin Danielsson and Bodin, in press). There is, however, one aspect the employees in the cell-office are least satisfied when it comes to workspace design compared with employees in other office types – the workspace design’s support of affinity among colleagues. Employees in all other office-types find the workspace design more supportive than those working in cell-offices. Most satisfied with this aspect of the workspace design are employees in small open plan offices and flex-offices. With regard to the contribution of workspace design to job satisfaction the employees in cell-offices are most satisfied, while the least satisfied group of employees are those working in medium open plan offices once again. They are in this respect remarkably less satisfied than any other group of office employees. Office design Office design determines to a great extent the degree of spontaneous and informal meetings; it is known that employees tend to have these types of meetings with the people that are most physically accessible at the office (Sundstrom, 1986). When it comes to formal meetings the case is different. The design is then less important since people will attend an appointment independently of where the meeting will take place in the building. Since spontaneous and informal meetings are important for interaction and creativity among colleagues, office design has become an increasingly important tool for organizational success and survival. Satisfaction with office design among employees in different office types is measured by the office design’s reinforcement of interaction among employees, pleasantness of spaces for breaks and lunch areas, as well as the employees’ perception of the physical work environment in general (Bodin Danielsson and Bodin, in press). The overall picture of the office design experience in different office types shows that employees in celloffices, shared-room offices and flex-offices are equally satisfied with the office design. They are, however, satisfied with different aspects of office design. Employees in medium open plan offices are least satisfied with office-design altogether, followed by those in large and small open plan offices. The employees in cell-offices and shared-room offices are both satisfied with the same aspects of office design – the general physical work environment and the lunch areas. Flex-office employees experience other aspects of the office design as superior, such as the design’s reinforcement of interaction among employees and spaces for breaks. The least satisfied group of employees, medium open plan employees, is most dissatisfied with lunch areas and general physical work environment when it comes to office design. The importance of the workstation One category of the design-related factors, the workstation design, needs to be discussed further, since nothing is as important as the actual workstation for the individual employee’s experience of the physical office environment besides ambient factors (Sundstrom, 1986). The workstation itself consists of different physical objects such as furniture and office equipment. The perception of the individual workstation is dependent on both design features and location. These factors combined determine the employee’s perception of different environmental factors at the workstation. These factors should support the work carried out at the workstation and, most importantly, should not inhibit the employee’s work. The research conducted on workstations has mainly been concerned with different components from an engineering and ergonomic perspective or dealt with highly

616

Product Experience

specialized workstations. The aim has often been to find general guidelines for ideal workstations (McCormick, 1976) – which has turned out to be hard to do – instead of looking at psychological and behavioral implications of the workstation design. According to two surveys on office employees conducted by Harris and Associates in 1978 and 1980 the most important feature of the workstations, from an office employee’s perspective, is access to good equipment. Then come other features such as the ability to adjust the workstation and access to work related storage and working surfaces (Sundstrom, 1986). According to the same survey, the chair is one of the most important elements of the workstation, and at the same time one of the aspects employees are most satisfied with. The workstation furnishing tends to be standardized. Well known is a standardized modular-system for office furniture called ‘cubicles’, which is characterized by screens around the individual workstation. This system grew very popular in the United States with the introduction of open plan offices from West Germany in the 1960s. At the end of the 1990s the trends in office work started to move towards a more mobile, complex, and plural work. This trend has influenced furniture design at workstations, as well as the whole physical office environment. This has lead to furniture like modular units used on an as-needed basis for teamwork. These solutions for workstations are often found in open plan offices that are defined as flex and combi-offices in the office-type classification. This contemporary trend can, according to Duffy (1999), be called an ‘activity directed design’ of furniture and workstations, as opposed to the individually directed. There are, however, simultaneous trends in workstation design. In Sweden it has lately become common for desks to be adjustable in height, which allows the employee to raise and lower the worktable depending on the need. This is a means to easier meet the individual’s ergonomic needs at a personal workstation, as well as at shared workstations. The location of the workstation determines physical accessibility and lighting condition to a great extent. The actual design of the workstation determines the physical accessibility as well. The access of space at the workstation affects the ergonomic aspects greatly, since it determines the individual’s ability to stretch out and change working position. It is also known that workstations located too close to each other may lead to frustrations and experiences of crowding among employees, which has well documented effects on the individual stress levels as well as social behavior (Altman, 1975; Evans, 1979). Due to these factors the floor space available at the workstation is an important aspect in the workstation design. According to a study by Langdon conducted in 1966 (Sundstrom, 1986) it leads to dissatisfaction among office employees if the area goes below 60 square feet (approximately 18 m2) per employee. However, when discussing floor space one has to bear in mind that its value as a status symbol may also have an impact on the individual’s satisfaction (Sundstrom, 1986), as well as on individual stress symptoms. In this context one also has to consider cultural differences. There are very different national standards for minimum space at workstations. In Sweden traditionally larger workspaces have been used per employee than in, for example, the USA or Japan (Duffy, 1999).

3.2. Physical stimuli The environment interacts with the individual through physical stimuli, which are perceived and evaluated by the individual. Davis uses the term physical stimuli for those factors in the physical setting that intrude into the organization members’ awareness and influence their behavior (Davis, 1984). A host of physical stimuli in the daily office

Office experiences

617

environment compete for the employee’s attention. When a physical stimulus is perceived as a threat it is called an ‘environmental stressor’ in stress research, as formerly described. Since physical stimuli can arise and activate different behavior, they deserve attention in theories of organizational behavior, especially for those that deal with motivation and goal setting (Davis, 1984). Physical stimuli are known to have both positive and negative influence on the individual. The individual may react to physical stimuli by aroused psychological and physiological reactions and possibly also by certain behaviors (Porter and Lawler, 1965). The interaction with features and objects in an environment can be evaluated as levels of arousal, adaptation, fatigue, stress, safety, and security. At the office this is expressed by employees’ low job satisfaction, frustration and difficulties in concentrating on work. Examples of physical stimuli that may be perceived as environmental stressors in office environments are conversations, telephones ringing, e-mails, different objects and artifacts in the room, as well as the presence of colleagues or the supervisor. When office employees experience too many physical stimuli, they may cause a decline in concentration. At a group level, reactions toward physical stimuli can be evaluated as levels of communication and collaboration, status and identity, as well as crowding or privacy (Mitchell McCoy, 2002). The strongest physical stimuli in office environments are light, noise, temperature, and air quality. This group of environmental factors is often referred to as ambient factors. Physical stimuli by other environmental factors, such as colors and artifacts, have less physical but more symbolic and psychological effects on employees. They will therefore in this chapter be treated as symbolic artifacts in accordance with Davis’s theory (Davis, 1984). Physical stimuli from ambient factors such as temperature and ventilation can be classified as belonging to a general experience since they are not necessarily dependent on specific contextual factors in an environment. Instead they depend mainly on technical solutions, and there are general solutions, so called ‘cook book’ solutions, available to deal with problems involving these factors. The experience of light, which also is an ambient factor, on the contrary, is mainly a design-specific experience, since it is more dependent on the architectural context and the specific design solutions at a particular place than technical solutions. The disturbance by noise, which is a general experience, can often be adapted by general solutions. In a large open plan office, a general type solution to a noise problem would be to use textile flooring or acoustic panels. The experience of noise can, however, also be related to the architectural and functional features of a place, because noise is closely related to privacy and crowding. Thus if general type solutions to a noise problem in an office are not enough to solve the problem, it might be necessary to look at design-specific solutions to the problem. An example of the latter would be to design different rooms for different work assignments in order to avoid unnecessary disturbance. Human behavior, as well as psychological and physiological reactions in the work environment, is difficult to investigate, because there is a complex interaction between the individual, the physical work environment and social interaction with colleagues. An example of this complexity is provided by the general experiences of ambient factors like temperature and ventilation. These factors are not only important because they influence employees’ experiences directly, but also because they influence satisfaction with other environmental factors (Franzén, 1969). Besides their influence on environmental satisfaction they also influence the employees’ satisfaction with leadership in an organization (Sundstrom, 1986). An experience of frequent fluctuations in temperature and bad air quality seems to lead to a decline in job satisfaction (BOSTI, 1981). These

618

Product Experience

factors combined make them important not only to the individual employee, but also to organization management, since they unavoidably influence the behavior of the employees, intentionally or unintentionally (Davis, 1984). However, when office experiences are analyzed it is important to acknowledge that environmental factors cannot be studied in isolation from each other, due to their mediating effect on each other. Nevertheless they are described individually to provide a better overview below. Noise and sound Noise is known to be one of the most important environmental factors in office environments. It is widely regarded as a source of dissatisfaction and an impediment to performance. It is the most common source of complaints in office environments, particularly in open plan offices. Noise is by definition unwanted sound and is therefore often perceived as uncomfortable and stressful. Thus noise is often classified as an environmental stressor in office environments, since it may generate stress. Despite this, noise is not necessarily threatening to the occupants’ health and well-being in office environments (Vischer, 1996). What makes noise particularly interesting in office environment is not only the fact that it is the most common reason for dissatisfaction among office employees, but also that there is a significant correlation between disturbance by noise and dissatisfaction with the office environment (Nemecek and Grandjean, 1973). Sound, unlike noise, is not perceived as disturbing and can in fact even be stimulating. For example, some sounds generated by colleagues can make the employee feel that he/she is not working in isolation. It can stimulate performance during brief work sessions, which could possibly be explained as a positive distraction of attention (Sundstrom, 1986). The positive influence of sound from others on performance is known among telemarketing employees; consequently, they are often placed close to each other. It is hypothesized that more extrovert personalities may prefer more stimulation in general, whereas others may find noise detrimental to reading comprehension (Standing, Lynn and Moxness, 1990). In other words, the human reaction to noise/sound is dependent on the individual. The purpose of a certain noise, the possibility to foresee and control it, as well as the attitudes towards the noise source, affects the grade of annoyance. The acceptance and tolerance of noise is greater if one believes that one cannot eliminate it, for example noise caused by traffic (Byström, 1999). The most disturbing noise is not always the loudest. Research shows that colleagues’ conversations, as well as telephones ringing, are more disturbing than noise from office equipment and traffic. As a consequence, it has been suggested that noise carrying meaning and information is most disturbing to office employees (Sundstrom, 1986). With regard to performance and noise the results are contradictory. Noise seems to interfere with work performance; it has a more negative effect on complex tasks than on simple tasks. Type of work assignments affects the grade of annoyance as well; it is easier to get annoyed when working with difficult work assignments. The tolerance threshold decreases during difficult work assignments and already at a level of 35dB it is significant (Franzén, 1969). In a study among fourth grade pupils in an open space school Weinstein and Weinstein (1979) could not, however, reveal any reduction in performance during higher noise sessions compared to quiet sessions. This could possibly be explained by the young age of the pupils, since there is research which shows that age has a significant impact on how easily one is disturbed by noise (Byström, 1999). However, there is also research which suggests that disturbance by noise is independent of age, gender, and education (Canter and Stringer, 1975). When discussing noise and health, low frequency noise is of specific interest. It is proved to have a negative influence on the stress level (Bengtsson, 2003). People also

Office experiences

619

tend to get tired easier under the influence of low frequency noise than under the influence of other noise. It is hard to get used to and to ignore it. Crowding and perception of privacy The experience of crowding is an environmental stressor that may be experienced among employees in offices with an open plan layout, where the employees share workspace as well as work facilities to different extents. The experience of crowding correlates to a high degree with the perception of privacy, thus they should often be considered in connection to each other. The perception of cooperativeness among colleagues at the office seems to be influenced by crowding. Employees perceive their colleagues as less cooperative and more competitive in crowded conditions (Freedman, Klevansky and Ehrlich, 1971). In addition a less social behavior among people has been found as a consequence of stress caused by high density. Spatial factors such as room partitions (Stokol et al., 1975) and division of corridors (Baum and Davis, 1980) seem to have a mediating effect on these stress symptoms. There is also research which shows that architectural details and craftsmanship have a mediating effect on stress symptoms such as crowding (Wochel and Teddlie, 1976), as opposed to more sterile environments that enhance stress symptoms, such as complaints and less social behavior (Mazumdar, 1992). With regard to performance, some studies have found associations between crowding and poor performance (Sundstrom, 1986). High density seems to reduce the ability to perform complex tasks (Evans, 1979). According to a study by Sundstrom et al. (1980) crowding is also associated with dissatisfaction with the physical environment among employees. When discussing crowding and privacy, it is important to remember that less social contact than desired may lead to an experience of isolation between colleagues. Density to some degree in an office setting can be positive and enhance performance – a phenomenon called ‘social facilitation’ (Sundstrom, 1986, p. 294). Office type in relation to noise and privacy The experiences of noise and privacy, when analyzed in relation to an employee’s office type, show that cell-office employees are most satisfied with the noise conditions at the office (Bodin Danielsson and Bodin, in press). The results show great differences between office-types where employees share work space and work facilities. The results indicate that it is not the mere fact that sharing implies noise, but that differences are caused by how the office types handle noise by their architectural and functional features. The highest incidence of complaints about noise is found among the employees in the larger open plan offices, followed by those in medium open plan offices. Employees in shared room offices, small open plan offices and flex-offices show much lower incidence of complaints compared with the former. This can most likely be explained by the actual size of the group sharing workspace and facilities. In the case of flex-offices the low number of complaints is probably explained by the free choice of where to work, as well as by the good access to ‘back-up rooms’, which seems to have a mediating effect on noise disturbances. The former explanation is based on a functional feature of the office type and the latter on an architectural feature of the office type. Despite the close relation between noise and privacy, the results show no absolute correlation between the two. The office types where employees report most problems with the noise conditions are not always the same as where employees have problems with privacy issues. For example, employees in all open plan offices combined with those

620

Product Experience

in flex-offices report a lack of seclusion in the workspace. However, when it comes to complaints about being overheard or observed in the office among employees in different office types, a different picture appears. Employees in flex-offices experience no problem with these issues, whereas employees in small and medium open plan offices report the largest number of complaints with being observed and those in large open plan offices the largest number of complaints about being overheard. Those in flex-offices report less disturbance with noise compared with other open plan layout offices. A possible explanation for the flex-office lack of problem in these regards is that this office type is likely to yield less privacy and noise problems because of the ability to choose workstations by preference since there are plenty of ‘back-up spaces’. This officetype also offers the possibility to work from home, to some extent. The high satisfaction among employees in cell-offices is easier understood considering that this office type, due to its features, offers most privacy. Ambient factors: Temperature, air quality and lighting There seems to be a great individual sensitivity with regard to temperature. In fact, a substantial number of office employees find the temperature either too high or too low. Nevertheless, the ideal temperature condition for most employees seems to be 21ºC (70ºF) (Sundstrom, 1986). Small departures from the range of comfort can easily create dissatisfaction with temperature. It seems as if individual preferences in comfort can neither be attributed to gender, age, or geographical origin (Griffiths, 1975). Good air quality implies a moderate air movement, good humidity, and freedom from pollution. In office environments, bad air quality normally means that the air is not changed frequently enough and thereby perceived as stuffy. The ventilation requirements depend on a lot of factors, such as the population density, geographical position, season, building materials and plan layout (e.g. Franzén, 1969; Woodson, 1981). Polluted air is normally not a problem in office environments. Light has a significant influence on the perception of the physical environment. Our eyes are drawn towards light. Different types of lighting reinforce or weaken the perception of color. The eye has a great ability to adjust itself to different lighting conditions; as a result of this there are relatively few complaints about lighting among employees, even under obviously bad lighting conditions (Vischer, 1996). The lighting condition in an office is determined by natural daylight conditions as well as artificial lighting conditions. The former is dependent on the architecture and thus a design-specific experience. The experience of artificial lighting on the other hand is more of a general experience, since it is dependent on general solutions such as applied technical solutions and quality of artificial lighting. Our behavior and well-being is affected by the amount and quality of light. This is explained by the fact that the sleep hormone melatolin which influences our level of arousal and alertness, is affected by light (Ejhed and Liljefors, 1990). Also, the level of stress hormones, which at high levels influence sociability negatively, is also affected by light (Küller and Lindsten, 1992). Thus the access to light has a great influence on how employees behave in an office. The preference for windows among office employees is notable, and natural lighting in a room is perceived as something positive, and it is a source of satisfaction with the physical environment among office employees (Sundstrom, 1986). There are studies that suggest that the quality of light affects the job satisfaction among employees as well as the satisfaction with the physical work conditions (e.g. Ejhed and Liljefors, 1990; Finnegan and Zener Solomon, 1981; Küller and Lindsten, 1992). With regards to performance and light there are contradictory results; though many believe that natural

Office experiences

621

light, or daylight, is superior compared to electric light for work performance, there is scarce research to support this (Mitchell McCoy, 2002). Views The view from the individual workstation is, according to Davis’ model (1984) a physical stimulus, since it influences the office employee’s psychology and behavior. A view from a window towards nature is known to have positive impact on human well-being and satisfaction, thus the benefit from windows is not only natural daylight in office environments. Kaplan and colleagues (1988) found in a survey of three groups of office employees that those with an outdoor view including only built components, such as roads or buildings, had higher levels of job stress than others. Employees who could see at least some natural elements, such as trees and grass, showed higher job satisfaction levels than those with views of built elements outdoors or with no outdoor views from their workstations at all. Research has also shown that employees in windowless offices feel more restricted and tense (Ruys, 1970). Several studies have shown the positive effect of ‘natural environments’, such as forests and parks, on humans in different settings (Moore, 1982; Ulrich, 1984; Wilson, 1972). One factor highly appreciated by employees is contact with the external environment through windows, since this enables them to see how the weather is, how seasons change, etc. It has been observed that employees in windowless offices tend to decorate their workspaces in preference for more ‘natural’ themes, in comparison to office employees with access to windows (Heerwagen, 1990; 1986). The reason for dissatisfaction among employees in windowless rooms is not necessarily only environmental, but could also reflect social issues, e. g. status. Windows are well-known status symbols in office design and status is known to have a positive influence on satisfaction with physical office environment among employees (Sundstrom, 1986). Office type in relation to ambient factors In the present study office employees’ experiences of ambient factors such as temperature conditions, ventilation and lighting conditions are investigated in relation to office type. There are overall low incidences of complaints with these environmental factors among the employees in this study (Bodin Danielsson and Bodin, in press). The result should not be interpreted as if these factors are not important in office design, but rather that today we have the technical knowledge to handle these issues. The most satisfied group of employees is once again those in the cell-office. This is not surprising since the individual can control lighting, as well as the temperature and ventilation condition to great extent in the individual room in accordance with personal preference. The highest incidence of complaints with regard to these aspects is found among employees in shared-room offices, which is an office type that in other aspects does relatively well. The reason for employees in shared-room offices to report dissatisfaction could possibly be ascribed to the fact that a lot of shared-room offices are originally designed to be cell-offices for one individual. Due to lack of space, two to three individuals have to share one room. Another explanation could be that it is harder for two to three individuals to agree on a matter such as temperature than a larger group of people. The reason for this is that it is easier to accept a decision when one is not believed to have an impact, which is the case in a larger group.

3.3. Symbolic artifacts For those aspects of the physical setting that individually guide the interpretation of the social setting Davis uses the term symbolic artifact (Davis, 1984). For instance, the type

622

Product Experience

and style of the furnishing, the colors of the walls, the presence or absence of carpeting, framed certificates or photographs displayed on walls or desk all ‘communicate’ information about the organization and the people who work there (Davis, 1984). The experiences of these symbolic artifacts are to a great degree contextual. So is, for example, the experience of colors combined with the design of the office and its interior interpreted by the user in relation to the physical context, the architecture of a place. Experiences of colors are design-specific experiences since they are highly dependent on the lighting condition and materials in a room. There is a general aspect to experience of symbolic artifacts as well. For instance the use of exclusive wood in office furniture is generally interpreted as a status symbol. However, in a context where exclusive wood does not fit the setting, the signal will be counterproductive of the intention. Symbolic artifacts ‘communicate’ with their observers/users and they are frequently subject to multiple interpretations of both intended and unintended consequences (Davis, 1984). Symbolic artifacts are strongly associated with status cues and images of organization, internal as well as external, thus used as tools in design management. It is not unusual that hierarchical differences are reinforced by physical differences between offices occupied by officials of various degrees of grandeur (Duffy, 1978). Knowledge about the possible impact of symbolic artifacts has been used in the architectural design of banks, insurance companies and law firms. They use their offices to complement or confirm their professional status, as well as to meet the needs of their clients for comfort, security, and confidentiality. Administrative offices are, on the contrary, primarily set up for transacting business activity, where efficiency and access to information is of central concern. In the use of symbolic artifacts in office design it is important to remember not to convey conflicting messages with regard to the function of the office, neither to the employees nor to the clients (Davis, 1984). The traditional symbols of status in office environments are: Locations accessibility; floor space; furnishings; and freedom of personalization (Konar et al., 1982; Steele, 1973). Accessibility is a well-known status marker; the less accessible a person is, the more important he or she is. A physically enclosed office in an open plan office signals high rank and status. Floor space is also a well-used symbol of status, both in cell-offices and in open plan offices. The more floor space a person has access to, the higher the rank. Furnishing can operate as a status marker as well, by elements such as carpeting, draperies, artworks, sofa, coffee table, etc., (Davis, 1984). The location of an office in a building also signals status. Traditionally the top floors in a building have been occupied by the president and the board of a company. The actual location of workstations within the individual office may also be used as a status marker. For example, corner rooms have been attributed the highest status in a cell-office plan layout. The degree of freedom to personalize workstations by personal objects such as photos, plants, etc., may also be interpreted as a status marker in an organization. In recruitment of low paid office workers like clerical staff, the conditions that surround the job such as modern furnishing, nice office building, nice restrooms, etc., may be a primary reason for joining a particular organization or continuing working for a specific firm (Davis, 1984). Colors The experience of an environmental factor such as color at the workplace is a symbolic artifact in accordance with Davis’s model (Davis, 1984). Colors hold a lot of symbolic values that often lead to interpretations. The experience of colors is closely connected to the experience of light. It belongs to design-specific experiences, since the physical structure, the spatial context and the access to or lack of windows determines the experience of colors. The perception of color is determined by the lighting conditions,

Office experiences

623

which have a great impact on how the atmosphere of a room is perceived (Dahlin, 1999). Whether there is natural daylight through a window or artificial light also influences the perception of color. Colors affect us emotionally and to some extent also physically (Küller, 1995) though it seems like the perception of color is more of a cognitive process than a physiological process (Dahlin, 1999). The reason for red being associated with warmth could be that fire is warm and that, therefore, the color red is perceived as warm. Despite the fact that we like to make a distinction between the characteristics of colors, it is not the color itself but its intensiveness that determines how stimulating we find it. Therefore intense green or blue colors are perceived as equally ‘activating’ as intense reds, whereas a lighter red is perceived as equally ‘passive’ as a light green or blue (Sivik, 1995). This cognitive perception of color can be used in office design to reinforce different experiences, experiences that are aimed. ‘Warm’ colors can be used to encourage social behavior, and ‘cold’ ones can be used to achieve a calming effect. One has to be aware though, that in spite of some colors being described as ‘warm’ and ‘creative’ no such influence has been identified in creativity tests (Mitchell McCoy and Evans, 2002). There is only limited scientific proof with regard to the influence of colors on satisfaction and performance. Sundstrom (1986) suggests, however, that colors can contribute to employees’ satisfaction with the physical environment by creating favorable attitudes. Such attitudes depend on personal preferences and on attitudes toward colors, as well as on contemporary trends. Artifacts and artwork The knowledge of how physical objects such as artifacts and artwork influence human psychology and behavior, is limited. It is, however, believed that they can reinforce the human identification with a place by making it unique in design and architectural features. It is even hypothesized that ‘design features that provide little or no feedback evoke negative reactions’ (Evans and McCoy, 1998, p. 88). Artwork and artifacts are generally considered to provide an opportunity to personalize workstations within the workspace, and by these means give a sense of ‘aesthetic control’ of a place. This ‘aesthetic control’ can be used in an organization to reinforce the sense of coherence at both an individual and at a group level. These aspects are important in office environments since they are connected to satisfaction and work performance (Festinger, 1957; Mitchell McCoy, 2002). According to Mazumdar’s research (1992), employees in office environments that contain little or no architectonic detail indicate more deprivation, which affects a range of behaviors and psychological responses, such as distancing oneself from other members, or groaning and complaining to reduce anxiety. These are all symptoms of stress, thus in this context lack of architectural detailing and artwork is an environmental stressor. The calming effect of certain kinds of artwork in stressful situations has been studied by Heerwagen and Orians (1990) among patients at a Dental Fears Clinic. Their study showed that people in a waiting room with a wall depicting a landscape felt less tense and had less increase in heart rate than those in a plain waiting room. With regard to the influence of artifacts and artworks on creativity, most of the research is conducted in school settings and not in office environments. Wollin and Montagne (1981) found that students in classrooms that had softer lighting, plants, posters, cushions, and rugs scored significantly better in exams than students in a standardized, non-modified classroom. Similar tests have shown that students participate significantly more in discussion in a classroom with a ‘softer’ design than in a normal classroom (Sommer and Olsen, 1980). It also seems as if the architectural detailing and

624

Product Experience

the choice of material have a positive influence on creativity. Incorporating natural materials, not necessarily exclusively natural materials, seems to hold potential for increasing creativity (Mitchell McCoy and Evans, 2002). Natural material is defined by Mitchell McCoy and Evans as any elements grown or mined from the earth, though it may have been finished or enhanced (e.g. wood that has been lacquered and polished but retains enough of its visual and textual quality to easily be identifiable as natural). The positive result among students in the soft classroom compared to those in the normal classroom might also be an effect of artwork and artifact functioning as status markers, which are known to have positive influence on performance, as well as on coherence in organizations (Sundstrom, 1986). Office type in relation to symbolic artifacts In the current study (Bodin Danielsson and Bodin, in press) we have not looked specifically at the experience of symbolic artifacts as classified by Davis (1984). Nonetheless, the experience of design-related factors, which are described in the former section about physical structure, touches the field of symbolic artifacts. In fact, the perception of design-related factors can, to a great extent, be described as stimuli by symbolic artifacts. The result shows that the possibility to personalize the workstation is highest among employees in cell-offices. This is most likely explained by the fact that in this office type the individual can often determine which picture to hang on the walls in his or her room and how to place furniture in the room, etc. The least possibility for personalization is reported among employees in flex-offices, because office-type is defined by absence of individual workstations. However, when looking at how supportive the employees in different office types rate the design of the workstations with regard to different aspects related to work and social interaction, a different picture appears. In terms of overall satisfaction with the workstation, employees in medium open plan offices run the highest risk for being dissatisfied. The employee’s experience of his/her physical work environment is a total experience, consisting of experiences of different, individual aspects of the physical environment. The satisfaction with the physical environment at the office is thus rated in the study by different items in different categories of the physical work environment. It is rated by satisfaction with the general physical work environment, the physical environment’s contribution to job satisfaction and with the physical environment’s support of affinity among colleagues. The results show that employees in medium open plan offices are the least satisfied with the design-related factors in total at the office. Most satisfied with workspace design are cell-office employees. They are, however, the least satisfied of all employees with the workspace design’s support of affinity and interaction with colleagues. When it comes to social aspects of design-related factors, the cell-office rates low and the employees in flex-offices are the most satisfied. The possibility for personalization does not correlate fully with the satisfaction with design-related factors in the physical environment among office employees. It is, however, the group of employees who have the greatest possibility for personalization, cell-office employees, that are overall most satisfied with design-related factors. This can partly be explained by the fact that personalization is a traditional status marker and a sign of personal control over a territory, which has a positive influence on the attachment to a place. It seems as if personalization and personal control over the individual room in this case compensate for the lack of socially reinforcing design-related factors. When it comes to satisfaction with the office design’s support of social activities and interaction with colleagues, symbolic artifacts of a personal nature are less important. Since the survival of a

Office experiences

625

company on a competitive market depends highly on exchange of information between employees, the low rating in cell-offices in this respect is a serious drawback. The overall good results for cell-offices on satisfaction with design-related factors can also be related not only to the design-specific experience determined by the architectural and functional features, but also to the symbolic value of an individual room. An individual room is traditionally rated as a high status symbol and status, as well as privacy, correlates with employees’ satisfaction with the physical work environment (Sundstrom et al., 1980).

4. DISCUSSION The workplace includes such fundamental psychological factors as control and nonverbal self-expression, combined with functional factors like work opportunities and sharing of space and facilities with colleagues. Combined these aspects create the setting of the office work and influence office employees’ behavior and experiences. Despite the fact that many environmental factors in the office are the same as at home, they communicate a different meaning, due to the social context. The environmental factors also have to be considered in relation to individual factors, such as personal preferences and personalities, combined with the actual work carried out at the specific office. The complexity of environmental stimuli in office environments makes it possible to approach office experiences from different perspectives. In this chapter a model developed by organizational theorist (Davis, 1984) has been used to better understand how the office environment influences the employees and the organization as a whole. In addition, a classification of office experiences has been used. The latter classification is useful in the design process of offices, to analyze and handle problems connected to office experiences based on the knowledge of their origin. The two classifications: (1) designspecific experiences; and (2) general experiences, depend on the nature of the experiences and their components. Knowledge of these two different groups will lead to different solutions to a problem in an office which it is important to be aware of in the design process of office environments. The former group has case-specific solutions to problems related to it. The quality of these solutions is highly dependent on the skill of the individual architect/designer, based on experiences from similar cases, as well as the creativity of the architect. Whereas problems related to the latter group of experiences, the general experiences, are solved by general solutions, so called ‘cook book’ solutions, described in type-solutions, regulations and specific demands in programs of different kinds. The individual architect/designer has to have the insight of these general solutions to solve this type of problem. Only with scientific knowledge of the differences between the different office experiences can the solutions be found. Thus I want to stress the importance of transferring scientific knowledge into the design process. Guidelines for a research-based design are excellent transaction tools, since guidelines are theory put into practice. In my opinion, a special focus should be put on architectural and functional features in this context. The results of the study here presented show great differences between employees’ office experiences in different office types, which most likely can be ascribed to the architectural and functional features that define the office-types (Bodin Danielsson and Bodin, in press). In a larger context, it is important to recognize employees’ office experiences due to their influence on employees’ satisfaction and motivation, which are important factors for organizational success and survival.

626

Product Experience

REFERENCES Ahlin, J. and Westlander, G. (1991). Kontorslokaler och kontorsarbete – två perspektiv på kontoret som arbetsplats (Office spaces and office work – two perspectives on the office as a workplace). Solna, Sweden: Arbetsmiljöinstitutet (The Swedish National Institute for Working Life). Aiello, J. R., Epstein, Y. M. and Karlin, R. A. (1975). Effects of crowding on electrodermal activity. Sociological Symposium, 14, 42–57. Altman, I. (1975). The environment and social behavior: Privacy, personal space, territory, crowding. Monterey, CA: Brooks/Cole. Åsberg, M., Nygren, Å., Rylander, G. and Rydmark, I. (2002). Stress och utmattningsdepression (Stress and exhaustion depression). In: R. Ekman and B. Arnetz (Eds.) Stress – molekylerna, individen, organisationen, samhället (Stress – the molecules, the individe, the organization, the society). Stockholm: Liber. Baum, A. and Davis, G. E. (1980). Reducing the stress of high-density living: An architectural intervention. Journal of Personality and Social Psychology, 38, 471–481. Bechtel, R. (1976). Enclosing behavior. Stroudsburg, PA: Dowden, Hutchinson and Ross. Becker, F. (1981). Workspace: creating environments in organizations. New York: Praeger. Becker, F. and Steele, F. (1995). Workplace by design, mapping the high-performance workscape. San Francisco, USA: Jossey-Bass. Beehr, T. A. (1995). Psychological stress in the work place. London: Routledge. Bengtsson, J. (2003). Low frequency noise during work – effects on performance and annoyance. Unpublished Ph.D. Thesis, Sahlgrenska Academy, Göteborgs University, Göteborg, Sweden. Bodin Danielsson, C. and Bodin, L. (in press). Office environment and employees satisfaction: The impact of office-type. Journal of Architectural and Planning Research. BOSTI (1981). The impact of office environment on productivity and quality of working life: Comprehensive findings. Buffalo, NY: Buffalo Organization for Social and Technological Innovation. Bramel, D. and Friend, R. (1981). Hawthorne, the myth of the docile worker, and class bias in psychology. American Psychologist, 36(8), 867–878. Brill, M., Margulis, S. and Konar, E. (1985). Using office design to increase productivity (Vol. 1–2). Buffalo, NY: Workplace Design and Productivity. Brown, B. B. (1987). Territoriality. In: D. Stokol and I. Altman (Eds.) Handbook of environmental psychology, Vol. 1, pp. 505–531. New York: John Wiley. Byström, M. (1999). Hörselnedsättning och störning av buller – jämförelse mellan kvinnor och män (Hearing impairment and noise annoyance – a comparison between women and men) (3rd Ed.) Solna: Arbetslivsinstitutet (The National Institute for Working Life). Canter, D. (1976). Environmental interaction. New York: International Universities Press. Canter, D. and Stringer, P. (1975). Environment interaction: Psychological approaches to our physical surroundings. New York: International Universities Press. Carpman, J. R. and Grant, M. A. (1993). Design that cares. Planning health facilities for patients and visitors. American Hospital Publishing. Carpman, J. R. and Grant, M. A. (2002). Wayfinding – a broad view. In: R. B. Bechtel and A. Churchman (Eds.) Handbook of environmental psychology, pp. 427–442. New York: John Wiley. Christiansson, C. and Eiserman, M. (1998). Framtidens kontor – kontorets framtid (The office of the future – the future for the office) (32rd Ed.) Laholm: Byggförlaget. Cohen, S., Evans, G., Stokol, D. and Krantz, D. (1986). Behavior, Health, and Environmental Stress. New York: Plenum Press. Dahlin, Å. (1999). Om färgupplevelse – Ett arkitekturpedagogiskt perspektiv (About colours – In an architectural pedagogic perspective). Stockholm. Danielsson, C. (2005a). Applying Lynch’s theory on office environments. The Nordic Journal of Architectural Research, 4, 69–78. Danielsson, C. (2005b). Office environment, health and job satisfaction. An explorative study of office design’s influence. Stockholm: KTH (Royal Institute of Technology). Davis, T. R. V. (1984). The influence of physical environment in offices. Academy of Management Review, 9(2), 271–283. Donald, I. (1994). Management and change in office environments. Journal of Environmental Psychology, 14, 21–30. Duffy, F. (1978). Three offices: Reading. Architects Journal, March 29, 593–604. Duffy, F. (1999). The new office (2nd Ed.) London: Conran Octopus. Ejhed, J. and Liljefors, A. (1990). Bättre belysning (Better lightening). Stockholm, Sweden: Statens råd för byggforskning.

Office experiences

627

Evans, G. (1979). Behavioral and physiological consequences of crowding in humans. Journal of Applied Social Psychology, 9, 27–46. Evans, G. and McCoy, J. (1998). When buildings don’t work: The role of architecture in human health. Journal of Environmental Psychology, 18, 85–94. Festinger, L. (1957). A theory of cognitive dissonance. Stanford: Stanford University Press. Finnegan, M. C. and Zener Solomon, L. (1981). Work attitudes in windowed vs. windowless environments. Journal of Social Psychology, 115, 291–292. Franzén, B. (1969). Offices 2 – a study of climate in nine offices block (Vol. Rapport 21/1969). Stockholm: Byggforskningen. Freedman, J., Klevansky, S. and Ehrlich, P. (1971). The effect of crowding on humans task performance. Journal of Applied Social Psychology, 1, 7–25. Goodrich, R. (1986). The perceived office: The office environment as experienced by its users. In: J. D. Wineman (Ed.) Behavioral issues in office design, pp. 109–133. New York: Van Nostrand Reinhold. Griffiths, I. D. (1975). Thermal comfort: A behavioral approach. In: D. Canter and P. Stringer (Eds.) Environmental interaction: Psychological approaches to our physical surroundings, pp. 21–52. London: Surrey University Press. Harris, D. (1991). What do office workers place on their desks and in their offices? Salmagundi, 92, 202–210. Heerwagen, J. (1990). The psychological aspects of windows and window design. Paper presented at the EDRA 21st Annual Conference ‘Windows, windowlessness and simulated view’, Polytechnic Institute of New York, Brooklyn. Heerwagen, J. (1986). Adaptation to windowlessness: A study of the use of visual décor in windowed and windowless offices. Environment and Behavior, 18(5), 623–630. Kaplan, S., Talbot, J. and Kaplan, R. (1988). Coping with daily hassles: The impact of nearby nature on the work environment. Washington DC: U.S. Forest Service. Konar, E., Sundstrom, E., Brady, C., Mandel, D. and Rice, R. W. (1982). Status demarcation in the office. Environment and Behavior, 14(5), 561–580. Küller, R. (1995). Färgens inverkan på människan (The colour’s influence on the human being). In: A. Hård, R. Küller, L. Sivik and Å. Svedmyr (Eds.) Upplevelse av färg och färgsatt miljö (Perception of colours and coloured environments) (Vol. Färgantologi Bok 2). Stockholm: Statens råd för byggnadsforskning. Küller, R. and Lindsten, C. (1992). Health and behavior of children in classrooms with and without windows. Journal of Environmental Psychology, 12, 305–317. Lazarus, R. S. (1966). Psychological stress and the Coping Process. New York: McGraw-Hill. Lu, L. (1999). Work motivation, job stress and employees’ well-being. Journal of Applied Management Studies, 8, 61–72. Mazumdar, S. (1992). Sir, please do not take away my cubicle: The phenomenon of environmental deprivation. Environment and Behavior, 24(6), 691–722. McCormick, E. J. (1976). Human Factors in Engineering and Design. New York: McGraw-Hill. Mitchell McCoy, J. (2002). Work environments. In: R. B. Bechtel and A. Churchman (Eds.) Handbook of environmental psychology, pp. 443–460. New York: John Wiley. Mitchell McCoy, J. and Evans, G. (2002). The potential role of the physical environment in fostering creativity. Creativity Research Journal, 14(3 and 4), 409–426. Moore, E. O. (1982). A prison environment: its affects on healthcare utilization. Ann Arbor: University of Michigan. Nemecek, J. and Grandjean, E. (1973). Results of an ergonomic investigation of large-space offices. Human Factors, 15(2), 111–124. Oldham, G. R. and Brass, D. J. (1979). Employees reactions to an open-plan office: A naturally occurring quasi-experiment. Administrative Science Quarterly, 24(2), 267–284. Porter, I. W. and Lawler, E. E. (1965). Properties of organization structure in relation to job attitudes and job behavior. Psychological Bulletin, 64, 23–51. Proshansky, H. M., Ittelson, W. H. and Rivlin, L. G. (1976). Environmental psychology: People and their physical settings (2nd Ed.) New York: Holt, Rinehart and Winston. Roethlisberger, J. and Dickson, W. (1939). Management and the worker. Cambridge, MA: Harvard University Press. Ruys, T. (1970). Windowless offices. University of Washington, Seattle. Scheiberg, S. L. (1990). Emotions on Display: The personal decoration of workspace. American Behavioral Scientist 33, 330–338. Siegrist, J. (2003). The Effort/Reward Model. Paper presented at the Stress Research from Biology to Society (Doctoral Course), 12 December 2003, Karolinska Institutet, Stockholm. Sivik, L. (1995). Om färgers betydelse. In: A. Hård, R. Küller, L. Sivik and Å. Svedmyr (Eds.), Upplevelse av färg och färgsatt miljö (Vol. Färgantologi Bok 2). Stockholm: Statens råd för byggnadsforskning.

628

Product Experience

Sommer, R. and Olsen, H. (1980). The soft classroom. Environment and Behavior, 12(1), 3–16. Standing, L., Lynn, D. and Moxness, K. (1990). Effects of noise upon introverts and extroverts. Bulletin of the Psychonomic Society, 28(2), 138–140. Steele, F. I. (1973). Physical settings and organisational development. Reading, Mass.: Addison-Wesley. Steele, F. (1986). Making and managing high quality workplaces: An organizational ecology. New York: Teachers College Press. Stokol, D. (1972). On the distinction between density and crowding. Psychological Review, 79(3), 275–277. Stokol, D. (1976). The experience of crowding in primary and secondary environments. Environment of Behavior 8, 49–86. Stokol, D., Smith, T. E. and Prost, J. J. (1975). Partitioning and perceived crowding in a public place. American Behavioral Scientist, 18, 792–814. Sundstrom, E. (1986). Work places: The psychology of the physical environment in offices and factories. Cambridge: Cambridge University Press. Sundstrom, E., Burt, R. and Kamp, D. (1980). Privacy at work: Architectural correlates of job satisfaction and job performance. Academy of Management Journal, 23(1), 101–117. Sundstrom, E., Herbert, R. and Brown, D. (1982). Privacy and communication in an open-plan office: A case study. Environment and Behavior, 14, 379–392. Sundstrom, E., Town, J. P., Rice, R. W., Osborn, D. and Brill, M. (1994). Office noise, satisfaction, and performance. Environment and Behavior, 26(2), 195–222. Svedberg, L. (1992). Gruppsykologi – Om den inre och yttre scenen – teori och tillämpning Group psychology – About the internal and external scene. Lund: Studentlitteratur. Ulrich, R. S. (1984). View through a window may influence recovery from surgery. Science, 224, 420–421. Vischer, J. (1996). Workspace strategies: Environment as a tool for work. New York: Chapman and Hall. Weinstein, C. S. and Weinstein, N. D. (1979). Noise and reading performance in an open space school. Journal of Educational Research, 72, 210–213. Weisman, J. (1981). Wayfinding and architectural legibility: Design considerations in housing environments for the elderly. In: V. Regnier and J. Pynoos (Eds.) Housing for the elderly: Satisfaction and preferences, pp. 441–464. New York: Garland. Wells, M. (2000). Office clutter or meaningful personal displays: The role of office personalization in employee and organizational well-being. Journal of Environmental Psychology, 20, 239–255. Wener, R. and Kaminoff, R. (1983). Improving environment information: Effects of signs on perceived crowding and behavior. Environment and Behavior 15, 3–20. Wilson, L. M. (1972). Intensive care dilerium. The effect of outside deprivation in a windowless unit. Archives of Internal Medicine, 130, 225–226. Wochel, S. and Teddlie, C. (1976). Factors affecting the experience of crowding: a two-factor theory. Journal of Personality and Social Psychology, 34, 30–40. Wohlwill, J. F. (1973). Human adaptation to levels of environmental stimulation. Human Ecology, 2(2), 127–147. Wollin, D. D. and Montagne, M. (1981). College classroom environment: Effects of sterility versus amiability on student and teacher performance. Environment and Behavior, 13, 707–716. Woodson, W. G. (1981). Human Factors Design Handbook. New York: McGraw-Hill.

27

THE SHOPPING EXPERIENCE ANN MARIE FIORE Iowa State University, Ames, IA

1. THE CHANGING SHOPPING EXPERIENCE New formats for brick-and-mortar retail centers, such as one-stop superstores and lifestyle centers (i.e. retail centers with a mixture of adjacent stores, restaurants, recreational settings, and entertainment venues) are attracting shoppers while centers of retail trade popular in the past, such as traditional enclosed shopping malls and downtown shopping districts, struggle to retain shoppers (Kim et al., 2005; McMahon, 1999; Fenley, 2003; ‘Small Business – Wolves in Shops’ Clothing’, 2005). One visit to a Niketown store with its museum case-like display of Sneakers of the Rich and Famous and stunning multimedia shows or to an American Girl Place store with its theater, café, and special events centered around its primary product (dolls), and it becomes clear that the shopping experience has come a long way from merely selecting from the never-ending shelves of products for purchase. Concurrent with expansion of these new brick-andmortar retail venues, Internet retail sales continue to expand (‘US Census’, 2005) assisted by new technologies, such as virtual tours, that have changed the online shopping experience in exciting ways. These retail venues have one thing in common; they reflect changes in consumer demand including expectation of the shopping experience. Parallel to the changing nature of retail venues, frameworks used in empirical study of the shopping experience have expanded. Models for exploring a consumer’s rational assessment of product qualities leading to product purchase have been complemented with models for exploring the effects of the broadly defined shopping environment on hedonic aspects of experience and consumer behaviors beyond product purchase. The shopping experience entails consumer processes (e.g. product evaluation, attitude formation) and responses (e.g. satisfaction, enjoyment, or purchase behavior) affected by aspects of the shopping environment (e.g. brick-and-mortar retail store, mall, catalog, and online store), situation, and consumer characteristics. To give order to the expansive topic, two models and a typology that capture variables explaining the current nature of the shopping experience will be presented, accompanied by examples of empirical research. This chapter will emphasize research regarding the brick-and-mortar retail Product Experience Copyright © 2008 Elsevier Ltd.

629

630

Product Experience

shopping experience and weave in examples of literature addressing other retail venues. The chapter will conclude with a discussion of emergent marketing trends transforming the shopping experience.

2. FRAMING THE SHOPPING EXPERIENCE Much of the research regarding consumption experience during the 1970s was grounded in the information-processing approach (Bettman, 1979) that regarded the consumer to be a logical thinker who aimed to purchase the best product from available product choices. Based on this approach, the consumer is envisioned to be a goal-directed problem solver who searches for product-related information, weighs evidence, and arrives at a carefully considered evaluation leading to a purchase decision (Hirschman and Holbrook, 1982). Hirschman and Holbrook’s (1982) article, which delineated the experiential view of consumption experience1, presented a new model for understanding consumer behavior. Hirschman and Holbrook proposed that, in contrast to the information-processing approach, some consumption experiences are better explained by an experiential approach. This approach posits that an interaction with the product, service, and/or shopping environment can be intrinsically satisfying, or satisfying for its own sake. Here information search activity during the shopping experience has more to do with providing sensory or cognitive stimulation and satisfying curiosity than determining a product’s potential for utilitarian functionality.

2.1. Consciousness-Emotion-Value (C-E-V) model and Cognition-Affect-Behavior (C-A-B) models for explaining shopping experience Two models, the Consciousness-Emotion-Value (C-E-V) model and Cognition-AffectBehavior (C-A-B) model, are relevant to explaining shopping experience. According to Holbrook (1986), the C-A-B model reflects an information-processing approach where purchase decision and brand choice are key outcomes. In the C-A-B model, thoughts or beliefs (C) about the product precede product attitude (A), which is followed by purchase decision and brand choice (B). Yet, as will become evident, the C-A-B model does not fully capture the nature of many shopping experiences, particularly those fostered by emergent retail trends. Building on Holbrook and Hirschman’s (1982) experiential approach to consumption, Holbrook (1986) proposed the C-E-V model of the consumption experience, which he compared to the widely held C-A-B model. The C-E-V model captures elements of the shopping experience not represented by the C-A-B model. However, components of both models are useful to identify the diversity of mechanisms underlying the shopping experience. The C-E-V model is dynamic with feedback loops between components. In Holbrook’s C-E-V model, Consciousness includes not only the C-A-B model’s Cognitions or beliefs about consumer products and services, but also includes a variety of mental events (in response to informational inputs) such as fantasies, imagery, memories, subconscious thoughts, and unconscious processes that occur during the consumption experience. Likewise, Emotion expands beyond narrowly conceived Affect (i.e. favorable disposition or 1 Consumption experience includes shopping experience. Consumption experience also includes preshopping behavior such as attention to product ads, as well as post-purchase product use and product disposal. As the reader will see in the last section, the shopping experience has expanded to include non-retail environments for staging marketing efforts, expanding the boundaries of shopping experience.

The shopping experience

631

liking) found in the C-A-B model to include subjective feeling states within the individual, such as joy, delight, and excitement. Research supports Holbrook’s proposition that emotion is a key link in the shopping experience. Emotional states stimulated by brick-and-mortar store design (Baker, Levy and Grewal, 1992; Bellizzi, Crowley and Hasty, 1983; Bellizzi and Hite, 1992; Beverland et al., 2006; Bruner, 1990; Crowley, 1993; Donovan et al., 2002) and online store design (Eroglu, Machleit and Davis, 2003; Menon and Kahn, 2002) mediated consumer responses towards products or shopping environments. Instead of focusing on purchase behavior, which is the consumer goal in the C-A-B model, Holbrook’s C-E-V model focuses on value derived by the consumer during the consumption experience. Value taps what the consumer perceives he or she gains from the consumption experience and includes play, fantasies, fun, and aesthetic pleasure from sensory qualities of the shopping experience. Holbrook and Hirschman (1982) saw these forms of pleasure as the experiential (non-instrumental) value of the consumption experience. Experiential value differs from instrumental (utilitarian) value, which entails shopping efficiency and making the right product choice based on logical assessment of information regarding the product’s performance or functionality. Figure 27.1 (p. 634) provides a typology of value, which will be discussed in a later section. Completing the description of the C-E-V model, consumption experience is influenced by inputs of the person variable (i.e. attributes of the individual, such as personality, intelligence, and gender that influence thinking, feeling, and behavior), environment variable (i.e. the physical elements of the product/brand and the images used to designate the product such as a website promotion), and lastly, the person-environment interaction variable or the situation (e.g. shopping with friends). The importance of these two models to the understanding of shopping experience is not their structures, but rather their definitions of component variables, because research supports: (1) the combination of experiential and utilitarian perspectives; and (2) the amalgamation of variables from each model for explaining shopping experience, as presented in the following section.

2.2. Amalgamation of C-E-V and C-A-B model components Whereas the C-E-V and C-A-B models differ in their view of human nature, each has received empirical support as an underlying mechanism of consumer shopping experience. The mechanism at work may fluctuate depending on type of product purchased and other inputs identified by Holbrook (1986). For instance, a consumer may make a rational decision – buy the brand of white athletic socks with odor guard – when shopping alone, in a hurry, and faced with the choice of two brands. The same consumer may be so mesmerized by the fanfare of an in-store promotion for big screen televisions that rational thought of where to put the gargantuan object in a studio apartment is overshadowed by fantasies of hosting jubilant playoff game parties where friends gather to watch one’s team win championship matches. Consumers do not only move between these two underlying mechanisms of shopping experience, but variables from both mechanisms may be relevant to the same shopping experience. Consumer behavior research tends to amalgamate components from the C-E-V and C-A-B models within the conceptual framework of a single study. For example, surveying 118 US university students, Mano and Oliver (1993) examined the underlying dimensionality of three aspects of post-consumption experience, namely product evaluation, productelicited affect, and product satisfaction. In their study, product evaluation (Cognition) captured both utilitarian (e.g. use, need) and experiential (e.g. interest, appeal) dimensions of the product. They found that both utilitarian and experiential dimensions were antecedents to pleasure and arousal (Emotion) and to product satisfaction (Cognitive and

632

Product Experience

Affect). Mano and Oliver’s study shows that logical information-processing and experiential elements may be simultaneously at work during shopping experience. Likewise, research (Mattila and Wirtz, 2001) exploring the effect of congruity between ambient scent and music on consumers’ evaluations and behavioral responses towards a store environment meshed components from both models. Using quasi-experimental store environments and 343 store customers, Mattila and Wirtz examined the effect of congruity of arousal from scent and music on emotional responses (Emotion), approach responses such as exploring the store or impulse buying (Behavior), and overall satisfaction with the store experience (Cognitive and Affect). They found that when ambient scent and music were perceived as having a congruent level of arousing qualities, consumers rated the store environment as significantly more positive, and exhibited higher levels of approach responses and satisfaction than when there was incongruence in perceived arousal level. Perceived value from the shopping experience entails perceived benefits derived from the elements of the shopping environment, including goods (products); services; experiential events (e.g. educational cookware demonstration); and social (e.g. interactions with brand representatives), design (e.g. colors of website, size of store), and ambient (e.g. temperature, music, scent) retail environment factors. (See Baker et al., 2002 for discussion of brick-andmortar retail environment factors.) Perceived value (Value) influences Affect and Behavior such as selection, evaluation, purchase, use of, participation in, and/or ultimate satisfaction with products, services, and environments available during the shopping experience. Moreover, perceived value derived from one element of the shopping experience may affect value and outcomes (Behavior) towards other elements. For instance, research using an experimental design involving videotaped store scenarios and US college students illustrated that experiential value from a pleasing store design influenced perceived utilitarian value of a product, which in turn affected store patronage intentions (Baker et al., 2002). A number of studies (e.g. Babin and Attaway, 2000; Eroglu, Machleit and Chebat, 2005; Fiore, Jin and Kim, 2005a; Fiore, Lee and Kunz, 2004) have tapped the perceived value (Value) derived from a shopping environment and its effect on consumer attitude (Affect) and/or approach responses (Behavior). Babin and Attaway (2000) found that positive affect (Emotion) was positively associated with both hedonic and utilitarian shopping value (Value), negative affect was inversely associated with the two value measures, and positive relationships existed between the two value measures and repeat purchase behavior (Behavior) using 144 mall shoppers. Whereas many such studies adapt a shopping value scale (e.g. see Babin, Darden and Griffin, 1994) that differentiates between general hedonic and utilitarian value of the shopping experience, finer distinctions within value derived from consumption experience have been proposed. The following section presents a typology of value (see Figure 27.1) and provides examples of research in support of these distinct value dimensions.

3. PERCEIVED VALUE AND SHOPPING EXPERIENCE Designing current shopping experiences involves more than development of environments for facilitating goal-directed acquisition of goods. It also entails offering customer services (e.g. gift wrapping), carefully orchestrated aesthetic details of retail environments, and seemingly tangential experiential events to acquisition of goods, such as educational, recreational, and entertainment events. Empirical research confirms that both experiential value and utilitarian value are derived from aspects of the shopping experience, including:

• Eating establishments (Gilmore and Pine, 2002; Hanefors and Mossberg, 2003; Pullman and Gross, 2003).

The shopping experience

633

• Experiential events (Anderson, Burns and Reid, 2003; Sit, Merrilees and Birch, 2003). • Goods (Bell, Holbrook and Solomon, 1991; Fiore, Lee and Kunz, 2003; Morganosky, 1987). • Interactions with sales staff (Hornik, 1992). • Marketing materials (Fiore and Yu, 2001). • Online shopping environments (Childers et al., 2001; Fiore et al., 2005a; Koufaris, Kambil and LaBarbera, 2001–2002). • Store environments (Baker, Grewal and Parasuraman, 1994; Moye and Kincade, 2002; Swinyard, 1993; Yalch and Spangenberg, 1990). Perceived value associated with a shopping experience is the culmination of perceived benefits derived during search, acquisition, trial, participation, appreciation, recollection, fantasy, and discussion of consumer offerings (i.e. goods, services, shopping environments, and experiential events). For instance, the time saving nature of product acquisition in a one-stop-shopping superstore (that is, of course, discounting the time it takes to assemble the search team to recover one’s wandering partner in the vast domain) and the emotional arousal and pleasure experienced during discussion about new products with one’s shopping partner may contribute to the perceived value of the shopping experience. Research (Moye and Kincade, 2002) employing a national survey of 900 US female consumers over age 18 supports the simultaneous importance of experiential and utilitarian value from the shopping experience. Moye and Kincade found that apparel consumers fitting the ‘bargain shopper’ orientation reported statistically higher levels of importance for (utilitarian) sensory/layout and (experiential) music/aesthetics (e.g. background music, flooring, signage) elements of the shopping environment than did consumers fitting the ‘decisive shopper’, ‘confident shopper’, and ‘appearance conscious shopper’ orientations. In line with these examples, Holbrook (1999) and Fiore and Ogle (2000) emphasized ‘compresence’, which is defined as ‘a co-mingling of multiple types of value in any one consumption experience’ (Holbrook, 1999, p. 186).

3.1. Typology of value Holbrook (1999) and Fiore and Ogle (2000) forwarded typologies of value relevant to consumption experiences. Both typologies itemize similar benefits supported by empirical research, but implement different organizational dimensions. Holbrook’s intrinsic and extrinsic dimensions of value are similar to Fiore and Ogle’s (2000) experiential and utilitarian dimensions, respectively. Experiential benefits are non-instrumental, or rewarding and pleasurable in and of themselves. Utilitarian benefits are rewarding because they help one attain external aims or goals such as social or economic gain (Berlyne, 1974; Holbrook, 1987). However, Fiore and Ogle uniquely organized benefits according to their emphasis on formal (sensory), expressive, or symbolic qualities. Fiore and Ogle’s typology (Figure 27.1) will be adapted for this chapter, because its organization complements the outline of this book and, more importantly, aligns with the Cognition/Consciousness and Affect/Emotion elements of the models. Each benefit will be briefly explained and literature supporting their impact on consumer responses will punctuate the section. Sensory quality based benefits Drawing on Fiore and Ogle’s work (2000), formal qualities, or what will be termed ‘sensory qualities’, refer to the perceivable features of the composition or structure of consumer offerings. For instance, color, texture, line, shape, space, balance, rhythm, and

634

Product Experience

Value derived from shopping experience

Utilitarian benefits

Experiential benefits

Sensory quality based benefits

Expressive quality based benefits

Symbolic quality based benefits

Sensory quality based benefits

* Sensual pleasure

* Aroused emotion

* Identity

* Physical comfort

* Beauty

* Creative expression

* Alternative existence * Cognitive challenge

Expressive quality based benefits * Elevated emotion

Symbolic quality based benefits * Self-acceptance * Status

* Protection and safety

* Social acceptance/ affiliation

* Quality

* Quest for knowledge

* Efficiency * Sexual attractiveness

FIGURE 27.1 Typology of value derived from shopping experience; adapted from Fiore and Ogle (2000).

proportion are the perceivable features of a physical store environment just as pitch, tempo, dynamics (loudness), harmony, and melody are perceivable features of music in the store environment. Experiential value from sensory qualities is the result of pleasant sensory stimulation from the consumer offerings of the shopping experience. The two sensory quality based experiential benefits are sensual pleasure and beauty. Sensual pleasure is positively evaluated stimulation of the senses resulting from the degree of stimulation, novelty, complexity, or unity (Fiore and Kimle, 1997). Many sensory (e.g. visual, tactile, olfactory) aspects, such as scent of a retail environment, provide sensual pleasure (Fiore et al., 2000; Michon, Chebat and Turley, 2005). As noted earlier, music/aesthetics (e.g. background music, flooring, signage) elements of the shopping environment were more important to apparel consumers fitting the ‘bargain shoppers’ orientation than they were to ‘decisive shoppers’, ‘confident shoppers’, and ‘appearance conscious shoppers’ orientations (Moye and Kincade, 2002). Sensory pleasure also affects shopping experiences involving groceries (Richardson, Jain and Dick, 1996). Using 99 subjects, Richardson et al. (1996) found that level of attractiveness of actual stores of a United Kingdom (UK) grocery chain influenced shoppers’ judgments of quality of store brands. Lower attractiveness (sensory pleasure) ratings led to lower judgments of product quality of store brands, but not national brands. Beauty is the enhancement of the physical features of the consumer’s body to achieve a perceived ideal. This ideal is personal and may not appeal to others, thus emphasizing its intrinsic nature. Beauty does not require sensual pleasure (Fiore and Ogle, 2000); consider painful corseting of the body to achieve an hourglass shape. Shopping environments may enhance beauty with the aim of increasing sales, such as adding warm lighting in a dressing room to enhance skin tone (Schiro, 1990) or mirrors that reflect a slimmer image. The sensory quality based utilitarian benefits are physical comfort, protection and safety, quality, efficiency, and attraction to the opposite (or same) sex. The same sensory quality may provide both experiential and utilitarian value; bright lighting may enhance

The shopping experience

635

the clear colors and texture of the store exterior, as well as provide safety for customers. Whereas both physical comfort and sensual pleasure are dependent upon bodily sensations, they are separate benefits because comfort does not include enhanced pleasurable sensations. Instead, physical comfort involves avoidance of negative sensations (Hollies, 1989), a neutral state or ‘unawareness’ of sensations, or a means to some other goal. Research confirms the logic of separating physical comfort from the domain of sensual pleasure; consumers differentiate between the concern for comfort and the concern for aesthetic (experiential) qualities (d’Astous, 2000; Morganosky, 1987; Winakor, Canton and Wolins, 1980). Temperatures of air-conditioning in Manhattan stores (e.g. Bergdorf Goodman, Macy’s, Old Navy) varied with their price point, according to Salkin (2005). The higher the price-point of the store, the colder was the air-conditioning. This was done to create a comfortable shopping environment for its target customer, so that they would shop longer. At luxury stores, customers are more warmly dressed because they come from air-conditioned cars straight from their air-conditioned homes. If the temperature is too cold, however, it may drive customers from the store, because of lower levels of perceived comfort. Physical comfort (Holt, 1996; Sherry, 1990) (e.g. sufficient space to move freely in a dressing room) and safety influence consumers’ attitudes towards the shopping experience and patronage (Dennis, Newman and Marsland, 2005; Grossbart et al., 1990; ‘The Benefits’, 1991). For instance, lighting in a parking area influenced perceptions of safety and consumers’ willingness to patronize a retailer (‘The Benefits’, 1991). Structural quality entails meeting certain standards in workmanship signified by integrity of object appearance (Scheller, 1993). Service quality of retail stores (Dabholkar, Thorpe and Rentz, 1996) taps perceptions of store performance on dimensions of physical aspects, reliability, personal interaction, problem solving, and policy. Quality is an important factor in consumer decision-making, signaled partly by sensory qualities of the product or environment present at the point of purchase (Fiore and Damhorst, 1992; Richardson et al., 1996; Zeithaml, 1988). For example, Baker et al. (2002) found that perceptions of store employees and store design features positively influenced consumer perceptions of interpersonal service quality, whereas store design features positively affected merchandise quality perceptions. Perceptions of interpersonal service quality had a significant direct effect and perceptions of merchandise quality had a significant indirect effect on store patronage intentions. Baker et al. (2002) tapped US university students and an experimental method involving videotaped card-and-gift store scenarios representing low or high levels of design, social, and ambient store environment cues. In the absence of a physical product, such as in the case of catalog or online purchase, the consumer depends more heavily on external cues, such as store environment features, store image, policy, or brand name (Beverland et al., 2006; van der Heijden and Verhagen, 2004; Wilde, Kelly and Scott, 2004) to determine quality. Efficiency is the ratio of outputs to inputs (Holbrook, 1994). Inputs include money, time, and effort. The efficiency of time and effort is seen as convenience. Efficiency, and consequent consumer behavior, is affected by the type and design of the retail environment, including brick-and-mortar stores (Baker et al., 2002; Morganosky, 1995), online stores (Dennis et al., 2005; Lee, Fiore and Kim, 2006), and catalogs (Eastlick and Feinberg, 1995). In addition to the effect of perceived store design features on perceived merchandise quality, Baker et al. (2002) found an inverse relationship between perceived store design features and time/effort cost perceptions. In other words, they found that perceptions of a better-designed physical store environment decrease the perceived amount of time or effort needed to complete a shopping task. This perceived efficiency had a significant direct effect on store patronage intentions. Protection of personal information and efficiency, two important elements of online shopping (Lee, 2002) are due to

636

Product Experience

interface design and technological aspects of the website. These technological aspects do not fit neatly under sensory qualities, but should be mentioned. The last of the sensory quality based utilitarian benefits, sexual attractiveness, is an instrumental outcome of beauty (Berscheid and Walster, 1974). Humans, like many other species, use their physical appearance to attract a mate. This is equally true for human attractiveness to the opposite or same sex. Research on gay men (Freitas, Kaiser and Hammidi, 1996; Rudd and Tedrick, 1994) supports the importance of using the benefit of sexual attractiveness in the promotion of products. Expressive quality based benefits Expressive quality based benefits result from the use of sensory elements (e.g. color, lines, textures) and symbolic images (e.g. image of the sun, fluffy kittens) to express or evoke emotion (Arnheim, 1986; Gibson, 1979; Lindauer, 1984). When expressing emotion, the consumer is the creator of the expressive qualities (Fiore, Kimle and Moreno, 1996a). This results in the benefit termed creative expression, which is prefaced on a pleasurable release of emotion during the creative process (Koestler, 1981). Research (Fiore et al., 2004) showed ‘having an exciting experience’ was a significant reason for enticing consumers to try co-design, a new shopping experience involving consumers as creators. This study employed survey data derived from 521 university students, predominately female students, from five different regions of the US. When a consumer’s emotion is evoked by expressive qualities of offerings created by others (Fiore, Moreno and Kimle, 1996b), the experiential benefit of aroused emotion is the result. Expression of emotion is pleasurable or satisfying, because it makes a person feel alive, engaged with the world, and vibrant (Dissanayake, 1988). Emotional experience, itself, may be the motivation or desired gain from the shopping experience (Dennis et al., 2005). Research in the area of environment psychology has looked at the effect of store environment cues on emotional pleasure and arousal. These studies may be classified as examining aroused emotion. These studies include: • • • • •

Mall environment (Sayed, Farrag and Belk, 2003; Wakefield and Baker, 1998). Music in stores (Sweeney and Wyber, 2002). Scent (Chebat and Michon, 2003; Knasko, 1995). The combination of music and scent in stores (Mattila and Wirtz, 2001). The brick-and-mortar store environment (Babin and Darden, 1995).

Looking at a specific example, Sayad et al. (2003) found that emotional state (pleasure and arousal) was affected by mall environment cues (background music, crowdedness, lighting, and mall location) and the resulting emotional state positively affected behavioral intentions towards the mall. Their experimental design consisted of 308 Egyptian consumers exposed to video recordings and still images of mall environments with different music treatments. Both experiential and utilitarian benefits from expressive qualities can be therapeutic. The difference is that the therapeutic effect (i.e. elevated emotion) is the goal of utilitarian value. In aesthetic experience, expressive qualities may shift emotions from a neutral state to a pleasurable or satisfied state (Apter, 1984); these qualities may also serve the utilitarian benefit of elevated emotion by shifting emotion from a negative state to a homeostatic neutral state or elated state as found in the compulsive shopping experience (Faber and Christenson, 1996). Kim, Kang and Kim (2005) found that loneliness level of older consumers was associated with visiting a mall to alleviate boredom. This alleviation of boredom may be an example of the shopping experience used to elevate

The shopping experience

637

emotion. This motivation was, however, negatively associated with the level of money these consumers spent in the mall over a three-month period. Another study (Arnold and Reynolds, 2003) classified shopping motivations into six categories, including ‘gratification shopping’, which entails shopping as a way to relieve stress, alleviate a negative mood, and as a special treat for oneself. These shoppers viewed shopping as a ‘pick-meup’ and a ‘lift’ when they felt depressed. Symbolic quality based benefits Symbolic qualities represent the meaning or content communicated by consumer offerings. Symbolic quality based benefits come from cognitively processing meaning or content. Experiential benefits of identity and alternative existence are derived from exercising the mind while physically creating or imagining one’s reality-based or fantasy-based ideas, respectively. Another experiential benefit, cognitive challenge, is derived from understanding the meaning or content put forth by others. Cognitive pleasure from cognitive challenge results from intrinsically driven discovery (Deci, 1975; Koestler, 1981) involved in reconciling humorous, novel, complex, and/or ambiguous aspects of the consumer offering. Regarding identity, consumer offerings can communicate, in a creative and sometimes humorous or whimsical way, something about the consumer or his/her view of the world. Communication of these messages leads to a pleasurable experience (Fiore et al., 1996a). For instance, congruity between brand or store image and a whimsical aspect of one’s self-image (del Rio, Vázquez and Iglesias, 2001; Sirgy, Grewal and Mangleburg, 2000) may provide identity-based value. Likewise, consumer offerings may support fantasies or an alternative existence, where a desired situation and/or persona are created physically and/or in imagination. Cognitive pleasure comes from fantasizing (Fiore and Kimle, 1997), such as imagining pleasurable product use scenarios when shopping in a store (Fiore, Yan and Yoh, 2000), online (Song, Fiore and Park, 2007), and by catalog (Fiore, 2002). As an example of alternative existence, in a laboratory experiment with 109 US university students, Fiore et al. (2000) found that adding a pleasant and appropriate fragrance to a store display containing a lingerie product significantly enhanced respondents’ level of seeing oneself in the fantasy image involving the product, compared to a display with a pleasant but inappropriate fragrance or the product simply hung on a hanger. Seeing oneself in the fantasy image (alternative existence) and sensory pleasure predicted both global attitude and purchase intention toward the product. However, seeing oneself in the fantasy image was more strongly related to these two approach responses than was sensory pleasure. Both symbolic quality based experiential and utilitarian benefits involve cognitive processes. However, the intended goal of the former is pleasure for its own sake from presentation of and/or ‘playing with’ ideas, whereas psychological comfort is the goal of the latter. These utilitarian benefits are labeled self-acceptance, status, social acceptance/ affiliation (Cialdini et al., 1976), and quest for knowledge. Self-acceptance entails confidence in one’s own character and can be affected by consumer decisions of what to buy or where to shop. Shoppers select consumer offerings such as goods (Kalafatis et al., 1999) and retail environments that are congruent with their personal values (Erdem, Oumlil and Tuncalp, 1999; Ogle, Hyllegard and Dunbar, 2004) and characteristics (Beverland et al., 2006; Hu and Jasper, 2006; Rintamäki et al., 2006). When the shopper reflects on these behaviors, they become tangible indicators of personal character for the shopper. In line with self-acceptance, Ogle et al. (2004) investigated the importance of sustainable store design (i.e. design and material selection limiting harm to the environment) to consumers at an REI store in Colorado. REI is a major US customer-owned cooperative

638

Product Experience

of outdoor equipment and apparel and is positioned as a leader of corporate citizenship. A store intercept method tapped 186 US consumers ranging in age from 18 to 79. The researchers concluded that because of the importance of store design to consumer approach responses, REI should incorporate sustainable design features into the buildings, interiors, and landscapes of their stores. Whereas Ogle et al. (2004) examined overall store design, other researchers (Beverland et al., 2006) have examined the misfit of a single store environment feature and found that the misfit can negatively affect brand image leading to a decline of the consumer-brand relationship, because the brand image no longer fits with the consumer’s self-concept. Beverland et al. (2006) analyzed data from 20 in-depth interviews with Australian consumers between the ages of 18 and 48. They found that a misfit between music and the brand image of a store may lead consumers to believe the brand is targeted to a different market, diminishing quality or authenticity perceptions of the brand, which ultimately influenced attitude and behavior towards the brand. Shopping behaviors may lead to status, which, unlike self-acceptance, requires validation by others. However, both help maintain the health of one’s ego or create a state of psychological comfort. The psychological health or comfort associated with the benefit status results from validation of one’s favorable standing, role, or achievements (e.g. personal, professional, social, moral, or economic). The image (Dennis et al., 2005) or exclusivity of a shopping environment or event may cause envy of outsiders, leading to status of participants. For instance, special trunk shows for riotously wealthy ‘preferred’ customers at London’s Harrod’s Department Store where customers are flown in by helicopter and pampered in a private room as only Harrod’s can do may affect status. Whereas status is accomplished by distinguishing oneself from others, the utilitarian benefit of social acceptance/affiliation is attained through achieving acceptance by, or showing similarity to, members of a desired group. Both benefits, however, require validation by others. Social acceptance/affiliation also has a positive effect on psychological health or comfort. Shopping with friends, a primary motivation for teen shoppers (Meyer and Anderson, 2000), cements social bonds and reflects social acceptance/affiliation. This benefit is an important determinant in UK and US consumers’ selection of shopping venues including shopping in thrift stores, at market stalls, and through home party direct sales (Thomas and Peters, 2006). In-depth interviews with 16 US women revealed that strengthening interpersonal relationships with current friends and building connections with acquaintances in the community were two important reasons for shopping at ‘underground malls’, which consist of socially networked buyers and sellers meeting face-to-face in a private setting (e.g. seller’s home or garage) for the purpose of legal trade of non-counterfeit goods (Thomas and Peters, 2006). Lastly, quest for knowledge is similar to the benefit cognitive challenge. Both involve appeasing cognitive curiosity. However, in the quest for knowledge, curiosity is satisfied by information delivered in a straightforward prosaic fashion, rather than in the more ambiguous, aesthetic manner, and the information is likely gathered for deliberate use in a decision-making process. The shopping environment may provide the customer information through packaging, signage, and knowledgeable sales staff or brand representatives. Hetzel’s (1995) interpretive study noted how the product and the environment of Nature and Découvertes, a French retailer similar to Nature Company in the US, satisfies the customer’s quest for knowledge: Once inside the store, the clients can explore at leisure, just as though they were in a natural history museum. Every product gives the client an opportunity to widen his field of knowledge. In addition, the information cards show when and where the product was invented, a résumé of the manufacturer’s background or else the natural phenomenon the product represents. (p. 127)

The shopping experience

639

4. EMERGENT MARKETING TRENDS AND THE ENGAGED CONSUMER As has been presented, experiential value from the shopping experience is not new, but emergent trends reveal that experiential elements have become more pervasive, holistic, customized, interactive, and transient. The consumer increasingly desires holistic shopping experiences that immerse their senses, evoke emotion, and stimulate their thinking, resulting in more rewarding and memorable encounters (Schmitt, 1999; Pine and Gilmore, 1999). Through interactive experiential elements, the consumer is able to become a creator of value within the shopping experience. Moreover, experiential elements are a powerful draw for consumers, fed in part by the demand created by their increasing transience. These emergent trends reflect the evolving expectations of the new ‘engaged consumer’. This section will draw on empirical and industry literature to illustrate emergent marketing trends that reflect the nature of the engaged consumer and provide suggestions for future research that address these trends.

4.1. Pervasive experiential elements Pine and Gilmore (1999), Richards (2001), and Postrel (2003) agree that there is a pervasive change in consumer behavior; shoppers are no longer singularly concerned with purchase of goods and services; they increasingly crave engaging experiences. Pine and Gilmore (1999) defines this as the ‘experience economy’. Pine and Gilmore proposed that from 1959 to 1996 US growth in employment and the nominal Gross Domestic Product had been due primarily to experiences, followed closely by services, with goods and commodities trailing behind. They envisioned that future economic growth in the US would come from businesses offering enriched, distinct experiences that engage the customer. In support, Richards (2001) stated that the fastest growing sectors of the global economy are related to consumption of experiences. Postrel (2003), an economics columnist for the New York Times, provides a cornucopia of data and examples from various nations that document the centrality of experiential value for consumers from goods, designed environments, and personal appearance. Consumers nowadays have copious opportunities to engage in experiences sponsored by retailers, brands, and shopping centers. Retailers such as Recreational Equipment, Inc. (REI) offer a flagship store experience complete with a climbing wall, bike track, and walking trails (Gilmore and Pine, 2002). Experiential environments are created around brands, such as Hershey’s Chocolate World. To stay competitive, shopping malls offer increased entertainment (Anderson et al., 2003; Christiansen et al., 1999) and creative use of space and design (Fenley, 2003) including use of shopping centers as alternatives to traditional marketing media (‘Simon Malls’, 2004). Moreover, the shopping experience has left the building, literally. Not only can the shopping experience take place anywhere an Internet or mobile phone signal can be received, and on a ‘24/7/365’ basis, but the engaged consumer embraces new lifestyle marketing strategies that enmesh marketing of consumer offerings with personally relevant lifestyle experiences, such as films, sporting events, motorcycle rallies, or concerts. Traditionally, benefits created by goods or services were central to consumers. Now, consumers expect marketing campaigns themselves to provide a meaningful benefit (‘The Manifesto’, 2004), such as aroused emotions, social affiliation, or cognitive challenge. The consumer should be engaged and stimulated by relevant content to ensure the experience works with, rather than interrupts, his/her lifestyle (Garbett, 2006). A recent survey in Australia, China, the US, and the UK, found that experiential marketing, ‘broadly defined as live events where audiences interact with a product or brand face to face’ (‘Global’, 2006, p. 1), continues to grow. The survey showed that

640

Product Experience

80% of respondents agreed that participating in experiential marketing would give them more information than other media and that they would be more likely to purchase after attending a live marketing event. The survey also showed that 85% agreed that they would tell others about participating in the experience. To better understand these results, empirical research should explore how variables related to Emotion, Consciousness, Cognition, and Affect are affected by elements of experiential marketing and consequently how these variables mediate approach responses. Furthermore, the desired outcomes of experiential marketing (e.g. positive word-of-mouth versus immediate purchase of a product) should be identified. Composite measures (e.g. Zeithaml, Berry and Parasuraman, 1996) that contain items capturing word-of-mouth communication should be updated to include metrics for more technology-savvy consumers, such as ‘I text messaged my friends about the event’, or ‘I sent pictures from my mobile phone to friends to show them what was happening at the event’.

4.2. Customized and interactive experiential elements The consumer’s role in the retail experience has transformed; consumer and producer roles now intermingle (Fiore, in press; Solomon, 2005). Accordingly, consumers contribute to the shaping of brand identity and the physical product through: • Shared information during face-to-face interactions with brand representatives and product designers. • Proactive online communication with companies and other consumers (e.g. product reviews, blogs, product commercials). • Mass customization of goods in brick-and-mortar and online stores (Pine, 1993; Pine and Gilmore, 1999) such as stuffed animals (Build-a-Bear Workshops®) and athletic shoes and bags (NikeiD.com®; see Figure 27.2). • ‘Customer-made’ activities, where consumers supply design ideas for a product that is put into production and becomes part of the product line for a firm (‘Customer made’, 2005). Along with the mass customization of products, selling environments are becoming more customized. Market optimization software covertly captures information on a consumer’s shopping and buying patterns to be used by the retailer to customize the combination of products or special promotions appearing on its website (Demery, 2005). The consumer can also engage in customization of product combinations shown together on a screen by using product search functions (e.g. show all brown leather, size 8, 2-inch heel shoes) and new image interactivity technology (Fiore, Kim and Lee, 2005b) such as My Virtual Model™, which allows customization of apparel product combinations viewed on a customized, computer-generated image of a body form. Technology will continue to change the interactive nature of the shopping experience; ‘The retailer of the future will use technology to not only link shoppers with information or products, but also guide them through the shopping experience they prefer’ (Snell, 2006, p.3). Research by Fiore et al. (2003; 2004; 2005a; 2005b) provided empirical evidence that mass customization and interactivity may provide experiential and utilitarian value, as well as enhance approach responses. However, further research is needed because these studies focus on apparel-related mass customization and image interactivity technology stimuli, which limits generalizability to other product categories and technologies. Future studies should first expand the dimensions of shopping environment cues proposed by Baker et al. (2002) and Bitner (1992). Dimensions should include technologies that facilitate customization and interactivity, such as in-store kiosks that allow consumers to retrieve product

The shopping experience

641

FIGURE 27.2 Example of steps involved in mass customization of a NIKEiD® product.

information. Person variables affecting the use of customization and interactivity such as opinion leader or opinion seeker status (Flynn, Goldsmith and Eastman, 1996) should be explored. It would be interesting to test whether person variables regulate the influence of actual mass customization of products on attitude toward the product or retailer and approach responses such as satisfaction with the product or retailer.

4.3. Holistic experiential elements Designing a holistic experience entails more than combining pleasing environmental cues. Moreover, creating a holistic experience entails congruency among these cues to dimensionalize or give form to the message of the brand or store image (Snell, 2006; Ogle et al., 2004). This reflects the rapidly expanding strategy (‘Gen Y’, 2005; Gordon, 2004) of experiential marketing, which aims to translate the brand’s essence or associations ‘into a set of tangible, physical, [and] interactive experiences’ (McNickel, 2004, p. 1). Pragmatically, it involves creating opportunity for consumers to have face-to-face interactions with brand representatives, as well as weaving products and brands into lifestyle activities to capture a brand’s essence (‘Gen Y’, 2005). According to Schmitt’s experiential marketing text (1999), the set of experiences should be well integrated and synergistic to positively affect consumer responses towards the brand. Holistic experiences instill a sense of familiarity and participation in the consumer which helps produce a feeling of membership, a noted requirement for brand loyalty (Bigham, 2005; Snell, 2006). In addition, experiential marketing strategies are effective at reaching the growing number of consumers jaded towards traditional mass media (‘Gen Y’, 2005; Palmer, 2002) and expecting a shopping experience that is unique, entertaining, and engaging (e.g. Kang and Kim, 1995; Pine and Gilmore, 1999; Wilhelm and Mottner, 2005). To quote Gilmore and Pine (2002, p. 3), ‘People have become relatively immune to messages targeted at them. The way to reach your customers is to create an experience within them’.

642

Product Experience

Empirical evidence is needed to support industry claims of the positive influence of holistic experiences on these consumer responses. This may require researchers to implement a more longitudinal and/or qualitative approach. In addition, new measures of congruity may be required to capture the holistic experience. For instance, store personality (d’Astous and Levesque, 2003) or brand personality (Kim, 2000), which may be used to measure congruity between the store/brand and consumer self-image, may not fully capture the store/brand’s essence expressed by lifestyle marketing cues. Pine and Gilmore’s (1999) four types of consumer experience – educational, aesthetic, entertainment, and escapist experiences (4Es) – may be useful in defining the mediating nature of the holistic marketing experience. Researchers (Oh, Fiore and Jeong, 2004) have developed and tested a 4E scale. Results showed, contrary to Pine and Gilmore’s (1999) proposition, that aesthetic experience rather than all 4Es was responsible for creating memorable experiences and consumer satisfaction for guests of Bed and Breakfast hotels in a US state. Using a survey to examine apparel specialty store experiences of 205 university students in the US, researchers (Jeong, Niehm and Fiore, 2006) found significant relationships between entertainment and aesthetic experiences, emotional pleasure and arousal, and consumer patronage intentions. Therefore, further research is needed to determine the mediating role of the 4Es in effective experiential marketing of various consumer product categories. The mediating effect and range of value derived from experiential marketing should also be explored. For instance, Table 27.1 posits experiential value constituents for each of the 4Es that may mediate consumer responses to an experiential marketing environment (Fiore et al., 2005c).

4.4. Transient experiential elements Experiential marketing involves new programs, many with short response timeframes. For instance, as traditional rewards programs (e.g. free merchandise as a reward to loyal customers) wane, firms have turned to experience-based rewards to strengthen their customer relationships (‘Experiential’, 2006). As an example, General Motors held a musical concert (i.e. reward) for loyal customers to launch the Hummer H3. Invitations to this lifestyleoriented brand experience were redeemed within 48 hours (‘Experiential’, 2006). Similarly, the Simon Property Group has implemented a live music tour targeted at teenagers for its malls that includes brand sponsorships, contests, and entertainment (‘Simon DTour’, 2003). Firms large and small have begun to embrace another new experiential marketing technique, pop-up retail (not to be confused with pop-up ads on websites), to provide the experiential environment desired by consumers, build brand image, and attract attention and new customers (‘Pop-up Marketing’, 2006; ‘Pop-up Retail’, 2005). Pop-up retail entails creation of a retail environment that is highly experiential, focused on promoting a brand or product line, but temporary. Pop-up stores are designed to be open a few days to a year, may not sell products, and generally depend on word-of-mouth instead of mass media campaigns to draw people. Pop-up retail is not limited to businesses associated with creating retail experiences, such as apparel retailers or restaurants. For instance, Unilever opened a Suave shampoo pop-up salon for five days. Meow Mix Company opened a gift shop and café for cats for one week. (See www.meowmix.com; I speculate that it must be very difficult to get cats to politely queue for their double lattés – hold the espresso.) Health and beauty magazine, Self, opened a spa for one month where $25 allowed full access to the facilities including free makeovers and consultations with a physician regarding facial treatments (‘Pop-up Marketing’, 2006; ‘Pop-up Retail’, 2005). In-situ research techniques (e.g. participant observation) may be useful in capturing the richness of influence of these new transient experiential marketing programs.

643

The shopping experience

TABLE 27.1 Proposed experiential value constituents of Pine and Gilmore’s (1999) 4E strategies (Fiore et al., 2005c) Experiential value constituents 4E Strategies

Sensory-based value

Consciousness-based value Emotion-based value

Educational experience

Beauty

Identity Cognitive challenge

Creative expression Aroused emotion

Aesthetic experience

Sensual pleasure

Entertainment experience

Cognitive challenge

Aroused emotion

Escapist experience

Alternative existence

Creative expression

Aroused emotion

Moreover, expansion of the environmental cues listed in current models (e.g. Baker et al., 2002; Bitner, 1992) to include a ‘here today, gone tomorrow’ temporal cue may be helpful in capturing the factors affecting the impact of these programs on consumer responses. Likewise, ‘having a memorable experience’ should be added to response variables, to reflect a main goal of experiential marketing (Pine and Gilmore, 1999). The effect of moderator variables, such as demographics and market segment, on the acceptance and effectiveness of various new experiential marketing tools should be explored. Younger consumers (Wilhelm and Mottner, 2005) and the trendy, fun-loving, and impulsive ‘Striver’ segment (Kim et al., 2003), for instance, may be highly receptive to experiential marketing tools. It remains to be seen how shopping experience will change in the future, but experiential marketing should make it deliciously stimulating for shoppers and researchers.

ACKNOWLEDGMENTS I would like to acknowledge Yi-Tung Lo and Hye-Jeong Kim who provided assistance in the compilation of literature and assembly of the reference list. I would also like to thank the editors, Rick Schifferstein and Paul Hekkert, for the opportunity to contribute this chapter.

REFERENCES Anderson, C. E., Burns, D. J. and Reid, J. S. (2003). The next evolutionary step for regional shopping malls. Journal of Shopping Center Research, 10(2), 27–59. Retrieved February 23, 2006, from http://www.reuw. washington.edu/JSCR/2005Articles/JSCRV12_1A2Teens.pdf Apter, M. J. (1984). Reversal theory, cognitive synergy and the arts. In: W. R. Crozier and A. J. Chapman (Eds.) Cognitive processes in the perception of art, pp. 411–426. New York: Elsevier Science. Arnheim, R. (1986). New essays on the psychology of art. Berkeley, CA: University of California Press. Arnold, M. J. and Reynolds, K. E. (2003). Hedonic shopping motivations. Journal of Retailing, 79, 77–95. Babin, B. J. and Attaway, J. S. (2000). Atmospheric affect as a tool for creating value and gaining share of a customer. Journal of Business Research, 49, 91–99. Babin, B. J. and Darden, W. R. (1995). Consumer self-regulation in a retail environment. Journal of Retailing, 71(1), 47–70. Babin, B. J., Darden, W. R. and Griffin, M. (1994). Work and/or fun: Measuring hedonic and utilitarian shopping value. Journal of Consumer Research, 20, 644–656. Baker, J., Grewal, D. and Parasuraman, A. (1994). The influence of store environment on quality inferences and store image. Journal of the Academy of Marketing Science, 22, 328–339. Baker, J., Levy, M. and Grewal, D. (1992). An experimental approach to making retail store environmental decisions. Journal of Retailing, 68(4), 445–460.

644

Product Experience

Baker, J., Parasuraman, D., Grewal, D. and Voss, G. B. (2002). The influence of multiple store environment cues on perceived merchandise value and patronage intentions. Journal of Marketing, 66(2), 120–141. Bell, S.S., Holbrook, M.B. and Solomon, M.R. (1991). Combining esthetic and social value to explain preferences for product styles with the incorporation of personality and ensemble effects. Journal of Social Behavior and Personality, 6(6), 243–274. Bellizzi, J. A. and Hite, R. E. (1992). Environmental color, consumer feelings, and purchase likelihood. Psychology and Marketing, 9, 347–363. Bellizzi, J. A., Crowley, A. E. and Hasty, R. W. (1983). The effects of color in store design. Journal of Retailing, 59, 21–45. Berlyne, D. E. (1974). Studies in the new experimental aesthetics: Steps toward an objective psychology of aesthetic appreciation. Washington, DC: Hemisphere. Berscheid, E. and Walster, E. (1974). Physical attractiveness. In: L. Berkowitz (Ed.) Advances in experimental social psychology: Vol. 7. New York: Academic Press. Bettman, J. R. (1979). An information processing theory of consumer choice. Reading: MA, Addison-Wesley. Beverland, M., Lim, E. A. C., Morrison, M. and Terziovski, M. (2006). In-store music and consumer-brand relationships: Relational transformation following experiences of (mis)fit. Journal of Business Research, 59(6), 982–989. Bigham, L. (2005, February 15). Designing the customer experience: Experiential Environments®. 360⬚ Newsletter. Retrieved June 20, 2006 from http://www.jackmorton.com/360/industry_news/print.asp Bitner, M. J. (1992). Servicecapes: The impact of physical surroundings on customers and employees. Journal of Marketing, 56, 57–71. Bruner, G. C. (1990). Music, mood, and marketing. Journal of Marketing, 54, 94–104. Chebat, J.-C. and Michon, R. (2003). Impact of ambient odors on mall shoppers’ emotions, cognition, and spending: A test of competitive causal theories. Journal of Business Research, 56, 529–539. Childers, T. L., Carr, C. L., Peck, J. and Carson, S. (2001). Hedonic and utilitarian motivations for online retail shopping behavior. Journal of Retailing, 77, 511–535. Christiansen, T., Comer, L., Feinberg, R. and Rinne, H. (1999). Effects of mall entertainment value on mall profitability. Journal of Shopping Center Research, 6(2), 7–23. Retrieved 23 February 2006, from http:// www.reuw.washington.edu/JSCR/IndArticles/Christiansen_N299.pdf Cialdini, R., Bordon, R., Thorne, A., Walker, A., Freeman, S. and Sloane, L. (1976). Basking in reflected glory: Three (football) field studies. Journal of Personality and Social Psychology, 34, 366–375. Crowley, A. E. (1993). The two-dimensional impact of color on shopping. Marketing Letters, 4, 56–59. Customer-made. Trendwatching.com. Retrieved August 24, 2006 from http://www.trendwatching.com/trends/ CUSTOMER-MADE.htm Dabholkar, P. A. Thorpe, D. I. and Rentz, J. O. (1996). A measure of service quality for retail stores: Scale development and validation. Journal of the Academy of Marketing Sciences, 24(1), 3–16. d’Astous, A. (2000). Irritating aspects of the shopping environment. Journal of Business Research, 49(2), 149–156. d’Astous, A. and Levesque, M. (2003). A scale for measuring store personality. Psychology and Marketing, 20(5), 455–469. Deci, E. L. (1975). Intrinsic motivation. New York: Plenum Press. del Rio, A. B., Vázquez, R. and Iglesias, V. (2001). The effect of brand associations on consumer response. Journal of Consumer Marketing, 18(5), 410–425. Demery, P. (2005, March). Personalization plus. Internet Retailer [online version]. Retrieved 1 May 2005 from http://www.internetretailer.com/article.asp?id⫽14250 Dennis, C., Newman, A. and Marsland, D. (2005). Objects of Desire. New York: Palgrave Macmillan. Dissanayake, E. (1988). What is art for? Seattle, WA: University of Washington Press. Donovan, R. J., Rossiter, J. R., Marcoolyn, G. and Nesdale, A. (2002). Store atmosphere and purchasing behavior. Journal of Retailing, 70(3), 283–294. Eastlick, M. A. and Feinberg, R. A. (1995). Differences in attitudes toward catalog retailers of apparel among social/economic risk orientation groups. Clothing and Textiles Research Journal, 13(4), 220–226. Erdem, O., Oumlil, A. B. and Tuncalp, S. (1999). Consumer values and the importance of store attributes. Journal of Retail and Distribution Management, 27(4), 137–144. Eroglu, S. A., Machleit, K. A. and Chebat, J. C. (2005). The interaction of retail density and music tempo: Effects on shopper responses. Psychology and Marketing, 22(7), 577–590. Eroglu, S. A., Machleit, K. A. and Davis, L. M. (2003). Empirical testing of a model of online store atmospherics and shopper responses. Psychology and Marketing, 20, 139–150. Experiential rewards: Customer retention via experiences is smart strategy. (2006, April 18). 360⬚ Newsletter. Retrieved 20 June 2006 from http://www.jackmorton.com/360/industry_insight/feb05_industry in.sp.asp

The shopping experience

645

Experiential rewards: Customer retention via experiences is smart strategy. (2006, April 18) 360⬚ Newsletter. Retrieved 20 June 2006 from http://www.jackmorton.com/360/industry_news/april06_in3.asp Faber, R. J. and Christenson, G. A. (1996). In the mood to buy: Differences in the mood states experienced by compulsive buyers and other consumers. Psychology and Marketing, 13(8), 803–819. Fenley, G. (2003). Reinventing the mall. Display and Design Ideas, 15(4), 20–23. Fiore, A. M. (in press). The digital consumer: Valuable partner in product development and production. Clothing and Textiles Research Journal. Fiore, A. M. (2002). Effects of experiential pleasure from a catalogue environment on approach responses towards fashion apparel. Journal of Fashion Marketing and Management 6(2), 122–133. Fiore, A. M. and Damhorst, M. L. (1992). Intrinsic cues as predictors of perceived quality of apparel. Journal of Consumer Satisfaction, Dissatisfaction, and Complaining Behavior, 5, 168–178. Fiore, A. M. and Kimle, P. A. (1997). Understanding aesthetics: For the merchandising and design professional. New York: Fairchild Publications. Fiore, A. M. and Ogle, J. P. (2000). Facilitating the integration of textiles and clothing subject matter by students. Part I: Dimensions of model and taxonomy. Clothing and Textiles Research Journal, 18(1), 31–45. Fiore, A. M. and Yu, H. (2001). Effects of imagery copy and product samples on responses towards the product. Journal of Interactive Marketing, 15(2), 37–47. Fiore, A. M., Jin, H.-J. and Kim, J. (2005a). For fun and profit: Hedonic value from image interactivity and responses towards an online store. Psychology and Marketing, 22(8), 669–694. Fiore, A. M., Kim, J. and Lee, H.-H. (2005b). Effects of image interactivity on approach responses towards an online retailer. Journal of Interactive Marketing, 19(3), 38–53. Fiore, A. M., Kimle, P. A. and Moreno, J. M. (1996a). Aesthetics: A comparison of the state of the art outside and inside the field of textiles and clothing. Part I: Creator and creative process. Clothing and Textiles Research Journal, 14(1), 30–41. Fiore, A. M., Lee, S.-E. and Kunz, G. (2003). Psychographic variables affecting willingness to use bodyscanning. Journal of Business Management, 9(3), 271–287. Fiore, A. M., Lee, S.-E. and Kunz, G. (2004). Individual differences, motivations, and willingness to use a mass customization option of fashion products. European Journal of Marketing, 38(7), 835–849. Fiore, A. M., Moreno, J. M. and Kimle, P. A. (1996b). Aesthetics: A comparison of the state of the art outside and inside the field of textiles and clothing. Part III: Appreciation process, appreciator, and summary comparisons. Clothing and Textiles Research Journal, 14(3), 169–184. Fiore, A. M., Niehm, L., Oh, H., Jeong, M. and Hausafus, C. (2005c). Experience economy strategies: Adding value to small rural retail and hospitality businesses. Unpublished manuscript, Iowa State University at Ames. Fiore, A. M., Yan, X. and Yoh, E. (2000). Effects of product display and environmental fragrancing on approach responses towards a product and pleasurable experiences, Psychology and Marketing, 17(1), 27–54. Flynn, L. R., Goldsmith, R. E. and Eastman, J. K. (1996). Opinion leaders and opinion seekers: Two new measurement scales. Journal of the Academy of Marketing Science, 24(2), 137–147. Freitas, A., Kaiser, S. and Hammidi, T. (1996). Communities, commodities, cultural space, and style. Journal of Homosexuality, 31(1–2), 83–107. Garbett, J. (2006, June 21). Experiential marketing vs. field marketing: Experience the difference. 360⬚ Newsletter. Retrieved June 21, 2006 from http://www.jackmorton.com/360/market_focus/june06_mf.asp Gen Y to marketers: Engage us with experiential marketing. (2005). PR Newswire. Retrieved February 5, 2006, from http://biz.yahoo.com/prnews/051130/nyw013.html?.v⫽30 Gibson, J. J. (1979). The ecological approach to visual perception. Boston: Houghton Mifflin. Gilmore, J. H. and Pine II, B. J. (2002). The experience is the marketing. Aurora, OH: Strategic Horizons LLP. Global experiential marketing study: 2006 survey reveals insights, benefits. (2006, June 21). 360⬚ Newsletter. Retrieved 21 June 2006. http://www.jackmorton.com/360/industry_insight/jun06_industryin.asp Gordon, K. T. (2004, September 1). Give it a go: A ‘hands-on’ approach to marketing your product could be just the thing to win customers (Tactics). Entrepreneur Magazine. Retrieved 5 February 2006, from http:// www.highbeam.com/library/doc3.asp?DOCID⫽1G1:121644778andnum⫽1andctrlInfo⫽Round18%3AP rod%3ASR%3AResultandao⫽andFreePremium⫽BOTH Grossbart, S. Hampton R., Rammohan, B. and Lapidus, R. S. (1990). Environmental dispositions and customer response to store atmospherics. Journal of Business Research, 21, 225–225. Hanefors, M. and Mossberg, L. (2003, Summer). Searching for the extraordinary meal experience. Journal of Business and Management, 9(3), 249–270. Hetzel, P. (1995). The fall and rise of marketing fundamentalism: The case of the Nature and Découvertes distribution concept. Proceedings of the Marketing Eschatology Conference, pp. 121–131. Jordanstown, Northern Ireland: University of Ulster.

646

Product Experience

Hirschman, E. C. and Holbrook, M. B. (1982). Hedonic consumption: Emerging concepts, methods and propositions. Journal of Marketing, 46, 92–101. Holbrook, M. B. (1986). Emotions in the consumption experience: Toward a new model of consumer behavior. In: R. A. Peterson, W. D. Hoyer and W. R. Wilson (Eds.) The role of affect in consumer behavior: Emerging theories and applications, pp. 17–52. Lexington, MA: Heath. Holbrook, M. B. (1987). The study of signs in consumer esthetics: An egocentric review. In: J. Umiker-Sebeok (Ed.) Marketing and semiotics: New directions in the study of signs for sale, pp. 73–121. New York: Mouton de Gruyter. Holbrook, M. B. (1994). Axiology, aesthetics, and apparel: Some reflections on the old school tie. In: M. R. DeLong and A. M. Fiore (Eds.) Aesthetics of textiles and clothing: Advancing multidisciplinary perspectives, pp. 131–141. Monument, CO: International Textiles and Apparel Association. Holbrook, M. B. (1999). Introduction to consumer value. In: M. B. Holbrook (Ed.) Consumer value: A framework for analysis and research, pp.1–28. London: Routledge. Hollies, N. R. S. (1989). Visual and tactile perceptions of textile quality. Journal of the Textile Institute, 80(1), 1–18. Holt, P. W. (1996). An exploration of the cognitive and affective responses involved in the consumption of women’s swimwear. Unpublished Master’s thesis, Iowa State University, Ames. Hornik, J. (1992). Tactile stimulation and consumer response. Journal of Consumer Research, 19, 449–458. Hu, H. and Jasper, C. R. (2006). Social cues in a store environment and their impact on store image. International Journal of Retail and Distribution Management, 34(1), 25–48. Jeong, S.W., Niehm, L. and Fiore, A. M. (2006, November). The relationship of perceived shopping experiences, shopping values, and consumer patronage intentions towards apparel specialty stores. Poster session presented at the annual meeting of the International Textiles and Apparel Association, San Antonio, TX. Kalafatis, S. P., Pollaed, M., East, R. and Tsogas, M. H. (1999). Green marketing and Ajzen’s theory of planned behaviour: A cross market examination. Journal of Consumer Marketing, 16(5), 441–460. Kang, J. and Kim, Y. K. (1995). Consumer perception of shopping costs and its relationship with retail trends. Journal of Shopping Center Research, 2(1), 27–61. Retrieved February 23, 2006, from http://www.reuw. washington.edu/JSCR/IndArticles/Kim_N297.pdf Kim, H.-S. (2000). Examination of brand personality and brand attitude within the apparel product category. Journal of Fashion Marketing and Management, 4(3), 243–252. Kim, Y.-K., Sullivan, P., Trotter, C. and Forney, J. (2003). Lifestyle shopping center: A retail evolution of the 21st century. Journal of Shopping Center Research, 10(2), 61–94. Retrieved 6 February 2006, from http:// www.reuw.washington.edu/JSCR/IndArticles/Kim_N203.pdf Kim, Y., Jolly L., Fairhurst A. and Atkins, K. (2005). Mixed-use development: Creating a model of key success factors. Journal of Shopping Center Research, 12(1), 53–76. Retrieved 23 February 2006, from http:// www.reuw.washington.edu/JSCR/2005Articles/JSCRV12_1Preface2.pdf Kim, Y.-K., Kang, J. and Kim, M. (2005). The relationships among family and social interaction, loneliness, mall shopping motivation, and mall spending of older consumers. Psychology and Marketing, 22(12), 995–1015. Knasko, S. C. (1995). Pleasant odors and congruency: Effects on approach behavior. Chemical Senses, 20, 479–487. Koestler, A. (1981). The three domains of creativity. In: D. Dutton and M. Krausz (Eds.) The Concept of Creativity in Science and Art, pp. 1–17. Boston, MA: Marinus Nijhoff. Koufaris, M., Kambil, A. and LaBarbera, P. A. (2001–2002). Consumer behavior in web-based commerce: An empirical study. International Journal of Electronic Commerce, 6(2), 115–138. Lee, P.-M. (2002). Behavioral model of online purchasers in E-commerce environment. Electronic Commerce Research, 2, 75–85. Lee, H.-H., Fiore, A. M. and Kim, J. (2006). Technology acceptance model, shopping orientation, image interactivity technology, and consumer responses toward an online retailer. International Journal of Retail and Distribution Management, 34(8), 621–644. Lindauer, M. S. (1984). Physiognomy and art: Approaches from above, below, and sideways. Visual Arts Research, 10(1), 52–65. Mano, H. and Oliver, R. L. (1993). Assessing the dimensionality and structure of the consumption experience: Evaluation, feeling, and satisfaction. Journal of Consumer Research, 20, 451–466. Mattila, A. S. and Wirtz, J. (2001). Congruency of scent and music as a driver of in-store evaluations and behavior. Journal of Retailing, 77(2), 273–289. McMahon, E. T. (1999). Smart growth trends. Planning Commissioners Journal, 33, 3–5. McNickel, D. (2004, May 1). Hands on brands: Feel them. Do them. Be them. (Cover story: Experiential marketing). New Zealand Marketing Magazine. Retrieved 5 February 2006, from http://www.highbeam.com/ library/doc3.asp?docid⫽1G1:117182235andrefid⫽SEO

The shopping experience

647

Menon, S. and Kahn, B. (2002). Cross-category effects of induced arousal and pleasure on the Internet shopping experience. Journal of Retailing, 78, 31–40. Meyer, D. J. C. and Anderson, H. C. (2000). Preadolescents and apparel purchasing: Conformity to parents and peers in the consumer socialization process. Journal of Social Behavior and Personality, 15(2), 243–257. Michon, R., Chebat, J. C. and Turley, L. W. (2005). Mall atmospherics: The interaction effects of the mall environment on shopping behavior. Journal of Business Research, 58(5), 576. Morganosky, M. A. (1987). Aesthetic, function, and fashion consumer values: Relationships to other values and demographics. Clothing and Textiles Research Journal, 6(1), 15–19. Morganosky, M. A. (1995). Consumer patronage of manufacturers’ outlets. Clothing and Textiles Research Journal, 13(4), 273–279. Moye, L. N. and Kincade, D. H. (2002). Influence of usage situations and consumer shopping orientations on the importance of the retail store environment. International Journal of Retail, Distribution and Consumer Research, 12(1), 59–79. Ogle, J. P., Hyllegard, K. H. and Dunbar, B. (2004). Predicting patronage behaviors in a sustainable retail environment: Adding retail characteristics and consumer lifestyle orientation to the belief-attitude-behavior intention model. Environment and Behavior, 36(5), 717–741. Oh, H., Fiore, A. M. and Jeong, M. (2004). Conceptualizing and measuring the four realms of tourism experience. Travel and Tourism Research Association (TTRA) 35th Annual Conference Proceedings. Montreal, Canada. Palmer, T. (2002, January 26). Sensing a winner: Touchy-feely marketing is being employed by FMCG companies as they seek alternatives to the traditional marketing media to give key consumers a comprehensive ‘brand experience’. (Experiential marketing.) Grocer. Retrieved 5 February 2006, from http://www.highbeam.com/library/doc3.asp?DOCID⫽1G1:82650200andnum⫽3andctrlInfo⫽Round18%3AProd%3ASR %3AResultandao⫽andFreePremium⫽BOTH Pine II, B. J. (1993). Mass customizing products and services. Planning Review, 21(4), 6–13. Pine II, B. J. and Gilmore, J. H. (1999). Experience economy: Work is theater and every business a stage. Boston, MA: Harvard Business School Press. Pop-up marketing store. (2006). EventNet. Retrieved 25 January 2006 from, EventNet USA Web site: http:// www.eventnetusa.com/popup.html Pop-up retail. (2005). Trend watching. Retrieved 6 February 2006 from TREND WATCHING.COM Web site: http://www.trendwatching.com/trends/POPUP_RETAIL.htm Postrel, V. (2003). The substance of style. New York: Harper Collins. Pullman, M. E. and Gross, M. A. (2003, Summer). Welcome to the experience economy: where you can check out, but you can never leave. Journal of Business and Management, 9(3), 215–231. Richards, G. (2001). The experience industry and the creation of attractions. In: G. Richards (Ed.) Cultural attractions and European tourism, pp. 55–69. New York: CABI Publishing. Richardson, P., Jain, A. K. and Dick, A. (1996). The influence of store aesthetics on evaluation of private brands. Journal of Product and Brand Management, 5(1), 19–28. Rintamäki, T., Kanto, A., Kuusela, H. and Spence, M. (2006). Decomposing the value of department store shopping into utilitarian, hedonic, and social dimensions. International Journal of Retail and Distribution Management, 34(1), 6–24. Rudd, N. A. and Tedrick, L. S. (1994). Male appearance aesthetics: Evidence to target a homosexual market? In: M. R. DeLong and A. M. Fiore (Eds.) Aesthetics of textiles and clothing: Advancing multidisciplinary perspectives, pp. 200–211. Monument, CO: International Textiles and Apparel Association. Salkin, A. (2005, June 26). Shivering for luxury. New York Times. Retrieved November 22, 2006 from http://www.nytimes.com/2005/06/26/fashion/sundaystyles/26air.html?ex⫽1277438400anden⫽ f59eb0516c452941andei⫽5090andpartner⫽rssuserlandandemc⫽rss Sayed, I. M., Farrag, D. A. and Belk, R. W. (2003). The effects of physical surroundings on Egyptian consumers’ emotional states and buying intentions. Journal of International Consumer Marketing, 16(1), 5–27. Scheller, H. (1993). The meaning of apparel quality through the perceptions of apparel production managers and operators. Unpublished master’s thesis, Iowa State University, Ames. Schmitt, B. H. (1999). Experiential marketing. New York: Free Press. Schiro, A. M. (1990, April 22). Fabric, fit, and rose-tinted mirror. New York Times, p. 10F. Sherry, J. F. (1990). A socio-cultural analysis of a Midwestern American flea market. Journal of Consumer Research, 17, 13–30. Simon D. Tour live brings music and fun to the mall, delivering teen segment to top brands. (2003, August 8). PR Newswire. Retrieved 5 February 2006, from http://www.highbeam.com/library/doc3.asp?DOCID⫽1G 1:131667753andnum⫽1andctrlInfo⫽Round18%3AProd%3ASR%3AResultandao⫽andFreePremium⫽ BOTH

648

Product Experience

Simon malls, an attractive alternative to traditional advertising. (2004, May 20). PR Newswire. Retrieved 5 February 2006, from http://www.highbeam.com/library/doc3.asp?DOCID⫽1G1:132382262andnum⫽1a ndctrlInfo⫽Round18%3AProd%3ASR%3AResultandao⫽andFreePremium⫽BOTH Sirgy, M. J., Grewal, D. and Mangleburg, T. M. (2000). Retail environment, self–congruity, and retail patronage: An integrative model and a research agenda. Journal of Business Research, 49, 127–138. Sit, J., Merrilees, B. and Birch, D. (2003). Entertainment-seeking shopping center patrons: The missing segment. International Journal of Retail and Distribution Management, 31(2), 80–94. Small Business – Wolves in Shops’ Clothing. (2005, February). FORTUNE SMALL BUSINESS. Retrieved 6 February 2006, from http://blog.brandexperiencelab.org/experience_manifesto/2005/02/small_business_.html Snell, M. (2006). Experience synergy: Design and technology at tomorrow’s retail. White paper retrieved 1 March 2006 from http://www.jackmorton.com/us/philosophy/whitepaper.asp Solomon, M. R. (2005). The hunter gets captured by the game. Marketing Research: A Magazine of Management and Applications, 17(1), 26. Song, K., Fiore, A. M. and Park, J. (2007). Telepresence and fantasy in online apparel shopping experience. Journal of Fashion Marketing and Management, 11(4), 553–55. Sweeney, J., C. and Wyber, F. (2002). The role of cognitions and emotions in the music–approach–avoidance behavior relationship. Journal of Services Marketing, 16(1), 51–69. Swinyard, W. R. (1993). The effect of mood, involvement, and quality of store experience on shopping intentions. Journal of Consumer Research, 20, 271–280. The benefits of effective retail lighting. (1991, December). Chain Store Age Executive, pp. 5A–7A. The Manifesto. (2004). IXMA International Experiential Marketing Association Website. Retrieved January 30, 2006 from http://www.ixma.org/manifesto.htm Thomas, J. B. and Peters, C. L. O. (2006). The underground mall: An investigation of factors influencing gray market consumption. International Journal of Retail and Distribution Management, 34(2), 106–120. US Census Bureau (2005). Quarterly e-commerce sales. Retrieved 18 February 2006, from http://www.census. gov/mrts/www/data/pdf/05Q4.pdf van der Heijden, H. and Verhagen, T. (2004). Online store image: Conceptual foundations and empirical measurement. Information Management, 41(5), 609–617. Wakefield, K. L. and Baker, J. (1998). Excitement at the mall: Determinants and effects on shopping response. Journal of Retailing, 74(4), 515–539. Wilde, S. J., Kelly, S. J. and Scott, D. (2004). An exploratory investigation into e-tail image attributes important to repeat, internet savvy customers. Journal of Retailing and Consumer Services, 11, 131–139. Wilhelm, W. B. and Mottner, S. (2005). Teen and shopping mall preference: A conjoint analysis approach to understanding the generational shift toward an experience economy. Journal of Shopping Center Research, 12(1), 23–52. Retrieved 23 February 2006, from http://www.reuw.washington.edu/JSCR/ 2005Articles/JSCRV12_1Preface2.pdf Winakor, G., Canton, B. and Wolins, L. (1980). Perceived fashion risk and self-esteem of males and females. Home Economics Research Journal, 45–56. Yalch, R. F. and Spangenberg, E. R. (1990). Effect of store music on shopping behavior. Journal of Consumer Marketing, 7(Spring), 55–63. Zeithaml, V. A. (1988). Consumer perceptions of price, quality, and value: A means-end model and synthesis of evidence. Journal of Marketing, 52, 2–21. Zeithaml, V. A., Berry, L. L. and Parasuraman, A. (1996). The behavioral consequences of service quality. Journal of Marketing, 2, 31–46.

CLOSING REFLECTIONS

Company slogans such as ‘Connecting people’ (Nokia), ‘I’m lovin’ it’ (McDonald’s), ‘Take care’ (Garnier), and ‘Auto Emoción’ (Seat) indicate that industrial companies all over the world are nowadays well aware of the fact that their ultimate aim and value is not in their products as such, but in the experiences they offer to their customers. Nokia suggests that through their mobile solutions, people can connect to friends and relatives around the globe, independent of place and time. Unilever’s mission to ‘add vitality to life’ suggests that their food products allow people to lead a healthy and energetic life. These personal and social experiences are meaningful to people, and form the basis of personal happiness and well-being. Thereby, these target experiences provide products their reason to exist – their ‘raison d’être’. Companies invest time and effort to identify the meaningful life experiences they may provide, and they keep exploring new ways to nurture these experiences. Yet, strictly speaking the mobile phones made by Samsung, Sony Ericsson, Motorola, or Siemens also allow people to connect to others and, probably, they are approximately equally successful in establishing these connections. So, what makes a person select one mobile phone over the other ones? Here is where the concept of product experience comes into play. Some phones are easy to operate, others are fancy, ‘cool’, or elegant, while others are pleasant to hold and ‘just feel right’ when you push their buttons. These product experiences are instrumental to the life experiences defined above and they are far from arbitrary. They define how the connection between two people is realized and unfolds. Is this connection made in a smooth and transient, uninterrupted and simple, surprising and fascinating way? According to Philips, all interactions with their products should be characterized by ‘Sense and simplicity’. That seems a fair choice. After all, why should we want to complicate matters? But designers’ abilities to create valuable and meaningful product experiences go far beyond ensuring sensuous pleasure and ease of use. How and where Product Experience Copyright © 2008 Elsevier Ltd.

649

650

Product Experience

to find the appropriate experience is, therefore, an important design step, and one that could do with much more support from empirical research. Once the desired product experience has been defined, the designer has an almost unlimited set of means to realize it. Depending on the (corporate) context in which designers work, they can choose a tangible or intangible solution; they can use the latest technologies and novel materials, or they can combine multi-media platforms with services; they can incorporate all kinds of sensory information in their design, and they can make these combinations compatible or conflicting; they could play with meanings through associations, metaphors and various cultural connotations. This book has mainly addressed the experiential consequences of choosing one or more of these options. For example, if the designer changes the product’s material – let’s say from aluminum to plastics – this change has consequences for its tactual and visual aesthetics, for the symbolic and social meaning attached to the product, for the emotions it can elicit, and for its durability, reliability and performance. Hence, this decision affects the way the product is experienced in multiple ways, and it will ultimately affect the quality of the life experience this product is supposed to support. Looking back at the chapters in this volume, we can conclude that all chapters touch upon and clarify some patterns in this intricate network of (inter)relationships. This understanding is of great importance and is highly valuable. However, as we have shown above, decisions at the technological, design, marketing, engineering, and psychological level affect each other in many different ways. As a consequence, we strongly believe that the field of product (or user) experience needs to be studied by new, multidisciplinary approaches. This implies that gaps between disciplines will need to be bridged, and that scientists and practitioners with different backgrounds will need to cooperate and be open-minded concerning developments in neighboring fields. Hopefully, the field of product experience can develop as a discipline in its own right, and we will be able to see the fruits of this movement in the next edition of this book in a few years time. For now, we would like to end by expressing our thanks to a number of people and institutions who contributed directly or indirectly to the making of this book. First of all, we would like to thank all authors who wrote a contribution to this book and the staff at the publisher’s office (Fiona Barron, Ben Davie, Barbara Makinster, Nikki Levy and many others) for their excellent work. We also thank our colleagues at the Faculty of Industrial Design Engineering of Delft University of Technology, who have helped us in finding knowledgeable authors for the various topics we wanted to cover. The Netherlands Organization for Scientific Research (NWO) is acknowledged for funding one of the editors throughout the editorial process (MAGW VIDI grant 452-02-028). And last but not least, we would like to thank our families and friends, who have missed us during evenings and weekends, while we were working on this book… The Editors

INDEX

A Acquisition, of expertise Dreyfus and Dreyfus on, 226 Ericsson and Charness on, 226–227 in design process de:cycle model of, 228–231 Active learning, 502 Active touch, 45 Active user theory, 502 Adaptation level theory, 404 Aesthetic experiences Coleridge, Samuel Taylor on, 243 Schlegel, August on, 243 Schlegel, Johann on, 243 Aesthetic judgments consequences of interactive products, value of, 295–296 self-referential nature, 296–298 task performance, impact on, 298–299 correlates of, 291–294 definition of, 288–291

Product Experience Copyright © 2008 Elsevier Ltd.

in human-computer interaction (HCI), 288–291 Aesthetic object, 242–243 Aesthetics definition of, 260 in product design meaningful properties, 266–271 organizational properties, 261–266 unifying properties, 262–264 misconception of, 260–261 research in, 261 universal principles of cross-sensory aesthetics in, 276–277 cultural differences over, 278–279 evolutionary aesthetic preferences in, 273–276 individual differences over, 277–278 study on cross-culture, 271–273 Affect dispositions, 384, 392–383 Affect grid, 415

Affective engineering, 480, 494 Affective experiences, 111–112 Affective states, 381, 383, 385 Affinities, 15 Affordances, 309–317 and digital information, 316 and product semantics, 315 application, in design for interaction, 316–317 concept, 309–310 Gibson, James, theory of, 310–311 contradictions in, 311–313 Aircraft interior, 455–459 airline tickets, 455–456 design concept, 457–458 increasing knee space, 458 long-haul flights, useful for, 458 look, 458–459 more movement, 458 privacy, 459 maintenance costs, 455 passengers, concerns, 456 delay, 456 legroom, 456 lost luggage, 456 rude flight attendants, 456

651

652 Albedo, 16, 23–24, 28–29, 31 Ambient intelligence, 517, 527 Angle of incidence, 24 Anomaly recognition, 204–205 Anxiety, 404 purchase decision, 404 Appearance, of nondurable consumer goods carrier of brand equity, 582–585 mediating effects of, 581 role of, 583 as carrier of brand equity, 583 visual cues, 597 Apple iPod, 519 Appraisal, 389–390 checks, 391–393 Arnheim, R., on Gestalt principles in aesthetics, 245–246 Art and Experience, 244 Art and Visual Perception, 245 Artifacts, 353–375, 623–624 aesthetic control, 623 design of, 354–375 influence on human psychology, 623 meaning of, 355, 356, 362 species of, 357 Artistic change, theory of, 279–280 Assembly line, 449–451 razor machines, 449–450 Attention bottlenecks, 202 Attention switching, in multisensory modalities, 139–142 Ho and Spence on, 141 Spence and Driver on, 139 Spence and Read on, 142 ventriloquist illusion and, 140 Attributed affect, 382–383 object, 382–383 stimuli vs. object, 383 Attuned space, and design, 245 Auditory perception, process of, 77–81 conceptual framework of flowchart on, 78 inner ear, 79 memory’s influence in, 80–81 psychoacoustical measures, 79–80

B Bahco Tools, 452–453 11-step design process, 451–452 erogotools, 452 Bas-relief ambiguity, 22–24

INDEX

Beauty judgments. see aesthetic judgments Beauty/aesthetics, judgmental approach to. see aesthetic judgments Bergson, Henri, on lived experiences, 244 Berlyne, D. E., on visual stimuli, 339 Bidirectional Reflection Distribution Function (BRDF), 16–18, 30, 34, 35 Biology of cognition, 356 Blaich, Robert, 307 Body language, of objects familiarity, 58–59 integrity, 58 intentions, 57–58 perfect match, 58 personality, 56–57 physical skills, 59–60 power and control, 59 tactual noise, 60 tactual transparency, 60 Brainstorming, 468, 479 Brand equity theory, 583 Brick and Mortar retail centers, 629 venues, 629

C Candidate receptors, 94 Carbon and Leder, on innovativeness, 270 Cartesian dualism, 356 Cell offices, 624–625 employees of, 624 Central location testing (CLT), 563 Centrality of visual product aesthetics (CVPA), 295 Chemesthesis anatomy of, 107 mucous membranes in, 107 nerve fibers in, 107 Chemical somesthesis, 107 Chemoreceptors, 92 Chemosensory experiences measurement of, 109 non-sensory effects on age, 117–119 context, 112–113 cultural factors, 120–121 expectations, 113–117 gender, 119–120 information, 113–117 social factors, 120–121 of consumers, 109 of experts, 109

quantification of, 109 by consumers, 110–112 by trained panels, 109–110 Chisei, 478 meaning and definition, 478 Closure, 30–31 Co-experience as a sensitizing concept, 464 building prototypes, 469 designing prototypes creating conditions, 469 for social organization, 469 required conditions, 468–469 naturalistic research setting, 468 openness, 468–469 social setting, 468 time span, 469 unfolding events, 469 interpretations, 462–464 pragmatic philosophy, 463 short lifespan, 467 social interaction, 463 storytelling, 463 types of actions lifting up, 465–466 reciprocating, 466–467 rejecting, 467–468 Coates, D., on product perception, 339 Cognition, and designs macro-cognitive perspective, 204–210 micro-cognitive perspective, 201–204 Cognitive approach, to design, 242–243 Cognitive engineering, by Norman, Donald, 310 Cognitive psychology, 502 cognitive flow, 507 first generation of HCI theory, 502 Cognitive theories, 386–387 Coleridge, Samuel Taylor, on aesthetic experience, 243 Collative-motivation model, 264 Collective efficacy, 512 Collinear transformation, of object’s shapes, 15 Color, and flavor interaction of, 590–592 relationship between, 590 Color, of nondurable consumer goods effect of, 586, 590 effectiveness of, 588–589 experimentation in marketing, 588–590 food, 590

653

INDEX

relativity of, 585–586 role of, 583–585 symbolic meaning of, 585 wavelength of, 585 long, 583, 589 short, 583, 589 Color, of objects, 19–22 Comfort definition, 441 design, 444–447 short-term vs. long-term, 444 design methods, 444–447 experience, 444–445, 454–455, 457–459 hand tools, 443–444 improvement, 447 influential factors, 442–443 context level, 442–443 human level, 442–443 measurement, 445–447 product level, 442–443 model, 442 seating studies, 442–443 vs. discomfort, 441–444 Comfort ratings, 455–456 Commission Internationale de l’Eclairage (CIE) chromaticity diagram, 20–21 color matching functions, 19–22 Common fate, 30–31 Communication process, in product designing, 185–188 perception, 186 physical actions, 187 symbolic actions, 187 Complexity and variety, of aesthetics in product design, 264–265 Computer games, 502 Computer Supported Collaborative Work (CSCW), 474 Computerized Axial Tomography (CAT) Scanners, 516 Conceptual model of product design, 596 Concerns, 390 types, 390 Conjunctive ambiguity, principle of, 265 Conscious awareness, of design objects, 247 Consequences, of aesthetic judgments self-referential nature, 296–298 task performance, impact on, 298–299 value of interactive products, 295–296 Consonant sounds, 71 Consumer behavior, 409, 415, 417

Consumer durables, 429–430 Consumption emotions duration of, 418 eliciting conditions, 404–408 identification, 414–415 measurement, 414–415 positive vs. negative, 407–408, 414 studies, 409–413 broad range, 410 single, 409–410 Consumption Emotions Set (CES), 415 Consumption experience, 630–631 Consumption hypotheses, 402–403 expectations, 402 influencing factors, 403 Consumption situation important, 406 novel, 406 product expectations, 406–407 product failure, 407 social comparisons, 406 Content analysis, 369 Contexts of artifacts, 362–375 anticipating, 368–371 artifact–context relationship, 362–364, 366, 373 interfacing, 363, 366–367 observing, 363–366 Contextual variables, in food product experience convenience, 568–570 eating duration, 567 food accessibility, 565–567 food choice, 568 physical environment, 570–572 service situations, 573 socialization/commensality, 572–573 Contrast illusions, 28–29 Contrast, of package, 586 Convergent illumination, 18, 19 Cook book solutions, 606, 617, 625 Coping strategy, 608–609 patterning, 609 personalization, 609 privacy, 609 Core affect, 381–382 causes, 381 circumplex model, 382 valence, 381 Correlates, of aesthetic judgments, 291–295 Cross-modal correspondence, between multisensory modalities, 142–146

Kemp and Gilbert on, 143 Schifferstein and Tanudjaja’s fragrance study, 143–144 Crowding and perception of privacy, 619 social facilitation, 619 Cultural identity risks of poor usability, 512 D Darwinian presupposition, 385 De:cycle model, of design process, 228–231 conceptual model in, 231 design hypothesis in, 230 exploration arc in, 228–229 implementation arc in, 230 Northwest Passage in, 229 observation arc in, 228 Decision making, by expert computer system, 220–221 Design process, of consequential product sounds, 81–86 analysis, 82–85 evaluation, 85–86 recording, 82 sound design concept, 85 Design project, at Delft University of Technology, 325–329 Design prototype. see IKEA style study Design research, 461–463 and experience, relationship, 462 based on academic discipline, 461 designer’s ideas, 474 extend traditional approaches, 461 Forlizzi and Ford’s model, 463 breaking down experience into, 463 in psychology, 461–462 in social sciences, 462 practical and academic experience, 462 research paradigm, 474–475 Rhea’s product role life cycle model, 463 Sander’s model, 463 Design specific experience, 612–613 Design strategies, 425–438 eco-design strategies, 428–429 Designers, 232–235 and users, 354 effective proposals, 369 technology-centered vs. human-centered, 354

654 Designers’ attachment to design, experimental study on, 248–252 Designs and attuned space, 245 cognitive approach to, 242–243 Gibsonian affordance, 242 Gestalt psychology in, 245–247 Lewin’s experimental phenomenology in, 247 macro-cognitive perspective, 204–210 micro-cognitive perspective, 201–204 perception in, 246 personal meaning of, 247–252 experimental study on, 248–252 phenomenology in, 243–245 social psychology in, 247 Determinants, 430–437 group affiliation, 435–436 memories, 436–437 pleasure, 430–432 self-expression, 432–435 Dewey, John on creative experiences, 244 on human-object interactions, 344 Dexterity functions, in product designing, 193–197 force exertion, 193–194 gripping power, 195 precision, 194–195 two handed operations, 195 DiamondHelp, 521 Differential emotions scale (DES), 415 Diffuse or ambient illumination, 18, 19 Digital Family Portraits system, 528 Digital games, 532–535 complex games, 542– 543 computer games, 532 Counter Strike, 534 games on Internet, 532 games on mobile phones, 532 Grand Theft Auto San Andreas (GTA), 533 hybrid games, 533 interactivity, 532 major genres, 533–535 platform games, 533 role playing, 533 simulation, 533 sports and driving, 533 strategy, 533 trivia and puzzle, 533

INDEX

The Sims, 534 video games, 532 Dilthey, Wilhelm, on aesthetic experience, 243–244 Discomfort factors, 442–443 model, 442 physical capacity, 442 physical process, 442 objective measures, 443 pressure, 446 vs. comfort, 441–444 Dissatisfaction, 407 events, 407 Divergent illumination, 18, 19 Dreyfus and Dreyfus, on acquisition of expertise, 226 Dufrenne, Mikel, on objects of use, 242–243 Duncker illusion, 28 Dynamic response, 339–340 Dynamic touch, 46 Dynamization. see dynamic response

E E-commerce, 505–506 trustworthiness, 505–506 design principles, 506 Eating duration, 567 Eco-design strategy, 428–429 person-product relationship, 429 product lifetime, 429 Ecological approach, to perception, 337–338 Efficiency of service quality, 635 Elasticity, 50 Electromyography (EMG), 447 Emotion studies broad range, 410–411 single, 409–410 Emotion theory, 381 Emotional bonding, 425–438 negative emotions, 426 positive emotions, 426 Emotional engineering, 479 Emotional experience, 428–429 events, 428 Emotions, 461–462 ambiguous emotions observed through social reflection process, 464 communicated through messages, 464 designing emotion, 462 emotional needs, product based, 462–463

expressed through sentic modulation, 464 inner emotions communicated through messages, 464 Empirical visual research, methodology, 595–600 Environmental sounds, 70, 72 Environmental stressors crowding, 608 coping strategy, 608–609 disorientation, 608 environmental deprivation, 608 physical environment, 607–609 influence of, 607–609 mental stress, 607–609 white collar workers, 608 Equality, 30–31 Ergomix, 445 Ergonomics design, 491–493 low lifting forklift trucks, 491–493 analyzing with Quantification theory I, 493 forklift prototype, 493 heel injuries, 492 improving suspension, 492–493 Kansei words selection, 493 Ericsson and Charness, on acquisition of expertise, 226–227 Ethnographer’s Challenge, 218 Experience dictionary meaning of, 243 origin of the word, 353–354 Experimental study, on designers’ attachment to their design Expertise acquisition of Dreyfus and Dreyfus on, 226 Ericsson and Charness on, 226–227 in design process, 228–231 and designers, 232–235 description of, 222–224 psychological assumption in, 222 Tempest model in, 223–224 eminent level of, 231–232 innovation of, 227–231 perspectives on, 217–219 study of, 219–222 by cognitive psychologists, 220 by Simon and Chase, 220–221 Experts characteristics of, 225–226 description of, 216, 224

655

INDEX

psychological strategies of, 225–226 Exploration, 361, 368 Exploratory procedures, for tactual properties, 46, 47 Expressive quality based benefits, 636–637 aroused emotion, 636–637 creative expression, 636–637

F Familiar size illusion, 593 Feelings, in tactual interaction action tendencies, 62 affection, 61 physical pleasure, 60–61 vulnerability, 62 Flavor–flavor conditioning, 101 Flex offices, 620 Folkes and Matta, 593 Food acceptance ratings, 564–565 Food choice, 568 Food consumption rates, 566, 569 Food product experience background to, 560–561 contextual variables in convenience, 568–570 eating duration, 567 food accessibility, 565–567 food choice, 568 physical environment, 570–572 service situations, 573 socialization/commensality, 572–573 in different contexts, 562–565 meals, 573–576 overview of, 559–560 product testing, 576–578 terminology in, 561–562 Force direction, 448–449 Forlizzi and Ford, on verbal reflection design, 247 Formants, 71 Framing of, shopping experience, 630–632 cognition Affect behavior model (CAB), 630–632 amalgamation of, 631–632 consciousness emotion value model (CEV), 630–632 amalgamation of, 631–632 Free-choice profiling, 111 Free-sorting procedures, 111 Functional requirements, 385–386 Funology, 500

G Game dimensions, 536 Game experience, 531–552 as a project, 551–552 complexity in games, 542–543 enjoyment, 546 expertise, 549–550 flow, state of mind, 543, 548 framework for describing, 539–551 game features, 535–538 affordances, 534 Challenge, 537 control, 537 fantasy, 536 mystery, 537 rules, 536 self-efficacy, 539, 547–548 intellectual emotions, 544 interestingness, 544 intrinsic motives, 550 MDA framework (Mechanics, Dynamics and Aesthetics), 535–536 motivation factors, 540, 542 multisession context, 534–535 risk of fatal error, 542 self involvement, 548 single session game experience, 534–535 social identity, 534–535, 550–551 visual attention, divided, 549 Game industry, 531–532 increasing popularity in, 532 negative effects, 532 public concerns, 532 vs. movie industry, 532 Gamers, 538–539 average age, 538 expert gamers, 550 gender differences, 531, 538–539 passionate gamers, 551 use of LAN, 538 Gatorade, 590, 591 Gauge figure, 24–25, 27 General experience, 612–613 Gestalt factors, 30–31 Gestalt laws, 30–31 Gestalt principles, in aesthetics, 245–247 Gestalt psychology, 245–247, 336–337 Gibson, J. J., 356, 359–361 on affordances, 310–311, 338 contradictions, 311–313 on designed object, 242 on human sensory systems, 92

Gibsonian affordance, 242 Giovannoni, Stefano, biscuit-box design by, 149 Glabrous skin, 53–54 Goals, 393–394 anticipating achievement, 394–395 human-product relationship, 393 probability check, 394 Golden section ratio, 262–263 Goniometers, 447 Good continuation, 30–31 Grand Theft Auto San Andreas (GTA), 533 Group affiliation, 435–436 Group level study, 444

H Hair treatment. see Milbon Desse Hairy skin, 54 Hand tools, 443–444, 446–448, 451–452 comfort factors, 443 design, 447–448 flow chart, 448 pressure distribution, 446–447 Hearing functions, in product designing, 176–179 sound detection, 177–178 sound localization, 178 speech discrimination, 178 Hearing-touch interactions, 147 Hedonic and utilitarian shopping value, 632 aspects of deriving, 632–633 Hedonic product attributes, 294, 296 Height state illusions, 593 Hemispherically diffuse illumination, 18, 19 Hidden Novelty (HN), 151 Hindu mythology, 371 Holbrook and Herschman, article, 630 consumer behavior, 630 Home use testing (HUT), 563 Hope, 403–404 Human computer interaction (HCI), 287, 309, 500–503, 512–513 aesthetic judgments in, 288–291 in house, 500 in workplace, 500 organizational usability, integral part of, 502 Human expressions, 335–336

656 Human-centeredness, 354, 360 designers, 354, 356, 361, 365, 368–370, 375 designs, 361, 368 Hypothalamus, 103

I IBM Corporation, 500 minicomputer office system, 500–501 Ibo robot, 519 IDEO, 468 IKEA style study, 470–471 experience of participants, 470 feedback on sound, 470–471 in home environment, 470–471 placing IKEA-esque lamp, 470–471 attachment with electrical devices, 470–471 memory trace function, 471 scenario interview, 470–471 private and social issues, 471 Illumination angle, 16 Illumination, of objects perceived form of, 29–30 types of, 18–19 Image schemas and product expression, 346–348 concept of, 345–346 Information processing program, 502 Fitt’s Law, 502 keystroke model, 502 InStink, 508 Intellectual functions, for product designing, 179–185 attention, 182 long-term memories, 183 visual-spatial thinking, 182–183 working memory, 181–182 Intelligent products central issues, in designing, 527–528 ‘task-oriented to experience driven’ perspective, 528 ‘use to presence’ perspective, 527–528 experience driven designing, 525–527 atmosphere control system, 525–526 future aspects, 528–529 European projects, 529 technology, role of, 528–529 diary tracking, 529

INDEX

in safety and health, 529 language learning for mentally challenged children, 529 robots, use of, 529 television and Internet, 529 in ambient intelligence, 517 intelligent user interface conference, 517 understanding of, 516–517 user interface capabilities, 517–518 with embedded electronics, 517 Intent bottlenecks, 202 Interaction Design, 500 Interactive intelligent product design, 515–516 Interactive systems, design of, 503 multidisciplinary view, 503 Interfaces, 354, 358–362 complex, 369 computer-related, 358 cultural practices, 358 descriptions, 359–360 dual vs. interactive, 359 human, 360, 375 exploration, 361 recognition, 360 reliance, 362 interaction, 358 Internet, 455–456, 517, 529 online shopping, 517 passengers, 456 social connectivity, 529 website report, 453, 457 Internet computing, 512 impact on health, 512 Interpretations, 462–464, 466–467 ISO 9241–11, 500

J Japanese design, 372 Joint activity, 205–206

K Kandinsky, Wassily, on basic plane expression, 334–335 Kansei in product design, 479 economical failure, 479 Kansei build, 479 product designers, role of, 479 meaning and definition, 478 Kansei engineering affective channel width, 481 affective flow, 481

affective windows, 481–482 change factors in, 480–481 experience of interaction, 481 fashion and trends, 481 personal interest, 481 time dependency, 481 classifications (models) Type I (Category Classification), 480 Type II (Kansei Engineering System), 480 Type III (Hybrid Kansei Engineering System), 480 Type IV (Kansei Engineering Modeling), 480 Type V (Virtual Kansei Engineering), 480 Type VI (Collaborative Kansei Engineering Designing), 480 criticism and drawbacks, 493–494 lack of innovativeness, 493–494 determining external senses, 480 Kansei domain, 483 Kansei engineering properties, 485 methodology, framework on, 482–488 origin, 478, 479 quantitative and qualitative approaches, 495 reductionism, 494–495 semantic and physical descriptions of product domain, 494 vs. holism, 495 study of, 482 channels, definition of, 482 degree of interaction, definition of, 482 using exterior design, 494 interior qualities, 494 Kansei words, 484 Kansei/Chisei model, 478 Klee, Paul, on artistic expression, 335 Klein umbrella, 13–14 transformation in, 14 Klein, Felix, on transformations, 13 Knowledge bottlenecks, 202 Krippendorff and Butter, on product semantics, 306 L Labeled Affective Magnitude (LAM), 111

657

INDEX

Laird, D. A., on odor-texture interactions, 147 Lakoff and Johnson, on metaphor, 344–345 Lakoff and Johnson, seminal work, 373 Lambert’s Law, 16 Language, use of, 354 Law of Prägnanz, 30, 246 Lebensphilosophie, 244 Legroom, 456–457 Lewin, Kurt, social psychology of, 247 Lifetime, of product, 429 Lifting up, 465–466 communicating emotions, 466 impression management, 465 initial emotional response, 465 making stories, 465–466 process, in social interactions, 465 Lipps, Theodor, on dynamic response, 340 Local postural discomfort (LPD), 444–445, 447 Locomotion functions, in product designing, 188–191 walking and balance, 189 Ludwig Wittgenstein, suggestion, 356

M Macro-cognitive functions and micro-cognition, difference in, 206–208 and technological developments, 208–210 adaptive capability, dimensions of, 209–210 anomaly recognition, 204–205 joint activity, 205–206 replanning activities, 206 sense making, 205 Maintaining face, 468 key motivator, 468 Managerially oriented research, 418 managers, 418 Marketing literature, 461 academic experience research, 462 Marketing trends, emergent, 639–643 customized, 640–641 consumer’s role, 640–641 market optimization software, 640–641 experiential marketing, 639–643

pop-up retail, 642–643 holistic, 641–642 interactive, 640–641 pervasive, 639–640 experience economy, 639–640 Pine and Gilmore, 639–640 recreational equipment Inc. (REI), 639 transient, 642–643 Marketing, color experimentation in, 588–590 Martindale, C., theory of artistic change by, 279–280 Masking, of tones, 79 Mass customization services, 434 Nike, 434 Nokia, 434–435 Material properties, of objects perceived form of, 28–29 reflection in, 15–18 Maximum acceptable work pace (MAWP), 450 Maximum effect for minimum means, principle of, 265–266 Mazda Miyata Project, 488–489 brain storming sessions, 488–489 human machine unity (HMU), 489 sporty low price car, 488 Mazda Motor Corporation, 479, 488 Meaningful properties, of aesthetics in product design, 266–271 familiarity, 267–268 innovativeness, 269–270 Most Advanced, Yet Acceptable (MAYA) principle in, 270 prototypicality, 268–269 Meaningfulness, 366 Meaninglessness, 371–372 Meanings, 354–357 theory, for artifacts ecologies, 357 in genesis, 357 in language, 357 in use, 356 Meissner’s corpuscles, 54 Memories, of determinant, 436–437 age constraint, 436–437 odors implementation, 436 Memory Trace, 471 Mental imagery, 137–138 Merleau-Ponty, on human-object interactions, 344 Metaphors in design, 246 in product expression, 340–342

Lakoff and Johnson on, 344–345 linguistic vs. non-linguistic, 373–375 Micro-cognition and macro-cognitive functions, difference in, 206–208 bottlenecks in, 202–204 concept of, 201–202 Milbon Desse, 489–491 arboART, 490–491 hair care survey, 489–490 hair problems, in ladies, 489 preparing questionnaire, 489–490 selecting Kansei words, 490 soft and breezy hair, 490 work flow, 490 zero-level Kansei concept, 490 Mixed emotions, 394–396 Mobile multimedia, 465–468 data, 475 etiquette, 468 trial and error, 468 Mobile phones, 516, 528. see smart phone as database and communication device, 516 as social connectivity, 516 Bluetooth, 517 for personal expression, 516 Mock-ups, 445 Monochromatic light, 19–20 Moods, 383–384 causes, 383 influences emotional responses, 383 human-product interaction, 384 unpleasant vs. pleasant moods, 383 Morphome, 469 designing presentations, 469 gathering participants’ experience in interviews, 469 installing in home environment, 469 proactive information technology, 469 date collection and interpretation, 469 using prototypes, 469 Most Advanced, Yet Acceptable (MAYA) principle, 270 Motivations, for tactual interaction, 46–48 map on, 48

658 Mr Java, 508 Multi-level analysis, 387–388 Multicomponential phenomena, 384–385 behavioral reactions, 384 expressive reactions, 384–385 physiological reactions, 385 subjective feeling, 385 Multisensory modalities attention switching in, 139–142 audition in, 136 coherence between, 148–151 cross-modal correspondence between, 142–146 dominance of, 151–154 interactions between, 146–148 olfactory cues, 136 sense blocking method for, 135–136 split-modality approach for, 135 visual information, 136 Munsell color system, 21–22 Musculoskeletal disorders, 447, 449–451 Musical sounds, 72 MusicCube, 522–524 comparison with iPod, 522–523 N Narratives, 368 Naturalistic decision making (NDM), 221 Negative emotion, 404, 406, 415 Neophobia, 100 Noise-producing musical instruments, 71 Norman, D., on impact of beautiful products, 298–299 Norman, Donald, cognitive engineering by, 310 Novelty of package, 586 O Object expression. see Product expression Objects, 355 perceived properties of illumination, 29–30 material, 28–29 shape, 24–28 physical properties of color, 19–22 illumination, 18–19 material, 15–18 shape, 13–15 SMI triangle, 22–24 reflection of, 15–18

INDEX

visual perception of, 30–35 Observations, 371–372 Odor preferences, 106 Odor-texture interactions, research on, 146–147 Office design, importance, 607 Office environment coping strategies in, 607–609 physical, 605–607 Olfactory adaptation, 104 Olfactory bulbs, 103 Olfactory experience modification of, 105–106 odor mixtures in, 104–105 Olfactory receptors for pheromones, 102–103 mechanism of experiencing, 101–103 nasal anatomy of, 101–102 processing, 103 Oral anatomy, 93 Orbitofrontal cortex (OFC), 94–95, 103 Organizational properties, of aesthetics in product design, 261–266 complexity and variety in, 264–265 unifying properties in balance, 262 proportion, 262–263 symmetry in, 263–264 Ownership, of product, 427

P Packages role of, appearance, 583 size, 592 effect on consumers, 592 effect on marketer, 592–593 visual test stimuli, 589 Paint scraping, 451–455 marketing strategy, 455 painters, health problem, 452 redesign, 451–452, 455 triangular scraper, 451 Papillae, 93 Parallel illumination, 18, 19 Participatory ergonomics, 444 Passive touch, 45 Patronizing, 473 PC Dinners, 508 Perceived object properties illumination, 29–30 material, 28–29 shape, 24–28 Perceived value, 632–636 affects of, 632–636

co-mingling of multiples, 633 typologies, 633–636 Perceptual systems, 92 Person-product relationship, 429 Personalization of workstation, 624–625 Phenomenology, in design, 243–245 Philips DAP, 449 Physical environment, in food product experience, 570–572 Physical stimuli ambient factors of, 617–621 air quality, 620–621 light, 620–621 temperature, 620–621 influence of, 617 noise and sound, 618–619 unwanted sound, 618 Pitch-producing musical instruments, 71–72 Pleasure, 386–387, 430–432 appearance, of product, 431 primary function, of product, 430 superior utility, of product, 430–431 surprise, use of, 431 types, 387 Pleasure-arousal-dominance (PAD), 415 affect grid, 415 Post-purchase product evaluation, 403 Posture/movement, 445– 447, 454 Pragmatic philosophy, 463–464 significance of observations, 463 use of devices by children, 463–464 Prepurchase emotions, 403 Pressure distribution, 445 Pretentious semiotizations, 373–374 Primary taste cortex, 94 Product appearance calibration for novelty of, 596–597 carrier of brand equity, 582–584 role of color, 583–585 roles for package, 583 visual components of, 582 Product attachment, 425–438 definition, 425–426 personal meaning, 427 special meaning, 427 utilitarian meaning, 427 designer implications, 428–430

659

INDEX

determinants, of products, 430–437 objects, 427 Product categories conditional, 401 extraordinary, 401 mundane, 401 Product designing, 380–381, 386 anthropometry in, 166–167 human capabilities and, 167–171 cognitive, 169–170 motor capability, 170 prevalence of reduced, 168–171 sensory, 169 Product emotion, 379–396 affect dispositions, 392–393 approaches, 386–391 appraisal, 389–391 pleasure, 386–387 process-level, 387–388 basic model, 389 explanation, basic requirements of, 386 goals, 393–394 sources, 391–396 standards, 394 Product expression and Gestalt psychology, 336–337 Arnheim, Rudolf on, 337 ecological approach to, 337–338 image schemas and, 346–348 metaphors in, 340–342 semiotics in, 342–343 Product personality, 432–433 irreplaceability factor, 433 Product personalization, 433–434 co-designer, 434 design authority, 434–435 Product properties, 480–488, 490 Product semantics, 356 application of, 307–308 concept of, 306–307 elements of, 308–309 Krippendorff and Butter on, 306 Product sounds consequential, 71, 76 design process of, 81–86 domain of, 70–71 spectral-temporal structure of, 72–77 control systems in, 73–74 spectrograms, 74–77 use of, 69 Product use, meaning in, 305–306 affordances in, 309–317 and product semantics, 306–309 design perspective in, 317–322

experimental analysis on, 318–320, 325–329 usecues in, 322–325 Propylthiouracil (PROP), 94 Prototyping, 468–469 Proximity, 30–31 Proximity of interaction, 482 degree of interaction, 482 interests, 482 prior experience, 482 Proximity of presentation, 481–482 Psycho-social factors job satisfaction, 443 social support, 443 Psychoacoustical measures, 79 Psychoacoustical metrics, 79 Psychobiological approach, to arousal, 339 Psychological assumption, of expertise, 222 Psychological strategies, of experts, 222–223 Purchase, 405 smart-shopper, 405 Purchase decisions, 428 Q Quality function deployment (QFD), 479 R Raghubir and Krishna, 593 Reach and stretch functions, in product designing, 191–193 Reality negotiation, 404 Reciprocating, 466–467 creating ideas, from story telling, 466 gift giving, 466 in communication products and technologies, 467 discouraging factors, 467 negative results, 467 responding gestures, 466 small greetings and favors, 466 Recognition, 360– 361 Recognition by components theory (RBC), 595 Recording, of consequential product sounds, 82 Reflection, of objects, 15–18 albedo in, 16 Bidirectional Reflection Distribution Function (BRDF), 16–18 illumination angle in, 16 Lambert’s Law, 16 shading in, 16

Reflective emotions, 388, 393 Rejecting, 467–468 types of rejection, 467 passive and active, 467–468 Relationship between office type and design factors, 614–616 office design, 614–616 workspace design, 614–615 workstation design, 614–616 importance of, 615–616 Reliance, 361 background, of interface, 361 Repeated-evaluation technique (RET), 270 Replacement process, 429–430 Replanning activities, 206 Robocrop, 508 Rough set analysis, 495 S Scandinavian design, 372 Schifferstein and Tanudjaja’s fragrance study, 143–144 Schlegel, August, on aesthetic experiences, 243 Schlegel, Johann, on aesthetic experiences, 243 Second-order understanding, 354, 356, 367, 370 technology-centered vs. humancentered designers, 354 vs. first-order understanding, 354 Self-expression, 432 personal identity, 432 Semantic description of environments (SMB), 487 Semantic differential method, 487 Semantic differential scales (SDscales), 479, 483 Semantic differential technique, 485 Semantic rating scales, 526 Semantic space, 483–487 spinning, 483–484 higher level Kansei, 483–484 Kansei structure identification, 484 Semantic structure identification, 484–485 tools and methods, 484–485 Semantic structures, 414 Semiotics, 342–343, 355 Sense making, 205 Sensitizing concepts, 464 co-experience, 464 observations and interpretation activities, 464 Sensory dominance, 151–154 Sensory imagery, 138–139

660 Sensory qualities, benefits of, 633–636 beauty, 634 sensual pleasure, 634 utilitarian benefits, 632–634, 637–638 quest for knowledge, 637–638 self acceptance, 637–638 social acceptance, 637 social acceptance/affiliations, 637 status, 637–638 Service qualities, of retail stores, 635 Shading, in reflection of objects, 16 Shanteau, J., on psychology of experts, 222–223 Shape and size appearance interaction of, 592–595 Shape, of objects perceived form of, 24–28 transformational approach, for differentiation of, 13–15 affinities, 15 collinear transformation, 15 order transformations, 13–14 similarity transformation, 14 Shape–material–illumination triangle. see SMI triangle Shopping experience expectations of, 629 framing of, 630–632 information processing approach, 630 of consumers, 629, 631 online, 629 shopping malls, 629 Similarity transformations, of object’s shapes, 14 Simon and Chase, study on expert chess players by, 220–221 Sims, 534 Size and shape, research on, 593–595 Size of context, 372–373 Skin sensations cold, 55 itch, 55 light touch, 54 map on, 53 pain, 55 physical pleasure, 55 pressure, 54 tickle, 55 vibration, 54 warmth, 55 Smart phones, 505–506 calendar feature, 506

INDEX

personalizing pictures and images, 505 saving and using text messages, 505 security, 505 SMI triangle, of objects perception of, 33–34 physical properties of, 22–24 Social capital systems and applications to enhance, 513 Social interaction maintaining face, 468 non-verbal and verbal communications, 465 norms and customs, 465 social object, 463 story telling, 463 turns in conversations, 465 Social judgment theory, 404 Socialization/commensality, in food product experience, 572–573 Sound design concept, of consequential product sound, 85 Sound, of coffeemaker, 83–84 in time–amplitude domain, 83 in time–frequency/intensity domain, 83 Space of properties, 485 spanning, 485–486 designers’ role, 486–487 obtaining raw data tools, 486 prioritization, 485 source of collection, 486 Spatial mapping, of frequency, 79 Spectral-temporal structure, of product sounds, 72–77 control systems in autonomous, 73 partly autonomous, 73–74 user-dependent, 74 Spectrograms, of product sounds alarm clock, 74–75 electrical toothbrush, 75 vacuum cleaner, 76 Stakeholders, networks, 357–358 Standards, 394 legitimacy, 394 Stereoscopic vision, 35 Stimulus, 390–391 actual vs. associated stimuli, 391 events, 390–391 Stress related diseases, 608 Suchman, L. A., on product use, 305, 320–321 Survey, of Mano and Oliver, 631–632 Sustainable consumption, 429–430 environmental factors, 429

Swiss Airlines, 457 Symbolic artifacts, 621–625 color, 622–623 determining perception of, 622–623 warm, 623 Symbolic interactionism, 464 Blumer’s theory, 464, 467 in sociology, 464 sensitizing concepts, use of, 464 study of interactions and data collection methods, 464 Symbolic quality based benefits, 637–638 alternative existence, 637–638 identity existence, 637–638 Symmetry, 30–31 Synesthesia, 143 Synthesis, 487 as core of Kansei engineering, 487 identification of semantic and physical structure, 487 linking semantic space and space of properties, 487 tools and methods, 487 System prototype, 471–473 combining elements, 471 going to sleep routine, 472–473 Misterhouse software, 471 participants interview, 471 process of waking up, 471–472 programming in PERL, 471 X10 home automation system, 471–472 T Tactual Experience Guide, 41, 42, 63–64 Tactual properties, of objects elasticity, 50 hardness, 50 map on, 49 plasticity, 50 shape, 51 size, 51–52 temperature, 50 texture and patterns, 50–51 weight and balance, 52 Tactual sensations, by objects, 52–56 body senses in, 55 skin in glabrous, 53–54 hairy, 54 sensations, 54–55 sensors, 54 Task intensity, 448–449 high vs. low, 448

661

INDEX

Taste aversions, 100 Taste experience learning’s effect on, 100–101 modification to, 97–99 oral anatomy, 93 perceptual phenomena adaptation, 95–96 receptor mechanisms for, 93–94 Taste preferences, 100–101 Taste receptor-proteins, 93–94 Taste suppression, 96–97 Taste-referred olfaction, 97–98 Taste-smell interactions, 148 Technology as Experience, 503–504 Technophobia, 370–371 Television viewing, 512 Temperature flow, 50 Tempest model, of expertise, 223–224, 232 Terminology, in food product experience, 561–562 Text editing, 501 vs. typewriting, 501 Texture perception, 50–51 The Electronic Guidebook, 509 museum, installation in, 509 The Gods Must Be Crazy, the movie, 365–366 The Phenomenology of Aesthetic Experience, 242 The Psychology of Everyday Things, 500 The semantic turn, 356 informatives, 356 Theory building research, 417–418 personal variables, 418 situational variables, 418 Theory of meaning, for artifacts ecologies, 357 in genesis, 357 in language, 357 in use, 356 Theory of signs. see semiotics Thermal comfort, 445 Through the Interface, 502 TNO, 449 Tone masking, 79 Tonotopic representation, of sound, 79 Touch meaning of feelings and emotions, 44 interpersonal affection, communication for, 44 material world, understanding of, 43–44 physical experience, 43 self-awareness, 43

Toyota, 491 Types of offices, 611–612 in relation to ambient factors, 621 in relation to symbolic artifacts, 624–625 office experience, 612–614 open plan offices, 611, 619–620 relation to noise and privacy, 619–620 Typologies of value, 633–636 experiential benefits, 633 organizational dimensions, 633 intrinsic and extrinsic, 633 utilitarian benefits, 633

U Unconventional appearance, 586 Understanding office experiences, framework physical stimuli, 316–321 physical structure, 309–316 Davis model, 610 symbolic artifacts, 621–625 Unifying properties, of aesthetics in product design balance, 262 proportion, 262–263 symmetry in, 263–264 Unity in variety, principle of, 265 Universal principles, of aesthetics cross-sensory aesthetics in, 276–277 cultural differences over, 278–279 evolutionary aesthetic preferences in, 273–276 individual differences over, 277–278 study on cross-culture, 271–273 Usability, 499–513 beyond simplicity, 501–502 active user theory, 502 activity theory, 501 deskilling, 502 European line of critique, 502 work psychology, 501 Usecues, in product use concept of, 322–323 for perceptions/cognition, 309, 323 for users’ actions, 324 limitations of, 324–325 Useful interfaces, 499–513 designing, 502 User experience, 499–513

ACM/interactions design award, 502 quality framework, 503 user-experience goals, 503 adverse experiences, in technology, 507–509 in desktop environment, 507–508 in home environment, 508 as co-experience, 503 through technology, 503 desired experiences, in technology, 504–507 emotion in, 507 reflective design, 507 stigmatizing assistive products, 507 framework, 462–463 human-focused, 462–463 interaction-focused, 462–463 product-focused, 462 in HCI literature, 503 processing, 464 research direction in, 502–504 pragmatic and hedonic needs, 503 smart phones, 505 multiple experience support, 505 unexpected experiences, in technology, 509–511 ambiguity, 510–511 Design Noir, 510 diversion, 510 Phone Table, 510 Projected Realities Project, 511 social topics, 510 The (De) Tour Guide, 510–511 User experience design (UXD), 500, 505 User experience issues, 518–524 emotionally appealing, 522–524 ergonomic and hedonic aspects, 522 important factors of richness, 524 use of advertisements, 522 functional performance, expected and perceived, 518–519 understanding and sense of control, 519–522 collaboration, 519–520 in tangible products, 519 User interface sounds, 71 User-product interactions, 516, 519

662 V Virtual reality, 445 Visible novelty (VN), 150–151 Visual marketing elements, 589 Visual novelty (VN) effects on consumers, 587 differential attention, 587–588 relativity of, 586–588 Visual perception, of objects, 30–35 Visual phenomena, relativity of, 585–588 Visual stimuli, 588, 589 Visual system, and product designing color perception in, 173–174 contrast sensitivity, 173

INDEX

usable visual field, 174 visual acuity in, 172–173 Vomeronasal organ (VNO), 102 Vowel sounds, 71

W Website experience, 504–505 credibility in, 504 guidelines, 505 e-commerece, 505 trustworthiness in, 504–505 web design, 504 Websites designing, 461–462 Whiteness. see albedo Word processor, 501

Work environment, influences, 605 Hawthorne effects, 607 Hawthorne studies, 607 work motivation, 607 motivators/satisfiers, 607 Workload, 449, 452, 454 Workload bottlenecks, 202 Workplace condition, 451 Workplace ergonomics, 512 Workstation design, 449 risk factors, 449 Workstation views, 621

X Xerox, designers, 374–375