Complete Proceedings of the NordiCHI 2010 Conference

Proceedings of the Sixth Nordic Conference on Human-Computer Interaction NordiCHI 2010 Extending boundaries 16-20 Octo...

Author: NordiCHI2010

19 downloads 3012 Views 38MB Size Report

This content was uploaded by our users and we assume good faith they have the permission to share this book. If you own the copyright to this book and it is wrongfully on our website, we offer a simple DMCA procedure to remove your content from our site. Start by pressing the button below!

Report copyright / DMCA form

DOWNLOAD PDF

Proceedings of the Sixth Nordic Conference on Human-Computer Interaction

NordiCHI 2010 Extending boundaries

16-20 October, 2010, Reykjavik, Iceland

NordiCHI 2010 Extending boundaries Proceedings of the 6th Nordic Conference on Human-Computer Interaction, Reykjavik, Iceland, October 16–20, 2010 Editors Ann Blandford, Jan Gulliksen Ebba Thora Hvannberg, Marta Kristin Larusdottir Effie L-C Law, Hannes Hogni Vilhjalmsson

In cooperation with ACM SIGCHI www.nordichi.org

ACM Conference Proceedings Series ACM Press

The Association for Computing Machinery, Inc. 2 Penn Plaza, Suite 701 New York, NY 10121-0701

ACM COPYRIGHT NOTICE. Copyright © 2010 by the Association for Computing Machinery, Inc (ACM). Permission to make digital or hard copies of this work for personal or classroom use is granted without fee provided that the copies are not made or distributed for profit or commercial advantages and that copies bear this notice and thefull citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, to republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permission to republish from: Publications Dept. ACM, Inc. Fax +1 (212) 869-0481 or E-mail [email protected]. For other copying of articles that carry a code at the bottom of the first or last page, copying is permitted provided that the per-copy fee indicated in the code is paid through the Copyright Clearance Center, 222 Rosewood Drive, Danvers, MA 01923, +1-978-750-8400, +1-978-750-4470 (fax). Notice to Past Authors of ACM-Published Articles ACM intends to create a complete electronic archive of all articles and/or other material previously published by ACM. If you have written a work that was previously published by ACM in any journal or conference proceedings prior to 1978, or any SIG Newsletter at any time, and you do NOT want this work to appear in the ACM Digital Library, please inform permissions@ acm.org, stating the title of the work, the author(s), and where and when published.

ACM ISBN 978-1-60558-934-3

© 2010 by the Association for Computing Machinery, Inc.

Cover design and typesetting by Einar Þ. Samúelsson; contributions camera-ready by authors.

Proceedings: NordiCHI 2010, October 16–20, 2010

Preface This volume contains the proceedings of the 6th NordiCHI conference on Human-Computer Interaction. NordiCHI is the main forum for human-computer interaction research in the Nordic countries. The first five NordiCHI conferences were held in Stockholm (2000), Aarhus (2002), Tampere (2004), Oslo (2006) and Lund (2008). This year’s conference, held on October 16-20 in Reykjavik, is hosted by Reykjavik University and the University of Iceland. The theme of the conference is Extending Boundaries. Understanding, respecting, crossing and expanding boundaries is a part of our daily lives. Boundaries underpin not only geographical and cultural differences between countries, but also differences between sectors and scientific disciplines. Understanding boundaries is an integral part of human computer interaction. It is a part of understanding how people behave and what they value, of designing technologies, of learning how people express their needs, intentions and feelings, of designing HCI technologies and of evaluating them. Selecting Extending boundaries as the theme of NordiCHI 2010, organizers want to encourage contributors and participants to consider HCI’s boundaries, staying within them, or to think beyond them to address grand challenges. The call for participation created an excellent response and we are happy to offer a high-quality programme. The programme combines five different categories of submissions: full papers, short papers, design cases, industrial experience presentations and interactive demonstrations. The industrial experience presentation category follows the initiative from NordiCHI 2006 and the design case category follows the initiative from NordiCHI 2008. This year, 218 full papers were submitted, of which 60 were accepted for presentation at the conference. A further 188 short papers, 5 design cases and 3 interactive demos were submitted. The acceptance rate for full papers was 28% and 31% for short papers. The submitted papers represent 31 countries, with the accepted ones representing 22. All papers were reviewed by at least three independent reviewers; most had four reviews. All papers were discussed at the Programme Committee meeting; those for which reviewers were unanimous in their recommendations were accepted or rejected in accordance with those recommendations. Where there were divergent views amongst reviewers, the substantive details of the reviews were discussed in order to reach a decision; in a few cases, a further review of a paper was commissioned to ensure that decisions were based on the best information possible. We are grateful to all reviewers, the vast majority of whom delivered high quality reviews in a timely way. We are also grateful to the PC members who gave their time to attend the meeting and contributed constructively to the decision making process. Furthermore we would like to thank the organizing committee for their great work. In particular, we want to thank Margret Dora Ragnarsdottir for all her organizing and public relation work. Last but not least, we thank our cooperating and sponsoring organizations for their support: NordiCHI past chairs, ACM, and our main sponsor Nokia. You are very welcome to NordiCHI 2010 Ann Blandford, Jan Gulliksen, Ebba Thora Hvannberg, Marta Kristin Larusdottir, Effie L-C Law and Hannes Hogni Vilhjalmsson

Proceedings: NordiCHI 2010, October 16–20, 2010

Sponsors

Proceedings: NordiCHI 2010, October 16–20, 2010

Organizing Committee

General chairs: Program chairs:

Ebba Þóra Hvannberg, University of Iceland, Iceland Marta Kristín Lárusdóttir, Reykjavik University, Iceland Ann Blandford, University College London Jan Gulliksen, Royal Institute of Technology, Sweden

Short paper & poster chairs:

Effie Law, University of Leicester, UK/ ETH Zürich, Switzerland Hannes Högni Vilhjálmsson, Reykjavik University, Iceland

Doctoral consortium chairs:

Ann Lantz, Royal Institute of Technology, Sweden Dag Svanæs, Norwegian University of Science and Technology, Norway Kaisa Väänänen-Vainio-Mattila, Tampere University of Technology and Nokia Research Center, Finland

Workshops chairs: Tutorials chair: Public relations chair:

Klaus Marius Hansen, University of Iceland, Iceland Kasper Hornbæk, Copenhagen University, Denmark Janet Read, University of Central Lancashire, UK Margrét Dóra Ragnarsdóttir, Síminn, Iceland

Interactive experience and design cases chairs:

Arnold P.O.S. Vermeeren, Delft University of Technology, The Netherlands Gunnar Grímsson, Web design consultant, Iceland Sigríður Sigurjónsdóttir, Iceland Academy of the Arts, Iceland

Industrial and Government Experience chairs:

Margrét Dóra Ragnarsdóttir, Síminn, Iceland Áslaug María Friðriksdóttir, SJÁ ehf., Iceland Birna Íris Jónsdóttir, Betware Gautam Ghosh, Norwegian Tax Authority, Norway Sigrún Eva Ármannsdóttir, Skýrr, Iceland

Sponsor chairs

Marta Kristín Lárusdóttir, Reykjavik University, Iceland Ebba Þóra Hvannberg, University of Iceland, Iceland

Student volunteer chair

Martha Dís Brandt, Reykjavik University, Iceland

Local arrangements

Inga Sólnes, Gestamóttakan, Your Host in Iceland

Proceedings: NordiCHI 2010, October 16–20, 2010

Programme Committee for Full and Short Papers Silvia Abrahao, Universidad Politecnica de Valencia, Spain

Steinar Kristoffersen, Østfold Regional College, Norway

Liam Bannon, University of Limerick, Ireland

Kari, Kuutti, University of Oulu, Finland

Olav Bertelsen, Aarhus University, Denmark

Ann Lantz, KTH Royal Institute of Technology, Sweden

Staffan Björk, Chalmers University of Technology, Sweden

Effie L-C Law, ETH Zurich / University of Leicester, United Kingdom

Ann Blandford, University College London, United Kingdom

Marta Kristín Lárusdóttir, Reykjavik University, Iceland

Tone Bratteteig, University of Oslo, Norway

Scott MacKenzie, York University, Canada

Susanne Bødker, Aarhus University, Denmark

Lone Malmborg, IT University of Copenhagen, Denmark

Torkil Clemmensen, Copenhagen Business School, Denmark

Andrew Morrison, University of Oslo, Norway

Gilbert Cockton, School of Design, Northumbria University, United Kingdom

Erik G. Nilsson, SINTEF, Norway

Anders Mørch, University of Oslo, Norway

Håkan Eftring, Lund University, Sweden

Margrét Dóra Ragnarsdóttir, Síminn / University of Iceland, Iceland

Morten Fjeld, Chalmers University of Technology, Sweden

Roope Raisamo, University of Tampere, Finland

Gautam Ghosh, Norwegian Tax Authority, Norway

Kari-Jouko Räihä, University of Tampere, Finland

Jan Gulliksen,KTH Royal Institute of Technology, Sweden

Janet Read, University of Central Lancashire, United Kingdom

Agneta Gulz, Lund University / Linköping University, Sweden

Virpi Roto, Nokia Research Center, Finland

Klaus Marius Hansen, University of Iceland, Iceland

Frode Eika Sandnes, Oslo University College, Norway

Michael Herczeg, University of Luebeck, IMIS, Germany

Jan Stage, Aalborg University, Denmark

Jo Herstad, University of Oslo, Norway

Erik Stolterman, Indiana University, USA

Morten Hertzum, Roskilde University, Denmark Kasper Hornbaek, University of Copenhagen, Denmark

Dag Svanæs, Norwegian University of Science and Technology, Norway

Ebba Thora Hvannberg, University of Iceland, Iceland

Loren Terveen, University of Minnesota, USA

Kristina Höök, Mobile Life @ Stockholm University, Sweden

Konrad Tollmar, KTH Royal Institute of Technology, Sweden

Giulio Jacucci, Aalto University, Finland

Kaisa Väänänen-Vainio-Mattila, Tampere University of Technology / Nokia Research Center, Finland

Lars-Erik Janlert, Umeå University, Sweden Victor Kaptelinin, Umeå University, Sweden Jofish Kaye, Nokia, USA

Arnold P.O.S. Vermeeren, Delft University of Technology, The Netherlands Hannes Högni Vilhjálmsson, Reykjavik University, Iceland

Proceedings: NordiCHI 2010, October 16–20, 2010

Additional Reviewers of Full and Short Papers Sisira Adikari, University of Canberra, Australia Abdullah Al Mahmud, Eindhoven University of Technology (TU/e), The Netherlands

Henry BL Duh, National University of Singapore, Singapore Mark Dunlop, University of Strathclyde, United Kingdom

Daniela Andrei, Romania

Helen Edwards, University of Sunderland, United Kingdom

Jenni Anttonen, University of Tampere, Finland

Magy Seif El-Nasr, Simon Fraser University, Canada

Carmelo Ardito, University of Bari, Italy

Asbjorn Folstad, SINTEF, Norway

Henrik Artman, KTH Royal Institute of Technology, Sweden

Tim French, University of Bedfordshire, United Kingdom

Pelin Atasoy, The Netherlands

David Geerts, K.U.Leuven, Belgium

Simon Attfield, Middlesex University, United Kingdom

Jens Gerken, University of Konstanz, Germany

Nikos Avouris, University of Patras, Greece

Marco Gillies, University of London, United Kingdom

Cedric Bach, University of Toulouse, France

Peter Gorny, Uni Oldenburg, Germany

Olle Bälter, KTH Royal Institute of Technology, Sweden

Solza Grceva, FON University, Republic of Macedonia

Regina Bernhaupt, University Salzburg, Austria

Anders Green, KTH Royal Institute of Technology, Sweden

Nigel Bevan, Professional Usability Services, United Kingdom

Erik Grönvall, Aarhus University Denmark

Franca Garzotto, Politecnico di Milano, Italy

Timothy Bickmore, Northeastern University, USA

Nils-Erik Gustafsson, KTH Royal Institute of Technology, Sweden

Mark Bilandzic, Queensland University of Technology, Australia

Simen Hagen, Oslo University College, Norway

Amy Bruckman, Georgia Institute of Technology, USA Anders Bruun, Aalborg University, Denmark Hrönn Brynjarsdóttir, Cornell University, USA Asa Cajander, Uppsala University, Sweden Georgios Christou, European University Cyprus, Cyprus Dan Cosley, Cornell University, USA Teresa C-Pargman, Stockholm University, Sweden Stefan Cronholm, Linköping University, Sweden Alma Leora Culén, University of Oslo, Norway Sergio Roberto da Silva, Universidade Estadual de Maringá, Brazil Alexander De Luca, University of Munich, Germany

Göran Hagert, Malmö University, Sweden Tabin Hasan, University of Trento, Italy Per-Olof Hedvall, Lund University, Sweden Ingi Helgason, Napier University, United Kingdom Bo Helgeson, Blekinge Institute of Technology, Sweden Niels Henze, University of Oldenburg, Germany Tobias Hesselmann, OFFIS, Germany Wilko Heuten, OFFIS, Germany Clint Heyer, ABB AB, Corporate Research, Norway Dirk Heylen, University of Twente, The Netherlands Rune Thaarup Hoeegh, Aalborg University, Danmark Harald Holone, Högskolen i Östfold, Norway Jettie Hoonhout, Philips, The Netherlands

Proceedings: NordiCHI 2010, October 16–20, 2010

Netta Iivari, Oulu University, Finland

Stacy Marsella, University of Southern California, USA

Emilio Insfran, Universidad Politecnica de Valencia, Spain

Stefan Marti, Samsung, USA

Poika Isokoski, University of Tampere, Finland

Maristella Matera, Politecnico di Milano, Italy

Mattias Jacobsson, Swedish Institute of Computer Science, Sweden

Emanuela Mazzone, University of Central Lancashire, United Kingdom

Mikkel R. Jakobsen, Copenhagen University, Denmark

Tom McEwan, Edinburgh Napier University, United Kingdom

Anders Jansson, Uppsala University, Sweden Janne Jul Jensen, Aalborg University, Denmark Anne Kaikkonen, Nokia, Finland Karrie Karahalios, University of Illinois, USA

Martin Mihajlov, Faculty of Economics - Skopje, Republic of Macedonia Tonja Molin-Juustila, Oulu University, Finland

Amela Karahasanovic, SINTEF, Norway

Francisco Montero, University of Castilla-La Mancha, Spain

Kristiina Karvonen, Aalto University, Finland

Camille Moussette, Umeå Institute of Design, Sweden

Jinsook Kim, Jacksonville State University, USA

Christina Mörtberg, University of Oslo, Norway

Clemens Nylandsted Klokmose, Aarhus University, Denmark

Stina Nylander, Swedish Institute of Computer Science, Sweden

Anders Kluge, University of Oslo, Norway

Marianna Obrist, University of Salzburg, Austria

Tomoko Koda, Osaka Institute of Technology, Japan

Lars Oestreicher, Uppsala University, Sweden

Andreas Komninos, Glasgow Caledonian University, United Kingdom

Alex Olwal, KTH, Sweden

Liisa Kuparinen, University of Jyväskylä, Finland

Benoît Otjacques, Centre de Recherche Public - Gabriel Lippmann, Luxemburg

Jarmo Laaksolahti, Swedish Institute of Computer Science, Sweden

Saila Ovaska, University of Tampere, Finland

Rosa Lanzilotti, Universita di Bari, Italy

Jacob Palme, Stockholm University / KTH Royal Institute of Technology, Sweden

Astrid Twenebowa Larssen, Ashesi University College, Ghana

Ignacio Panach, Technical University of Valencia, Spain

Pierre Leclercq, University of Liege, Belgium Henry Lieberman, MIT Media Laboratory, USA

Fabio Paterno, ISTI, Italy Thomas Pederson, IT University of Copenhagen, Denmark

Sara Ljungblad, Stockholm University, Sweden

Samuli Pekkola, Tampere University of Technology, Finland

Andres Lucero, Nokia, Finland

Mark Perry, Brunel University, United Kingdom

Martin Ludvigsen, Aarhus School of Architecture, Denmark

John Sören Pettersson, Karlstad University, Sweden

Sigrun Lurås, DNV, Norway

Greg Phillips, Royal Military College of Canada, Canada

Charlotte Magnusson, Lund University, Sweden

Martin Pielot, OFFIS, Germany

Päivi Majaranta, University of Tampere, Finland

Proceedings: NordiCHI 2010, October 16–20, 2010

Volkmar Pipek, IISI - International Institute for SocioInformatics, Germany

Anna-Liisa Syrjänen, Oulu University, Finland

Costin Pribeanu, ICI Bucharest, Romania

Jakob Tholander, Mobile Life, Stockholm University, Sweden

Taina Rajanti, University of Art and Design, Finland

Manfred Tscheligi, University of Salzburg, Austria

Tim Regan, Microsoft Research, United Kingdom

Brygg Ullmer, Louisiana State University, USA

Hayley Davison Reynolds, MIT, USA

Anna Vallgårda, Swedish School of Textiles, Sweden

Rosamelia Ribeiro, Universidade Federal de Goiás, Brazil

Jan Van den Bergh, University of Hasselt, Belgium

Mikko Rissanen, ABB AB, Corporate Research, Sweden

Konstantina Vassilopoulou, Harokopio University, Greece

Armanda Rodrigues, Universidade Nova de Lisboa, Portugal

Olli Vilkki, Laurea University of Applied Sciences, Finland

Mattias Rost, Swedish Institute of Computer Science, Sweden

Chris Vincent, University College London, United Kingdom Mattias Wallergård, Lund University, Sweden

Fredrik Rutz, Stockholm University, Sweden

Alexandra Weilenmann, Göteborg University, Sweden

Antti Salovaara, Aalto University, Finland

Håvard Wiik, Bouvet, Norway

Carmen Santoro, ISTI, Italy

Marco Winckler, University Paul Sabatier, France

Corina Sas, Lancaster University, United Kingdom

Yan Xu, Georgia Institute of Technology, USA

Martin Schmettow, University of Twente, The Netherlands

Jim Youll, RocketSync, USA

Tomas Sokoler, IT University of Copenhagen, Denmark

John Zimmerman, Carnegie Mellon University, USA

William, Soukoreff, University of Toronto, Canada

Tihomir Zoranovic, University of Novi Sad, Serbia

Mark Springett, Middlesex University, United Kingdom

Valdís Þorsteinsdóttir, Vinnuvernd ehf, Iceland

Chris Stary, Johannes Kepler University Linz, Austria

Rikke Ørngreen, Aarhus University, Denmark

Petra Sundström, Stockholm University, Sweden

Proceedings: NordiCHI 2010, October 16–20, 2010

Reviewers of Design Cases Mathilde Bekker, , Eindhoven University of Technology, The Netherlands

Sigríður Sigurjónsdóttir, Iceland Academy of the Arts, Iceland

Gunnar Grímsson, Skuggaþing, Iceland

Arnold P.O.S. Vermeeren, Delft University of Technology, The Netherlands

Marc Hassenzahl, Folkwang University of the Arts, Germany

Reviewers of Industrial Experinece Reports Sigrún Eva Ármannsdóttir, Skýrr, Iceland

Birna Íris Jónsdóttir, Betware, Iceland

Áslaug María Friðriksdóttir, SJÁ ehf., Iceland

Margrét Dóra Ragnarsdóttir, Síminn, Iceland

Gautam Ghosh, Norwegian Tax Authority, Norway

Table of Contents

Proceedings: NordiCHI 2010, October 16–20, 2010

Keynote Speakers Tone Bratteteig Changing Boundaries of Design and Use...................................................................................................................................... 1

Ari Kristinn Jónsson Developing Interactive Artificial Intelligence Software for NASA Mission Control.................................................................... 1

David Merrill The New Era of Physical/Digital Play........................................................................................................................................... 2

Full Papers Let’s All Get Up and Walk to the North Pole: Design and Evaluation of a Mobile Wellness Application Aino Ahtinen, Pertti Huuskonen and Jonna Häkkilä..................................................................................................................... 3

Location-based Crowdsourcing: Extending Crowdsourcing to the Real World Florian Alt, Alireza Sahami Shirazi, Albrecht Schmidt, Urs Kramer and Zahid Nawaz............................................................. 13

Analysis of Precedent Designs: Competitive Analysis Meets Genre Analysis Mattias Arvola, Jonas Lundberg and Stefan Holmlid.................................................................................................................. 23

Exploring Distance Encodings with a Tactile Display to Convey Turn by Turn Information in Automobiles Amna Asif, Wilko Heuten and Susanne Boll............................................................................................................................... 32

Facilitating Continuity: Exploring the Role of Digital Technology in Physical Rehabilitation Naveen Bagalkot, Elena Nazzi and Tomas Sokoler..................................................................................................................... 42

4Photos: A Collaborative Photo Sharing Experience Martijn ten Bhömer, John Helmes, Kenton O’Hara and Elise van den Hoven........................................................................... 52

Two-Handed Input in a Standard Configuration of Notebook with External Mouse Florian Block and Hans Gellersen............................................................................................................................................... 62

Tenori-on Stage: YouTube as Performance Space Mark Blythe and Paul Cairns....................................................................................................................................................... 72

Training Software Developers in Usability Engineering: A Literature Review Anders Bruun............................................................................................................................................................................... 82

Design and Evaluation of Player Experience of a Location-Based Mobile Game Tara Carrigy, Katsiaryna Naliuka, Natasa Paterson and Mads Haahr......................................................................................... 92

Understanding the Everyday Use of Images on the Web Boon Chew, Jennifer Rode and Abigail Sellen.......................................................................................................................... 102

HCI & Sustainable Food Culture: A Design Framework for Engagement Jaz Hee-jeong Choi and Eli Blevis............................................................................................................................................ 112

Distributed Usability Evaluation: Enabling Large-scale Usability Evaluation with User-controlled Instrumentation Lars Christensen and Erik Frøkjær............................................................................................................................................ 118

A Study of Mobile Mood Awareness and Communication through MobiMood Karen Church, Eve Hoggan and Nuria Oliver........................................................................................................................... 128

Engaging Spect-actors with Multimodal Digital Puppetry Céline Coutrix, Giulio Jacucci, Anna Spagnolli, Lingyi Ma, Matti Helin, Gabriela Richard, Lorenza Parisi, Stefano Roveda and Prayag Narula........................................................................................................................................... 138

Proceedings: NordiCHI 2010, October 16–20, 2010

Table of Contents

Comparing User Interaction with Low and High Fidelity Prototypes of Tabletop Surfaces Jan Derboven, Dries De Roeck, Mathijs Verstraete, David Geerts, Jan Schneider Barnes and Kris Luyten............................ 148

Drop-and-Drag: Easier Drag&Drop on Large Touchscreen Displays Sebastian Döweling and Urs Glaubitt........................................................................................................................................ 158

Extending Boundaries with Meta-Design and Cultures of Participation Gerhard Fischer.......................................................................................................................................................................... 168

Wattsup?: Motivating Reductions in Domestic Energy Consumption Using Social Networks Derek Foster, Shaun Lawson, Mark Blythe and Paul Cairns..................................................................................................... 178

Empirical Investigation of Web Design Attributes Affecting Brand Perception Franca Garzotto, Davide Bolchini, Tao Yang and Fabio Sorce................................................................................................. 188

“Luckily, I don’t need it”: Elderly and the Use of Artifacts for Time Management Leonardo Giusti, Eleonora Mencarini and Massimo Zancanaro............................................................................................... 198

Studying Mobile Context-aware Social Services in the Wild Paul Holleis, Sebastian Boehm, Johan Koolwaaij and Matthias Wagner.................................................................................. 207

Negotiating Privacy Boundaries in Social Applications for Accessibility Mapping Harald Holone and Jo Herstad................................................................................................................................................... 217

Transferring Qualities from Horseback Riding to Design Kristina Höök............................................................................................................................................................................. 226

Controlling the Use of Collaboration Tools in Open Source Software Development Heli Ikonen, Netta Iivari and Henrik Hedberg........................................................................................................................... 236

Piles, Tabs and Overlaps in Navigation among Documents Mikkel Rønne Jakobsen and Kasper Hornbæk.......................................................................................................................... 246

Determining Usability Requirements into a Call-for-tenders. A Case Study on the Development of a Healthcare System Timo Jokela................................................................................................................................................................................ 256

Descriptive Quality of Experience for Mobile 3D Video Satu Jumisko-Pyykkö, Dominik Strohmeier, Timo Utriainen and Kristina Kunze................................................................... 266

Unobtrusively Controlling and Linking Information and Services in Smart Environments Bastian Kriesten, René Tünnermann, Christian Mertes and Thomas Hermann........................................................................ 276

Evaluating Multimodal Systems - A Comparison of Established Questionnaires and Interaction Parameters Christine Kühnel, Tilo Westermann, Benjamin Weiss and Sebastian Möller............................................................................ 286

Rise of the Expert Amateur: DIY Projects, Communities, and Cultures Stacey Kuznetsov and Eric Paulos............................................................................................................................................. 295

Do People Say What They Think: Social Conformity Behavior in Varying Degrees of Online Social Presence Lieve Laporte, Christof van Nimwegen and Alex J. Uyttendaele............................................................................................. 305

Using the Hybrid Simulation for Early User Evaluations of Pervasive Interactions Karin Leichtenstern, Elisabeth André and Matthias Rehm........................................................................................................ 315

XTag: Designing an Experience Capturing and Sharing Tool for Persons with Aphasia Abdullah Al Mahmud, Rikkert Gerits and Jean-Bernard Martens............................................................................................ 325

Pipet: A Design Concept Supporting Photo Sharing Bernt Meerbeek, Peter Bingley, Wil Rijnen and Elise van den Hoven...................................................................................... 335

Table of Contents

Proceedings: NordiCHI 2010, October 16–20, 2010

User Experience (UX) Patterns for Audio-Visual Networked Applications: Inspirations for Design Marianna Obrist, Daniela Wurhofer, Elke Beck, Amela Karahasanovic and Manfred Tscheligi.............................................. 343

Collective Interaction by Design - Collective Controllers for Social Navigation on Digital Photos Marianne Graves Petersen, Peter Gall Krogh, Morten Boye Mortensen, Thomas Møller Lassen and Ditte Hvas Mortensen..... 353

Privacy-Awareness Information for Web Forums: Results from an Empirical Study Stefanie Pötzsch, Peter Wolkerstorfer and Cornelia Graf.......................................................................................................... 363

Tactile Camera vs. Tangible Camera: Taking Advantage of Small Physical Artefacts to Navigate into Large Data Collection Mathieu Raynal, Guillaume Gauffre, Cédric Bach, Bénédicte Schmitt and Emmanuel Dubois............................................... 373

Five Design Challenges for Human Computation Stuart Reeves and Scott Sherwood............................................................................................................................................ 383

Designing for Crowds Stuart Reeves, Scott Sherwood and Barry Brown..................................................................................................................... 393

Mobile Interaction with Real-Time Geospatial Data by Pointing Through Transparent Earth Erika Reponen and Jaakko Keränen.......................................................................................................................................... 403

Lightweight Personal Sensemaking Tools for the Web Brendan Ryder and Terry Anderson........................................................................................................................................... 413

The Effect of Aesthetically Pleasing Composition on Visual Search Performance Carolyn Salimun, Helen C Purchase, David R Simmons and Stephen Brewster...................................................................... 422

HandsDown: Hand-contour-based User Identification for Interactive Surfaces Dominik Schmidt, Ming Ki Chong and Hans Gellersen........................................................................................................... 432

ExoBuilding - Breathing Life into Architecture Holger Schnädelbach, Kevin Glover and Ainojie Alexander Irune........................................................................................... 442

Sustainable Energy Practices at Work: Understanding the Role of Workers in Energy Conservation Tobias Schwartz, Matthias Betz, Leonardo Ramirez and Gunnar Stevens................................................................................ 452

Body-Centric Interaction Techniques for Very Large Wall Displays Garth Shoemaker, Takayuki Tsukitani, Yoshifumi Kitamura and Kellogg S. Booth................................................................. 463

From Product Concept to User Experience: Exploring UX Potentials at Early Product Stages Sandra Sproll, Matthias Peissner and Christina Sturm.............................................................................................................. 473

Electronic Resource Discovery Systems: From User Behaviour to Design Hanna Stelmaszewska, B.L. William Wong, Simon Attfield and Raymond Chen.................................................................... 483

Design Qualities for Whole Body Interaction - Learning from Golf, Skateboarding and BodyBugging Jakob Tholander and Carolina Johansson.................................................................................................................................. 493

A Case Study: The Impact of Using Location-Based Services with a Behaviour-Disordered Child Lisa Thomas, Pam Briggs and Linda Little............................................................................................................................... 503

Eyes-free Text Entry with Error Correction on Touchscreen Mobile Devices Hussain Tinwala and Ian Scott MacKenzie............................................................................................................................... 511

User Experience Evaluation Methods: Current State and Development Needs Arnold P.O.S. Vermeeren, Effie Lai-Chong Law, Virpi Roto, Marianna Obrist, Jettie Hoonhout and Kaisa Väänänen-Vainio-Mattila.......................................................................................................................................... 521

Family Storytelling for Grandparents and Grandchildren Living Apart René Vutborg, Jesper Kjeldskov, Sonja Pedell and Frank Vetere.............................................................................................. 531

Proceedings: NordiCHI 2010, October 16–20, 2010

Table of Contents

Supporting Cooperative Design through “Living” Artefacts Dhaval Vyas, Anton Nijholt and Gerrit van der Veer................................................................................................................. 541

Crowdsourcing Human-Based Computation Doug Wightman......................................................................................................................................................................... 551

Curve: Revisiting the Digital Desk Raphael Wimmer, Fabian Hennecke, Florian Schulz, Sebastian Boring, Andreas Butz and Heinrich Hußmann..................... 561

How to Stay in the Emotional Rollercoaster: Lessons Learnt from Designing EmRoll Farnaz Zangouei, Mohammad Ali Babazadeh Gashti, Kristina Höök, Tim Tijs, Gert-Jan de Vries and Joyce Westerink....... 571

Keep Talking: An Analysis of Participant Utterances Gathered Using Two Concurrent Think-aloud Methods Tingting Zhao and Sharon McDonald....................................................................................................................................... 581

Short Papers tacTiles - A Low-Cost Modular Tactile Sensing System for Floor Interactions Jan Anlauff, Tobias Grosshauser and Thomas Hermann........................................................................................................... 591

Interaction Design Qualities: Theory and Practice Mattias Arvola............................................................................................................................................................................ 595

Virtual Fashion and Avatar Design: A Survey of Consumers and Designers Jeffrey Bardzell, Tyler Pace and Jennifer Terrell....................................................................................................................... 599

Space-Multiplexed Input on Mouse-Extended Notebooks Florian Block and Hans Gellersen............................................................................................................................................. 603

The Impact of Cognitive Load on the Perception of Time Florian Block and Hans Gellersen............................................................................................................................................. 607

Experiencing the Non-Sensuous: On Measurement, Representation and Conception in Urban Art Installations Morten Breinbjerg, Morten S. Riis, Tobias Ebsen and Rasmus B. Lunding............................................................................. 611

DisQo: A User Needs Analysis Method for Smart Home Joëlle Coutaz, Emeric Fontaine, Nadine Mandran and Alexandre Demeure............................................................................ 615

User Perception of Interruptions in Multimedia Annotation Tasks Chris Creed, Chris Bowers, Robert Hendley and Russell Beale............................................................................................... 619

Seeking a Theoretical Foundation for Design of In Sitro Usability Assessments Yngve Dahl................................................................................................................................................................................ 623

Why do Users Communicate via such or such Media? Some Insights from Users’ Daily Experiences Françoise Détienne, Béatrice Cahour and Liv Lefebvre............................................................................................................ 627

The Impact of Concept (Re)presentation on Users’ Evaluation and Perception Sarah Diefenbach, Marc Hassenzahl, Kai Eckoldt and Matthias Laschke................................................................................ 631

Exploration of Direct Bi-manual Interaction in Digitally Mediated Stop-motion Animation Mariza Dima, John Lee and Mark Wright................................................................................................................................. 635

Choosing Field Methods: A Reflection on a RepGrid Study Helen M. Edwards, Sharon McDonald and S. Michelle Young................................................................................................ 639

Mementos: A Tangible Interface Supporting Travel Augusto Esteves and Ian Oakley............................................................................................................................................... 643

Table of Contents

Proceedings: NordiCHI 2010, October 16–20, 2010

Analysis in Usability Evaluations: an Exploratory Study Asbjørn Følstad, Effie Lai-Chong Law and Kasper Hornbæk................................................................................................... 647

Pocket Bee - A Multi-modal Diary for Field Research Jens Gerken, Stefan Dierdorf, Patric Schmid, Alexandra Sautner and Harald Reiterer............................................................ 651

SmarterPhone – Supporting Meetings in a Mobile Context Humberto Glória, Tiago Guerreiro and Daniel Gonçalves........................................................................................................ 655

Evaluating Game Genres for Tagging Images Dion H. Goh, Rebecca P. Ang, Alton Y.K. Chua and Chei Sian Lee......................................................................................... 659

Designing a Personal Information Visualization Tool Paulo Gomes, Sandra Gama and Daniel Gonçalves.................................................................................................................. 663

The City at Hand: Media Installations as Urban Information Systems Roland Haring, Hideaki Ogawa, Christopher Lindinger, Horst Hörtner, Shervin Afshar and David Stolarsky....................... 667

Take me by the Hand: Haptic Compasses in Mobile Devices through Shape Change and Weight Shift Fabian Hemmert, Susann Hamann, Matthias Löwe, Anne Wohlauf, Josefine Zeipelt and Gesche Joost................................. 671

Experiments in the Wild: Public Evaluation of Off-Screen Visualizations in the Android Market Niels Henze, Benjamin Poppinga and Susanne Boll................................................................................................................. 675

TilePix: An Exploratory Interface to Astronomical Image Databases Avon Huxor and Steve Phillipps................................................................................................................................................ 679

Motor Efficiency of Text Entry in a Combination of a Soft Keyboard and Unistrokes Poika Isokoski, Benoît Martin, Paul Gandouly and Thomas Stephanov................................................................................... 683

Prototyping iPhone Apps: Realistic Experiences on the Device Anders P. Jørgensen, Matthijs Collard and Christian Koch....................................................................................................... 687

Onto-Frogger: The Making Of Amalia Kallergi and Fons J. Verbeek......................................................................................................................................... 691

Politics at the Interface - A Foucauldian Power Analysis Gopinaath Kannabiran and Marianne Petersen.......................................................................................................................... 695

Thumbs-Up Scale and Frequency of Use Scale for Use in Self Reporting of Children’s Computer Experience Akiyo Kano, Matthew Horton and Janet C Read...................................................................................................................... 699

EcoIsland: A Persuasive Application to Motivate Sustainable Behavior in Collectivist Cultures Hiroaki Kimura and Tatsuo Nakajima....................................................................................................................................... 703

Comprehending Parametric CAD Models: An Evaluation of Two Graphical User Interfaces Siniša Kolarić, Halil Erhan, Robert Woodbury and Bernhard E. Riecke.................................................................................. 707

Are Human-Computer Interaction Design Patterns Really Used? Christian Kruschitz and Martin Hitz.......................................................................................................................................... 711

Where are the Ionians of User Experience Research? Kari Kuutti................................................................................................................................................................................. 715

To what Extent Usability Truly Matters? A Study on Usability Requirements in Call-for-tenders of Software Systems Issued by Public Authorities Taina Lehtonen, Juha Kumpulainen, Timo Jokela and Tapani Liukkonen................................................................................ 719

Blue-Sky and Down-to-Earth: How Analogous Practices can Support the User-centred Design Process Sara Ljungblad and Clint Heyer................................................................................................................................................ 723

Proceedings: NordiCHI 2010, October 16–20, 2010

Table of Contents

GappaGoshti™ – A Social Networking Platform for Information Dissemination in the Rural World Sylvan Lobo, Pankaj Doke and Sanjay Kimbahune.................................................................................................................. 727

PINwI - Pedestrian Indoor Navigation without Infrastructure Markus Löchtefeld, Sven Gehring, Johannes Schöning and Antonio Krüger........................................................................... 731

Pointing for Non-visual Orientation and Navigation Charlotte Magnusson, Miguel Molina, Kirsten Rassmus-Gröhn and Delphine Szymczak....................................................... 735

Mobile or Desktop Websites? Website Usage on Multitouch Devices Max-Emanuel Maurer, Doris Hausen, Alexander De Luca and Heinrich Hussmann............................................................... 739

Venice Unfolding: A Tangible User Interface for Exploring Faceted Data in a Geographical Context Till Nagel, Frank Heidmann, Massimiliano Condotta and Erik Duval..................................................................................... 743

DESIGNi – A Workbench for Supporting Interaction Design Claudia Nass, Kerstin Klöckner, Sarah Diefenbach and Marc Hassenzahl............................................................................... 747

What makes a Butlerbot a Butlerbot? A Discussion of a Robot Classification System Mie Nørgaard............................................................................................................................................................................. 751

Towards a Model for Egocentric Interaction with Physical and Virtual Objects Thomas Pederson, Lars-Erik Janlert and Dipak Surie............................................................................................................... 755

Visualizing the Text of Philip Pullman’s Trilogy “His Dark Materials” Tim Regan and Linda Becker.................................................................................................................................................... 759

Teaching UbiComp with Sense Mike Richards and Neil Smith................................................................................................................................................... 765

Introducing Co-design for Digital Technologies in Rural Areas Fausto Sainz, Nacho Madrid and Jaisiel Madrid....................................................................................................................... 769

PyGmI – Creation and Evaluation of a Portable Gestural Interface Matthias Schwaller, Denis Lalanne and Omar Abou Khaled.................................................................................................... 773

Capital Music – Personal Expression with a Public Display of Song Choice Jan Seeburger, Marcus Foth and Dian Tjondronegoro.............................................................................................................. 777

Research Methods for Involving Hearing Impaired Children in IT Innovation Karin Slegers, Pieter Duysburgh and An Jacobs....................................................................................................................... 781

Evaluation of Adaptive SpringLens -- a Multi-focus Interface for Exploring Multimedia Collections Sebastian Stober, Christian Hentschel and Andreas Nuernberger............................................................................................. 785

Measuring Effects of Private and Shared Displays in Small-Group Knowledge Sharing Processes Sara Streng, Karsten Stegmann, Sebastian Boring, Sonja Böhm, Frank Fischer and Heinrich Hussmann.............................. 789

Mobile Device Interaction Gestures for Older Users Christian Stößel and Lucienne Blessing.................................................................................................................................... 793

Messenger Visual: A Pictogram-based IM Service to Improve Communications Among Disabled People Pere Tuset, Pere Barberán, Léonard Janer, Esteve Buscà, Sandra Delgado and Nuria Vilà...................................................... 797

From Magical Experience to Effortlessness: An Exploration of the Components of Intuitive Interaction Daniel Ullrich and Sarah Diefenbach........................................................................................................................................ 801

Role Playing with Fire Fighters – Using a Worst Case Scenario and Verbal Re-enactment in the Role Play Paula Valkonen and Marja Liinasuo.......................................................................................................................................... 805

Gesture Based Interaction for Visually-Impaired People Sylvie Vidal and Grégoire Lefebvre.......................................................................................................................................... 809

Table of Contents

Proceedings: NordiCHI 2010, October 16–20, 2010

Bridging Gaps with Pointer Warping in Multi-Display Environments Manuela Waldner, Ernst Kruijff and Dieter Schmalstieg.......................................................................................................... 813

Detecting the “Point of Originality” in Student Writing Brandon White and Johann Ari Larusson.................................................................................................................................. 817

Architectures of Interaction: An Architectural Perspective on Digital Experience Heather Wiltse and Erik Stolterman.......................................................................................................................................... 821

Measuring the Dynamics of User Experience in Short Interaction Sequences Benjamin Wimmer, Bernhard Wöckl, Michael Leitner and Manfred Tscheligi........................................................................ 825

Design and Interactive Demonstrations UCD Method Selection with Usability Planner Xavier Ferre, Nigel Bevan and Tomás Antón Escobar.............................................................................................................. 829

HawkEye: a novel process automation interface Kristoffer Husøy and Torgeir Enkerud...................................................................................................................................... 831

Temporal Relations in Affective Health Elsa Kosmack-Vaara, Iuliana Claudia Silvăşan, Anna Ståhl and Kristina Höök....................................................................... 833

linked. – A Relatedness Experience for Boys Matthias Laschke, Marc Hassenzahl and Kurt Mehnert............................................................................................................ 839

Exploratory Study of a Touch-based Gestural Interface for Elderly Chiara Leonardi, Adriano Albertini, Fabio Pianesi and Massimo Zancanaro........................................................................... 845

Bringing Playfulness to Disabilities Patrizia Marti............................................................................................................................................................................. 851

Untangling the Mess - A Redesign of a Technical Environment for a Telecommunication Company Call Centre. Part II: Capturing a Valid Order for a Telecommunication Service Margret Dora Ragnarsdottir....................................................................................................................................................... 857

Panel Grand Challenges for Future HCI Research: Cultures of Participation, Interfaces Supporting Learning, and Expansive Learning Yrjö Engeström, Annalisa Sannino, Gerhard Fischer and Anders I. Mörch.............................................................................. 863

Author index ................................................................................................................................................................................................... 867

Keynote Speakers

Proceedings: NordiCHI 2010, October 16–20, 2010

Tone Bratteteig

Associate Professor, Department of Informatics, University of Oslo

Changing Boundaries of Design and Use Biography Tone Bratteteig holds a Ph.D. from The University of Oslo, addressing design and use of IT as an interdisciplinary research field. She is associate professor at Department of Informatics, U of Oslo, where she leads a research group on Design of Information Systems. She is an experienced teacher, and her CV includes co-teaching a course at Stanford’s d.school. Tone was part of the first Participatory Design research project in Norway, and her current research addresses how to reframe PD for contemporary society.

Abstract Human-Computer Interaction concerns both humans and computers and involves a number of different disciplines and interdisciplinary areas. In this talk I discuss how recent technological and socio-cultural developments challenge the established categories of design and use of IT. I suggest seeing use as work, included in the work knowledge and constituting the conditions for the work. Drawing on traditional design disciplines design of IT can be conceptualized as a combination of ideas and materials, where the particular qualities of IT as a design material influence both the design process as well as the design result. In the talk I discuss these conceptualizations and how they can help us discuss how the changing relations between design and use of IT. I draw on both old and new research projects to illustrate how the concepts can be applied.

Ari Kristinn Jónsson

Rector of Reykjavik Unversity

Developing Interactive Artificial Intelligence Software for NASA Mission Control Biography Ari Kristinn Jonsson is the rector of Reykjavik University. He holds a Ph.D. and MS in Computer Science from Stanford University. Before joining the University of Reykjavik he was a senior research scientist at NASA Ames Research Center working on the Mars Exploration Rovers. For details, see his curriculum vitae (http://www.hr.is/starfsfolk/ari).

1

Abstract Over the last decade, artificial intelligence technology has moved from being an obscure research project within NASA to being an important tool for NASA mission controllers who operate spacecraft such as the Mars Exploration Rovers and the International Space Station. This achievement is in part due to advances in artificial intelligence, but a critical part is due to the development of a good understanding of mission controllers needs and how they interact with computer software. This talk presents the development of these interactive software tools, with focus on user involvement and how lessons learned were applied to improve the technology.

Keynote Speakers

Proceedings: NordiCHI 2010, October 16–20, 2010

David Merrill Co-Founder of Sifteo

The New Era of Physical/Digital Play Biography David Merrill is co-founder and president of Sifteo, (http:// sifteo.com/) a company based in San Francisco building the future of play. Sifteo’s first product is Siftables: a tabletop user interface for learning and play made of active physical +graphical tiles. David is a graduate of the Fluid Interfaces Group at the MIT Media Lab, where he studied with professor Pattie Maes and developed the first prototype of Siftables. His work explores how human interactions with computers can leave the limitations of the desktop interface behind, through the development of physical-digital tools that operate comfortably in our real-world environment to enable new forms of play, expressivity, problem-solving and collaboration. He has lectured in computer science at Stanford University and led music controller design workshops at the MIT Media Lab. David holds a Ph.D. and MS from the MIT Media Lab, and an MS in Computer Science and BS in Symbolic Systems from Stanford University. He was a Mayfield fellow in 2001, and was awarded graduate fellowships from Motorola and Samsung at MIT.

Abstract The landscape of human-computer-interface possibilities has changed dramatically in recent years. Rising popularity of mobile phones has driven down the cost of microprocessors and other electronic components, and driven their quality up. Sensors have become tiny and embeddable, and graphical displays can be found on more and more of our personal devices. We are now entering the era of ubiquitous computing that Mark Weiser famously wrote about, where these technologies are disappearing into the fabric of our everyday lives. How will these new capabilities affect our experience of being human? One domain already experiencing great change is play. In this talk I will discuss some changes to the ways we play that are arriving as a result of technological advancement. I will explore implications of these changes for human-computer and human-human interaction, and how technology-based play in the future may look more familiar to us than we may think.

2

Full Papers

Proceedings: NordiCHI 2010, October 16–20, 2010

Let’s All Get Up and Walk to the North Pole: Design and Evaluation of a Mobile Wellness Application Aino Ahtinen, Pertti Huuskonen Nokia Research Center Visiokatu 1, 33720 Tampere, Finland {firstname.lastname}@nokia.com

Jonna Häkkilä Nokia Research Center Yrttipellontie1, 90230 Oulu, Finland [email protected]

ABSTRACT

people to exercise more (e.g. heart rate monitors, pedometers, fitness games), eat healthy (e.g. mobile journaling tools for diet tracking, web-portals for peer support) and relax (e.g. mobile relaxation applications, wrist-watch-type stress measurement devices). Mobile phones provide a promising wellness application platform [5],[13] because of their ubiquitous presence, communication channels, sensor data recording and computing capabilities, combined with their perceived role as personal and trusted technology [28].

Mobile wellness applications can motivate people to exercise more because of their ubiquitous presence and suitable technological possibilities. Wellness applications utilize several factors that can increase physical activity levels of users, e.g., data tracking, social sharing and playfulness. We describe the design and evaluation of a mobile-phone based wellness application addressing social sharing and playfulness. In our user study with 37 participants we focus on presentation of physical activity data as a virtual trip on a map-based game world. The findings reveal design implications that support users to be motivated in doing the physical exercise, including setting departure and destination places and viewing up-to-date progress between them. The virtual trip was considered an understandable, concrete and interesting analogy. The familiarity of the map of the home country as the game world was liked because it added concreteness and understandability to the achievements. Author Keywords

Mobile applications, wellness, playfulness, social sharing, metaphor, analogy, user experience, user studies, evaluation.

Figure 1. Into, a wellness application on the S60 platform.

ACM Classification Keywords

H.1.2 User/Machine Systems: Human factors. H.5.2 User Interfaces: Evaluation/methodology, User-centered design.

This paper explores the design space of mobile wellness applications to support motivation towards physical exercise, which is one of the most important factors in treating and preventing health problems. We have developed a mobile social and playful wellness application called Into (Figure 1). The idea behind it traces back to our former user studies around the topic of mobile wellness applications, in which some participants used an analogy of distance travelled on a map to represent the amount of their physical activities. Those participants recorded the number of kilometers they had been walking, running and biking, marked it down on a map as a line and imagined how far from home they would have travelled and in which place they would be at the moment. Similar analogies have also been used by some organizations in exercise campaigns that aim at increasing the physical activity level of people [20],[26]. We set out to develop this idea into a mobile wellness application with the help of a usercentered design process.

INTRODUCTION

Motivating people to maintain their health through preventive actions has become a global concern that calls for urgent actions. A growing number of health problems due to the modern lifestyle is well-known – people around the world suffer increasingly from obesity, type 2 diabetes, and cardiovascular diseases. The impact of these problems extends to both personal and societal levels. Technologies have been harnessed for wellness support to motivate Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. NordiCHI 2010, October 16–20, 2010, Reykjavik, Iceland. Copyright 2010 ACM ISBN: 978-1-60558-934-3...$5.00.

3

Full Papers

Proceedings: NordiCHI 2010, October 16–20, 2010

A design that provides the user enjoyable and playful experiences leads to positive emotional and motivational states [10]. Therefore, in the domain of wellness applications, the so called ‘fun factor’ or a playful approach has often been chosen to engage and persuade people into wellness activities, especially physical activity. Several activating game concepts and prototypes have been presented in the past years. Fish'n'Steps mobile application in Lin et al. [22] visualizes the level of physical activity with a bowl of growing fish, adding the ‘fun-factor’ and playfulness to the design. In Neat-o-Games [15], a wearable accelerometer provides data that is used to control an avatar representing the player in a virtual race game. Multiple players can participate in the game, and winners are declared on a daily basis. The players can use activity points that they have gained to receive hints in mental games such as Sudoku that are included in the application. In the Play, Mate! design Berkovsky et al. [7] aim at raising the players' motivation by increasing the difficulty of the game and introducing physical activity as a part of the game. One of the most well-known wellness games is Nintendo Wii Fit [37], which includes several types of exercise activities (aerobics, yoga, etc.) in the form of a game.

In this paper we describe, along with the most important related work, a user study of our wellness application. To the best of our knowledge, visualizing physical activity results as a virtual trip on a map has not been studied and analyzed in the user experience studies before, although the analogy has been used in some exercise campaigns, e.g., [20],[26] and commercial games [25]. We present the setting and main findings of our user trial with 37 participants. CURRENT PRACTICES AND BEYOND Applications to Support Physical Activity

The features of current technical applications and solutions that are designed to support physical activity can be roughly categorized to, for instance, four categories [1], but in many cases the categories overlap: • Loggers • Personal trainers • Playful applications and games • Social applications Logger is a simple approach for supporting physical exercise. They are able to measure one or several parameters related to the exercise, and store measurement information into logs. Pedometers offer easy-to-use and inexpensive tools for assessing physical activity. The basic models of the heart rate monitors also act as loggers, as they save data about the pulse, but do not offer sophisticated tools for the analysis or interpretation, or give much guidance. Mobile journaling tools can also act as loggers. An example is provided by the Nokia Wellness Diary [36] for manual tracking of exercise activities and other wellness-related parameters. Tracking and storing physical exercise data in a diary or log can motivate people to increase physical activity levels [29],[31].

Social sharing of the exercise-related data is an important motivator to do more physical activity [21],[34]. Many studies show that social applications can be effective in supporting motivation towards physical activity [5],[12],[35]. Exercising with others provides social support, a feeling of team spirit, an opportunity for social learning as well as pressure to begin and continue with the activities. The often recommended design guideline to “support social influence” [12],[17] has been taken into account in some existing wellness concepts and products. For instance, encouraging results have been gained with mobile phone prototype applications that allow the sharing of activity data, such as step count, within a community of friends [5],[12],[35]. In these studies, sharing the data with peers increased the average of weekly activity. In a different concept, auditory cues of a friend “jogging over distance,” in another geographical location, was found to be encouraging and successful in supporting the need for socializing [27]. Bickmore et al. [9] reported that even sharing information of physical activity with a PC-based animated conversational agent that acted as an exercise advisor increased the motivation to exercise more.

The personal-trainer approach usually includes a logger feature and, in addition, an analysis functionality. Moreover, they can provide feedback on performance. In [11], Buttussi et al. focus on the ability of GPS-enabled mobile devices to offer location-aware feedback. Their solution employs GPS for location information, and the PDA is used for viewing maps and an animated, humanfigured 3D fitness trainer. The trainer offers real-time feedback about how the user is proceeding during the exercise in graphical and audible format, and gives directions for the navigation and assistance of performing fitness tasks. Another example is presented by Asselin et al. [6]. They have developed a wearable Personal Wellness Coach to support a variety of activities such as health data collection, interpretation, feedback and self-monitoring. The Mobile Coach [23] is a mobile application that assists users by giving training programs based on their current fitness and activity level. It analyzes how effective the workouts are and tunes the training program for the user when the schedules change.

Into can be regarded as a playful and social wellness application, as it uses some design elements from both categories (see section “The Into Application”). Trying to reach destinations on the game world can be seen as a playful element, as well as the animal icons representing the team’s speed and the postcard that is received as a reward when reaching the destination. The social elements appear in the team formation, setting a team challenge, progressing towards the challenge with the team and sharing of step count within the team.

4

Full Papers

Proceedings: NordiCHI 2010, October 16–20, 2010

Adventure is launched yearly by the Nuori Suomi organization and it takes place in Finnish primary schools. The school class collect daily exercise points to their personal exercise cards. The points are transformed to combined kilometers of the class, and the imaginary trip progresses from country to country. The progress is viewed on a web portal and a poster placed in the classroom. The campaign has proved to be a liked and effective – for example, in 2009 almost 10 000 school classes joined the campaign and the average time of exercise during the campaign was about 2,5 hours per pupil per day. The same idea is also used by Walking Campaign (Kävelykampanja) arranged by Finnish Kuntoliikuntaliitto [20] for working age citizens. The participants carry a pedometer, and the recorded exercises are manually transferred to the webportal, where the progress can be viewed as a trip on a map.

Our research seeks to articulate the earlier findings and guidelines into practical design decisions, and contributes to the state-of-the-art in further developing and validating the design rationale through research questions we state in section “Research Setting”. Usage of Analogies

The use of analogies and metaphors is common and reasonable in interaction design. Analogies can help in understanding the abstract and hard to imagine computerbased concepts or tasks, because “they provide a structure that is similar in some way to aspects of a familiar entity” [30]. The ‘match between system and real world’ principle has been one of the usability heuristics for a long time by now – the system should speak the user’s language, with words, phrases and concepts familiar to the user [24]. One of the most familiar metaphors is the desktop metaphor. The usage of interface metaphors has proven to be highly successful in helping users to use technological systems [30]. However, sometimes the metaphors cause confusion due to, for example, users’ cultural backgrounds [14].

A similar analogy is also in use in the Nintendo Personal Trainer Walking. Step info from the activity sensors is used to draw pictures of the earth, and the more the user walks, the more new pictures are drawn on the terrain. In another game from the same company, the users can post their step counts to combine them with other users’ data, to plot a virtual walk across the solar system. [25]

In the domain of wellness applications, the data (e.g. amount of exercise or burned calories) has traditionally been presented with numbers and graphs. However, new analogies of visualizing the wellness data have been proposed. Instead of numbers and graphs, Ståhl et al. [33] used animated human-like figures to visualize the wellness data. In their Affective Diary concept, the data was shown in a way that reminded users of their own bodies. Consolvo et al. [13] visualized the progress with a growing garden including butterflies and flowers and they reported that their system was effective in motivating to physical activity. Similarly, Albaina et al. [3] used an emotional flower in their photo frame concept to visualize walking activities. In addition, Lin et al. [22] presented the level of physical activity with a bowl of growing fish.

The analogy of presenting physical activity data as a virtual trip on a map differs from the approach where the physical activity data is recorded with GPS-based technology. The analogy should not be confused with GPS-based applications, like Nokia Sports Tracker [32] or Garmin Forerunner [16]. In the analogy, the map and trips are used as a metaphor and visualization, and they do not represent real routes conducted by the users in any other sense than the distance progressed. Even though the analogy of a virtual trip has already been successfully used in the exercise campaigns and is visible in some games, we are not aware of any mobile phone applications that utilize it, or user trials that have studied and analyzed the user experiences and perceptions of the approach. Our platform of choice, the mobile phone, offers capabilities that are superior to stand-alone diaries, game consoles or GPS devices—the phone is a device that is omnipresent but personal, yet gives natural access to social networks and networked services.

Compared to these delightful but abstract solutions, we explicitly seek presentations that are more concrete and measurable. We chose maps and distances travelled on them as the key presentational principles, since we hypothesised that these concepts would be very concrete for humans and provide a meaningful mapping from metaphors to physical activity. Visualizing Physical Activities as a Trip

RESEARCH SETTING Pre-study: Testing the Concept Design

Showing step-based physical activity as a distance is a familiar feature on pedometers. Even the simplest pedometers present the activity data both in steps and kilometers. A phone-based pedometer provides possibilities to design applications to support physical activity. For example, Apple provides several applications to track the activity with the accelerometer-based pedometer and view the distance and burned calories [18].

In our user-centered design process we first evaluated the Into concept with users in a simulated setup. A total of 12 people in 4 teams participated in a pre-study, where the idea of the game was evaluated with a step counter and paper prototype. We used pedometers on mobile phones to track the exercise, real world team formation, a paper map in which the progress was plotted (see Figure 2), and printed postcards as rewards when the team reached their target destinations (see Figure 3).

The analogy of presenting visual activity data as a trip has been used in a Finnish fitness campaign called “Exercise Adventure” (Liikuntaseikkailu) [26]. The Exercise

5

Full Papers

Proceedings: NordiCHI 2010, October 16–20, 2010

Figure 2. Initial concept evaluation. A team gathered around a paper map where departure and destination points and the progress of teams were plotted.

Figure 3. Initial concept evaluation. A printed postcard from the destination was delivered to the team members when they completed a challenge.

The results of the prestudy helped to refine the concept and highlighted the essential design aspects that should be taken into account when transferring the design to a mobile application. The salient findings of the pre-study revealed that the game idea was found compelling. The participants reported example situations where they had deliberately increased their amount of exercise to get more steps (e.g. walked to work instead of taking a bus) and thus better results for the team. The familiar environment the map was representing, the support of team play, and the need for constant feedback surfaced as essential design factors. Also, implementing the game board as a mobile phone application was perceived as a good idea – the paper prototype could of course not provide automated status updates, and the pedometer was already integrated with a phone anyway.

mobile phone. The users “proceed” (travel) on a map based on their step count. First, the user creates a challenge, such as “Proceed from Helsinki to Tampere.” The application then displays the length of the challenge in steps and kilometers. The user can either play alone, or more interestingly, invite members to the team. The team members’ step counts are synchronized to a server and sent back to the mobile phones of all team members. The team’s combined progress is currently shown as a trip on a map and as numerical information (see Figures 4A & 6B). Based on the speed that the team progresses, the application shows an animal figure, e.g., a snail, a galloping horse or a flying eagle (see Figure 4B). In addition, the progress of each team member (steps, kilometers) is displayed separately as a list, which can be viewed by all team members (see Figure 5A). When the target destination is reached, the team receives a reward – an electronic postcard from the target city. The collection of received postcards can be viewed in the application (see Figure 5B). The postcards can be opened as full screen images.

The second step of the process was the iterative concept and UI design with several usability tests and iteration rounds both in the laboratory and field settings. The participants of the iterative user evaluations were content with the concept of visualizing physical activity results as a virtual trip on a map. The findings from the early evaluations led us to formulate the following research questions: (Q1) How do the users perceive the visualization of physical activity results as a virtual trip on a map? (Q2) What do the users think about the physical activity challenges that are presented as trips between places, for example cities or villages? (Q3) How do the users react to a familiar map, i.e. the map of their home country as the game world? The Into Application

Into is a mobile java application currently available for the Symbian S60 platform. It combines social team play with other playful elements to increase motivation towards physical activity. Into records the user’s steps with an accelerometer-based pedometer, which is in-built in the

Figure 4. Into application. (A) The team’s combined progress in numbers and as a line on the map, (B) An animal figure showing the speed of the team’s progress.

6

Full Papers

Proceedings: NordiCHI 2010, October 16–20, 2010

appealing the participants found it; and whether they would recommend it to an interested friend. The responses were given with a 5-point Likert scale. The web link to the online questionnaire was sent to the participants’ email addresses after one week of use of Into, and the response rate was 100%. Responding to the questionnaire took about 5-10 minutes.

Methodology

We carried out a trial with 37 participants to examine the participants’ subjective responses towards Into, focusing especially to their perceptions of visualizing physical activity results as a virtual trip. The trial lasted one week, and it was conducted in Finland in late 2009. The recruited participants were existing social groups, i.e. groups of friends, colleagues, neighbors and families. In total, nine Into teams consisting of 2 to 6 members were formed. Most teams had 4 members. Kick-off sessions were arranged with each team separately, and all members of the team were present in the session. The functionality and features of Into were demonstrated to the participants, as well as the phases and tasks related to the study period. The participants were handed out Nokia 6210 mobile phones, with Into installed, as well as pre-paid SIM cards. One of the team members was named as the team leader and a team challenge was created for the team, to be visible in all members’ mobile phones. The participants were advised to keep the mobile phone with them all the time and have the application running on the background. The users were educated for the technical idiosyncrasies of the application used in the study. The team leader was advised to create a new challenge whenever they completed the previous challenge. For testing purposes, the application included an additional feature, a step multiplier function. The multiplier was set to 5 for this evaluation due to the relatively short duration of the period and long travel distances. This meant that the participants step count achievements were multiplied by 5, and to get the real step count they needed to divide it by 5.

The open-ended questions of the interviews dealt with, e.g., the following themes: how did the participants find the idea of representing physical activity results as a virtual trip on a map; how did they like the challenges that were formulated as trips between cities on a map; and what were their reactions towards the map of their home country as the game world. The single-person 15-20 minute recorded interviews were carried out by phone after the trial period. Most of the participants responded first to the online questionnaire; all participated to the interview. The interview data was analyzed with the qualitative thematic classification method, which is similar to the Affinity wall technique [8]. First the recordings were transcribed word by word. Secondly, the raw data was classified under the main research themes, e.g., physical activity as a virtual trip; challenges presented as trips between places; and familiar game world. The ideas for the further design were the fourth theme to be discussed in this paper. There were also other main themes formed, e.g., social competition, social cooperation, postcards as rewards and icons reflecting the team’s speed, but they are out of the scope of this article. Inside each main theme, the raw data was further classified to form subcategories. The classification was done on computer by using spreadsheets, and not on the wall as is done in the Affinity wall technique. The “Findings” section describes the main findings from the four main themes on focus.

Online questionnaires and semi-structured interviews were used as data collection methods after one week of use. In the online questionnaire, we asked questions about, e.g., the motivating role of Into; how suitable its features are; how

Participant Profile

The 37 participants (31 females, 6 males) from 20 to 55 years (80% were 25-39), were from different backgrounds, e.g., medical doctor, entrepreneur, salesperson, nurse and researcher. The majority were sedentary workers. Most of them had no technical background, but they were experienced mobile phone users: 75% of them had used mobile phones for over 10 years. Five of the participants had no previous experience in using wellness-related technologies, while the rest had tried or used actively gadgets or applications such as heart rate monitors, pedometers, or Wii Fit. The participants had varying backgrounds in their physical activity levels. The four most popular activities among the participants were walking, jogging, gym and ball games. Most participants (70%) reported that they did physical exercises several times a week, 16% exercised once a week and 13% more rarely.

Figure 5. Into application. (A) The list view of the team members and their current achievements. (B) The list view of received postcards from the reached destinations.

7

Full Papers

Some negative effect

No effect

Some positive effect

Very positive effect

50,0

18,4

Competition between the team members

0,0

2,6

15,8

52,6

23,7

13,5

Challenges (A to B)

0,0

0,0

31,6

60,5

7,9

13,5

Viewing progress as a trip on the map

0,0

0,0

18,4

60,5

15,8

Map of home country as a game world

0,0

2,6

42,1

42,1

7,9

Step multiplier

10,5

23,7

31,6

31,6

2,6

Rewards (postcards)

0,0

2,6

60,5

28,9

0,0

Icons reflecting the speed (animals)

0,0

5,3

47,4

31,6

7,9

Totally agree

21,1

Somewha t agree

0,0

Neither disagree or agree

0,0

Somewha t disagree

Cooperation between the team members

Totally disagree

Very negative effect

Proceedings: NordiCHI 2010, October 16–20, 2010

Encourages me to do more physical activity

0,0

13,5

2,7

67,5

16,2

Includes features & functionalities suitable for me

0,0

Is an appealing application

0,0

5,4

5,4

75,6

I would recommend Into to a friend

0,0

13,5

2,7

70,2

15,8

7,9

71,1

5,3

Table 1. Responses to the question: “Rate the following statements according to your opinion”. The distribution of the responses in percentages, n=37. “Don’t know” responses not included in the table. FINDINGS General Findings

In the online questionnaire we asked questions about the motivating role of Into; how suitable its features were; how appealing the participants found it; and whether they would recommend it to an interested friend. The questions and the distribution of the responses in percentages can be found in Table 1. In all questions, most of the responses range from “somewhat agree” to “totally agree”. This clear trend indicates that the participants perceived Into as a motivating and appealing application, which has a suitable set of features, and they would also like to recommend it to others.

Table 2. Responses to the question: “What aspects affected your usage motivation of Into?” The distribution of the responses in percentages, n=37. “Don’t know” responses not included in the table.

the usage motivation of the application. About 76% of the participants rated that viewing the progress as a trip on the map had some or very positive effect on the usage motivation of the application. Almost 70% of the participants rated that challenges had some or very positive effect. The map of the home country as the game world was rated as having some or very positive effect by half of the participants.

We also asked the participants to rate how the different aspects of Into affected the usage motivation of the application. The distribution of the responses in percentages is shown in Table 2.

The step multiplier (5) used in the study was the only aspect that seemed to affect the usage motivation of the application somewhat negatively: almost 35% rated it as having very or some negative effect on the usage motivation. The participants commented on the interviews that they would rather have used the application without any multiplier because they wanted to see easily the realistic progress, i.e. real steps taken and kilometers travelled. The rewards and icons reflecting the speed of the team had some positive effect on the usage motivation: about 30% and 40% of the respondents considered them as having some or very positive effect.

The responses reveal that social aspects (cooperation and competition between the team members) were rated as having a positive effect on the usage motivation of the application: more than half of the participants had the opinion that the social aspects affected positively to the usage motivation of Into (see Table 2, marked as dark grey). We are satisfied with the overall finding from the interviews that Into enabled both competition and cooperation at the same time, and they were perceived as motivating factors by the participants. The users could make their selection between mentally competing against the other team members by comparing the results in the Team view (see Figure 5A), or just cooperating with them by contributing as well as possible to reach the team challenge together.

Next, we will cover in detail (1) how the analogy of presenting physical activity results as a virtual trip worked in the Into application and (2) the related user perceptions from the user study. The user study findings are based on the qualitative data from the semi-structured interviews with the 37 participants. The findings will be presented with the authentic user comments from the interviews.

Marked as light grey in Table 2, the aspects related to visualizing physical activity results as a virtual trip (challenges reflecting trips between places; viewing the progress between the places on a map; and the map of the home country as the game world) had a positive effect on

Visualizing Physical Activity Results as a Virtual Trip

In Into, the physical activity results, i.e. steps, were visualized for the users with an analogy of a virtual trip. The users travelled was on a map of their home country,

8

Full Papers

Proceedings: NordiCHI 2010, October 16–20, 2010

achievements more concrete than viewing the achievements as kilometers or steps, which was an alternative way to view the achievements in Into. The following examples of participant comments reveal how positively the participants reacted to the concept of visualizing physical activity achievements as a virtual trip, and how they emphasized the concreteness of the analogy: “It is good to have a destination as a target where one needs to get. I believe it motivates more than just seeing how long distance one has walked or run.” (male, team of 6 neighbors) “The concreteness is a good factor.” (female, team of 6 neighbors) “To view the progress on the map makes the progress much more concrete. And it makes it more interesting when compared to seeing only kilometers.” (female, team of 4 friends)

Figure 6. Into application. (A) Setting a departure point. (B) The team’s combined progress visualized as a trip on the map.

Finland. The departure and destination places could be selected from a list including 15 cities or smaller towns. As the team members took steps, the application combined the achievements of each team member and visualized the combined progress as a trip between the departure and destination places. The more they took steps, the quicker the red line between the places turned to green. The line between the departure and destination places reflected the true distance between the places in the physical world, and respectively the users needed to take as many steps together as the real distance required.

Following the progress on a map made viewing the physical activity achievement interesting for the participants. Many users liked to view the map often to see if they were already on the half way or if they soon reached the destination. Between the departure place and the destination they could think which places they had already passed along the way and by which place they were at the moment: “I thought about by which city we were at the moment. For example, when travelling to Helsinki we were by Hämeenlinna at the certain moment.” (female, team of 5 friends) “I think it is great to be able to see where we are going at the moment, that’s the best thing. To see the up-to-date situation and how much there is still left.” (female, team of 6 neighbors)

Concrete, Understandable and Interesting Analogy

The following open-ended interview questions contributed to the findings of this section: • What was your general impression of the usage of Into? • What were the factors or aspects that motivated the use? Why?

In addition to being concrete and understandable, the participants also experienced positive feelings (fun, enjoyment) when looking at the map and the progress on it: ”The longer the green line, the better mood I got.” (female, team of 4 colleagues) “It is fun! I mean the idea that we are walking towards some destination on that map. The map made it more concrete.” (female, team of 4 family members)

• What do you think about presenting the achievements of the team as a trip on a map? • How do you feel about the current challenges? • What kind of challenges would you like to have in the application? Why? Visualizing physical activity achievements as a virtual trip between cities or villages was liked by the participants. They emphasized the concreteness and understandability of the approach, and also commented that it made viewing the achievements interesting and fun. They explained that the target felt more concrete because there was a clear goal of travelling virtually from one place to another and realistic departure and destination places. The participants liked the idea of travelling between real-life cities, towns and villages, as can be seen on the following comments: “Nice idea to be able to see that we are progressing from city to city. It is a realistic alternative.” (female, team of 4 colleagues) “As a mental image a trip between cities is very nice.” (female, team of 6 neighbors)

The Familiar Game World

The following open-ended questions on the interviews contributed to the findings of this section: • What do you think of the current game world of Into? • What kind of a game world would you like to have in the application? Why? • How do you feel about the current challenges? • What kind of challenges would you like to have in the application? Why? The game world where the teams were travelling was a map of Finland, their home country. Thus, the departure and destination places were Finnish cities, towns and villages. Naturally, the map of Finland was quite familiar

For most of the participants, travelling virtually between cities or villages made viewing the physical activity

9

Full Papers

Proceedings: NordiCHI 2010, October 16–20, 2010

“On the way it could tell that now you are in Hämeenlinna, for example. It should include that kind of intermediate things.” (male, team of 3 family members) “Sometimes it would be nice to get information like now you are proceeding by the Toijala church or Ideapark shopping complex, or some similar small feedback to the users.” (female, team of 4 colleaques)

for the participants as they were all native Finnish citizens. The participants emphasized that the familiarity of the game world was a very good aspect, as they knew in advance approximately how long the distances were between the places. They had former experience in travelling between places in the real world and thus they had a mental picture of the distances and how much time and effort those would require. The familiarity of the game world was an important factor to make the challenges feel concrete: “The challenges from one city to another are great. They are concrete because one knows those cities.” (male, team of 3 family members) “For me map of Finland worked very well. One has a point of comparison for real life. I thought that in some phase when Finland would be roamed through why not to expand to abroad but initially I would not need anything else than Finland.” (female, team of 4 colleagues)

Clearly connected to the previous wish, the participants wanted to be able to zoom in on the map to view their current location in more detail. In the current version there was no zooming option and it included only one, general map level. Alone this one map level was not considered to be enough, but the participants wanted to zoom in to see a more detailed part of the trip. This function would enable users to get more feedback about the progress. This related also to keeping up users’ interest and usage motivation. “It takes some time to see the progress in the beginning of the challenge, because the whole map of Finland is there. It would be great to be able to zoom in to the ongoing trip so that one could see the progress better. Now one needs to walk quite long before seeing any achievement.” (female, team of 4 colleagues) “The map increased the usage motivation of the application. But it could also be better in a way that one could zoom in closer to see more detailed map.” (female, team of 4 colleagues)

Most participants commented that travelling between the provided cities and villages was very clear and concrete way to view the progress. However, a minority of the participants said that they could not perceive the distances between the provided places, because they were not so familiar with them. These participants wished to have a possibility to set the challenge between the places that were even more familiar to them, for example one participant wanted set the route to grandmother’s house as a challenge, and another wished to use the trip to the summer cottage: “I’m so weak in geography that I have no idea what it means to travel from, for example, Mikkeli to somewhere.” (female, team of 6 neighbors) “It would be nice to create own, personal challenges, because not everybody live in bigger towns. Smaller places would be nice. I think I would like to travel to our own summer cottage. That would be more concrete and comparable then.” (female, team of 4 friends)

Some participants felt that even though a familiar game world was welcome, travelling the same routes on and on again might be less interesting in the long run. They wished that after Finland would be roamed through, the game world would expand to contain other countries as well. Some participants also wished to be able to travel between famous points of interest or along a well-known travel path: ”If one used the application for a long time, going same routes again and again could become boring. For the sake of meaningfulness and variability it would be good to be able to get the maps of other countries. One would learn geography as well.” (female, team of 4 colleagues) “It would be really great if there were some famous routes, for example the pilgrimage in Spain, so that you could go walking virtually to Santiago de Compostela or some other route. Or you could decide to walk a virtual tour in Rome this weekend.” (male, team of 3 family members)

Ideas for Further Design

The interviews revealed three main wishes from the participants for the future design of the application related to the virtual trips, challenges and the game world. They were intermediate points, map zooming and more versatile types of challenges, to be discussed in this section. As could be seen on the user comments and perceptions on the previous sections, many participants liked to view and think about where they were going at the certain moment on the map. The current version did not show any intermediate points on the way from the departure to the destination, so the users needed to guess what there would be along the way. To maintain the user interest in the longer time period the challenges could be divided into shorter stages, and reaching these stages could reveal some content on the map, for example, the name of the city, village, castle, or other landmark that the team has passed by, or some visual material from those places:

Some participants lamented the fact that accelerometerbased pedometers can not measure all exercise types (e.g. biking or swimming), and thus the users did not get any credit out of such efforts. Into shares this problem with other step counter based applications. The participants also mentioned that they did not want to keep the phone in the pocket at home, and consequently missed steps. Moreover, sometimes when exercising they did not have suitable pockets in which to keep the mobile phone. Although mobile phones are nowadays carried along in almost all situations, clearly there are some occasions when it is not convenient to carry one. Similar limitations apply to most wellness technology, however.

10

Full Papers

Proceedings: NordiCHI 2010, October 16–20, 2010

related lifestyle changes usually do not happen immediately. The participants wished for intermediate points when travelling between places, and a zoom infeature to view the game world and route in more detail. Those would enable better possibilities for exploring the game world, which we think is one of the factors to keep up the long term user interest.

DISCUSSION & CONCLUSIONS

We have presented the background and design of a mobile application for supporting physical exercise activities. The Into application utilizes two known motivating factors towards increasing the level of physical activity: social sharing and playfulness. In the application, the physical activity progress of the team is presented with an analogy of a virtual trip. The main findings (detailed in Section “Findings”) related to the use of the analogy can be summarized as follows:

More versatile challenges and routes, maybe also an expanding game world, might increase the motivation towards the use of the application and conducting physical activities over longer time periods. For this aim we would like to explore the future use of real-world maps that are already included in many high-end phones for navigational purposes. Alternatively, artificial yet familiar wellness game worlds might be inviting for players of non-wellness games. For instance people who spend substantial fractions of their waking hours in the realms of World of Warcraft might prefer to map their physical activity into the same familiar universe.

• Users liked the idea of presenting the team’s physical activity results as a virtual trip, including setting departure and destination places and viewing an up-todate progress between them. • A virtual trip was considered an understandable, concrete and interesting analogy. • According to the users, the familiarity of the game world was an important factor to make the challenges feel concrete.

As our present study was qualitative in nature, and the trial period of the application was relatively short, broad generalizations based on the results are not relevant. Moreover, in the existing social groups available for the study, gender balance could not be achieved. In addition, the participants all came from a nation with a strong culture of outdoor activities [19] and a pervasive use of mobile devices for communication and exercise tracking.

Our study showed that the familiarity of the game world, in this case a map of the participants’ home country, provided a successful mental model for perceiving and understanding the physical activity data presentation. The participants knew in advance approximately how long the distances were between the places as they had former experience in travelling between them. So they had a mental picture of the distances and how much time and effort it would require to travel between them. Our trip analogy proved successful in this experiment.

Interesting research questions for the future research do arise. We would like to study further the use of the application over a longer time period and in different contexts. Dix [14] warns about the cultural bias that the metaphors may portray. That is why we would like to compare users from different cultures to know how the analogy of a virtual trip for presenting physical activity data works globally. We would also like to study more the familiarity of the game world, whether we can generalize the results to the representatives from other cultures as well.

The same analogy has also been successfully used by some exercise campaigns [20], [26] though without mobile phone technology. Mobile phone as a platform for the application enables effortless data measuring with an inbuilt accelerometer, seamless and real-time data transfer between the team members, and up-to-date follow-up of the progress, when compared to the manual tracking and data transfer techniques used by the above mentioned exercise campaigns. Our study did remind us, however, that a mobile phone is not always the ideal tool for tracking the exercise, because in some situations it is not convenient to carry along. People may not want to carry a phone when they spend time at home, and sometimes when exercising they do not have suitable pockets for the phone. A very small wearable sensor could solve the problem, but on the other hand it would be another separate piece to remember. Sensors that are integrated into fitness clothing and able to communicate with mobile devices offer a partial solution to this problem. However, of the contemporary devices that are widely accepted among the consumers, the mobile phone was the best platform for our application.

We were satisfied with the application in relation to our research focus, as it allowed us to evaluate it in the field settings in real use by people without technical background. The application concept seems to be a good one, as the user comments were so positive in spite of the limitations. All participants used the application until the end of the period and a majority of them liked it a lot. Future work includes longer term experiments to provide quantifiable measures on the application’s ability to improve exercise motivation. REFERENCES [1] Ahtinen, A., Isomursu, M. Huhtala, Y., Kaasinen, J., Salminen, J. and Häkkilä, J., 2008. Tracking Outdoor Sports – User Experience Perspective. In: E. Aarts, ed. European Conference Ambient Intelligence, Springer-Verlag, 192-209. [2] Ahtinen, A., Isomursu, M., Mukhtar, M., Mäntyjärvi, J., Häkkilä, J., and Blom, J. 2009. Designing social features for mobile and ubiquitous wellness applications. In Proc. 8th

Encouraged by the positive feedback from the participants, we will continue with the development of the concept. Maintaining long term motivation and rewarding should be addressed in Into, like in any application meant for supporting and motivating physical activity [12]. Wellness

11

Full Papers

[3]

[4]

[5]

[6]

[7] [8] [9]

[10] [11]

[12] [13]

[14] [15] [16] [17]

Proceedings: NordiCHI 2010, October 16–20, 2010

intl. conf. on Mobile and Ubiquitous Multimedia (MUM '09). ACM, New York, NY, 1-10. Ahtinen, A., Ramiah, S., Blom, J. and Isomursu, M., 2008. Design of Mobile Wellness Applications: Identifying CrossCultural Factors. In Proc. 20th Australasian Conf. on Computer-Human Interaction. Albaina, I. M., Visser, T., Van der Mast, C. and Vastenburg, M. H., 2009. Flowie: A Persuasive Virtual Coach to Motivate Elderly Individuals to Walk. In 3rd International ICST Conf. on Pervasive Computing Technologies for Healthcare, 1-3 April 2009 London, UK. Anderson, I., Maitland, J., Sherwood, S., Barkhuus, L., Chalmers, M., Hall, M., Brown, B. and Muller, H. Shakra: Tracking and Sharing Daily Activity Levels with Unaugmented Mobile Phones. Mobile Network Applications, 12, 2-3 (2007), 185-199. Asselin, R., Ortiz, G., Pui, J., Smailagic, A., and Kissling, C. Implementation and Evaluation of the Personal Wellness Coach. In Proc. 25th IEEE Intl. Conf. on Distributed Computing Systems Workshops, pp. 529-535 (2005). Berkovsky, S., Bhandari, D., Kimani, S., Colineau, N., and Paris, C. 2009. Designing games to motivate physical activity. In Proc. Persuasive 2009, ACM Press (2009), 1-4. Beyer, H. and Holtzblatt, K. Contextual Design – Defining Customer-Centered Systems. Morgan Kauffman, 1997. Bickmore, T. W., Caruso, L., and Clough-Gorr, K. Acceptance and Usability of a Relational Agent Interface by Urban older Adults. Extended abstracts of the Conf. on Human Factors in Computing Systems, pp. 1212-215 (2005). Blythe, M.A., Overbeeke, K., Monk, A.F. and Wright, P.C. (eds.) Funology, From Usability to Enjoyment. Kluwer Academic Publishers, MA, USA, 2003. Buttussi, F., Chittaro, L., and Nadalutti, D. Bringing Mobile Guides and Fitness Activities Together: A Solution based on an embodied virtual trainer. In Proc. 8th Conf. on Humancomputer interaction with mobile devices and services, pp. 29—36 (2006) Consolvo, S. Everitt, K., Smith, I. and Landay, J.A. Design Requirements for Technologies that Encourage Physical Activity. In Proc. CHI 2006, ACM Press (2006), 457-466. Consolvo, S., McDonald, D. W., Toscos, T., Chen, M. Y., Froehlich, J., Harrison, B., Klasnja, P., LaMarca, A., LeGrand, L., Libby, R., Smith, I. and Landay, J.A., 2008. Activity Sensing in the Wild: A Field Trial of UbiFit Garden. In Proc. CHI 2008, ACM Press (2008), 1797-1806. Dix, A., Finlay, J., Abowd, G. and Beale, R. HumanComputer Interaction. 2nd edition. Pearson Education Limited, UK. Fujiki, Y., Kazakos, K., Puri, C., Buddharaju, P., Pavlidia, I., and Levine, J. NEAT-o-games: blending physical activity and fun in the daily routine. Comput. Entertain. 6, 2 (2008). Garmin. http://www.garmin.com Gockley, R., Marotta, M., Rogoff, C. and Tang, A. AVIVA: A Health and Fitness Monitor for Young Women. Ext. Abstracts CHI 2006, ACM Press (2006).

[18] iPhone pedometer applications. http://www.iphoneappsfinder.com/blog/15-best-pedometerapplications-for-iphone/ [19] Kafatos, A., Manios, Y., Markatji, I., Giachetti, I., Vaz de Almeida, M. D. and Engstrom, L. M. Regional, demographic and national influences on attitudes and beliefs with regard to physical activity, body weight and health in a nationally representative sample in the European Union. Public Health Nutrition: 2, 1a (1999). [20] Kuntoliikuntaliitto Kävelykampanja. http://195.17.28.12/kuntostart/index.asp [21] Laverie, D.A. Motivations for Ongoing Participation in Fitness Activity. Leisure Sciences 20 (1998). [22] Lin, J. J., Mamykina, L., Lindtner, S. Delajoux, G. and Strub, H. B. Fish’n’Steps: Encouraging Physical Activity with an Interactive Computer Game. In Proc. Ubicomp 2006, Springer (2006), 261-278. [23] Mobile Coach. http://www.firstbeattechnologies.com [24] Nielsen, J. Usability Engineering. 1993. Academic Press, US. [25] Nintendo. http://www.personaltrainerwalking.com/walk-theworld/ [26] Nuori Suomi Liikuntaseikkailu. http://maapalloseikkailu.nuorisuomi.fi/fi-FI/ [27] O’Brian, S. and Mueller, F. Jogging the Distance. In Proc. CHI 2007, ACM Press (2007), 523-526. [28] Patrick, K., Griswold, W.G., Raab, F. and Intille, S.S. Health and the Mobile Phone. American Journal of Preventive Medicine 35, 2 (2008), 177-181. [29] Schlenk, E. A., Dunbar-Jacob, J., Sereika, S., Starz, T., Okifuji, A. and Turk, D. Comparability of daily diaries and accelerometers in exercise adherence in fibromyalgia syndrome. Measurement in Physical Education & Exercise Science 4, 2 (2000). [30] Sharp, H., Rogers, Y. and Preece, J. Interaction Design: Beyond Human-computer Interaction. 2nd Ed. 2007. John Wiley & Sons, UK. [31] Speck, B. J. and Looney, S. W. Effects of a Minimal Intervention to Increase Physical Activity in Women: Daily Activity Records. Nursing Research 50, 6 (2001). [32] Sports Tracker. http://sportstracker.nokia.com/nts/main/index.do [33] Ståhl, A., Höök, K., Svensson, M., Taylor, A.S. and Combetto, M., 2009. Experiencing the Affective Diary. Personal and Ubiquitous Computing 13 (5), 365-378. [34] Ståhl, T., Rütten, A., Nutbeam, D., Bauman, A., Kannas, L., Abel, T., Lüschen, G., Rodriquez, D., Vinck, J. and van der Zee, J. The importance of the social environment for physically active lifestyle – results from an international study. Social Science and Medicine 52 (2001). [35] Toscos, T., Faber, A., Connelly, K. and Upoma, A.M. Encouraging Physical Activity in Teens – Can technology help reduce barriers to physical activity in adolescent girls? In Proc. Pervasive Health 2008. [36] Wellness Diary. http://betalabs.nokia.com/apps/wellnessdiary [37] Wii Fit. http://wiifit.com/

12

Full Papers

Proceedings: NordiCHI 2010, October 16–20, 2010

Location-based Crowdsourcing: Extending Crowdsourcing to the Real World Florian Alt, Alireza Sahami Shirazi, Albrecht Schmidt, Urs Kramer, Zahid Nawaz University of Duisburg-Essen Pervasive Computing and User Interface Engineering Schützenbahn 70, 45117 Essen, Germany {alireza.sahami, florian.alt, albrecht.schmidt}@uni-due.de, {urs.kramer, zahid.nawaz}@stud.uni-due.de ABSTRACT

Many projects emerged over the past years whose success is based on the contributions of a huge number of people. Wikipedia is a prominent example, which utilizes the broad knowledge of a massive number of people on the Internet. OpenStreetMap is another example where many users, living in different geographical regions, contribute, share, and process their location tracks to make a comprehensive online map. These are just two of many examples where a large number of people, who are often part of a community, make small contributions, which led to a completely new type of applications that would have been hardly imaginable before the pervasive availability of the WWW.

The WWW and the mobile phone have become an essential means for sharing implicitly and explicitly generated information and a communication platform for many people. With the increasing ubiquity of location sensing included in mobile devices we investigate the arising opportunities for mobile crowdsourcing making use of the real world context. In this paper we assess how the idea of user-generated content, web-based crowdsourcing, and mobile electronic coordination can be combined to extend crowdsourcing beyond the digital domain and link it to tasks in the real world. To explore our concept we implemented a crowdsourcing platform that integrates location as a parameter for distributing tasks to workers. In the paper we describe the concept and design of the platform and discuss the results of two user studies. Overall the findings show that integrating tasks in the physical world is useful and feasible. We observed that (1) mobile workers prefer to pull tasks rather than getting them pushed, (2) requests for pictures were the most favored tasks, and (3) users tended to solve tasks mainly in close proximity to their homes. Based on this, we discuss issues that should be considered during designing mobile crowdsourcing applications.

With the ubiquity of interactive mobile devices providing location awareness and network connectivity we expect this trend to accelerate. People carry their phones with them the entire day, providing them the opportunity to contribute at any time. We imagine that new forms of contributions (e.g., real-time media and tasks that require physical presence) will become accessible similar to knowledge work and information sharing in the WWW. Smart mobs [15] and dynamic ride sharing services are current examples that involve physical presence in order to complete a task. One specific form of harvesting wisdom of the crowd and contributions from users is crowdsourcing, as introduced by Jeff Howe [6]. The concept describes a distributed problem-solving and product model, in which small tasks are broadcasted to a crowd in the form of open calls for solutions. As a strategic model, crowdsourcing tries to attract interested and motivated crowds capable of providing the required solutions in return for incentives (mainly small amounts of money). Often, such so-called crowd workers gather in online communities consisting of experts, small businesses, and other volunteers working in their spare time. As a result, problems can be addressed very quickly, at little cost, and the task provider might exploit a wider range of talents [9]. Tasks are normally initiated by a client and are open either to anyone or to particular communities. The solution may be submitted by individuals as well as by a group. In comparison with ordinary “outsourcing”, a task or problem is outsourced to an undefined public rather than to a specific body. Crowdsourcing is effective in areas where the task can be easily described to humans and where these tasks are easier to do for humans than for computers, e.g., perception tasks and tasks involving creativity.

Authors Keywords

crowdsourcing, mobile phone, context, location Categories and Subject Descriptors

H.5.3 [Group and Organization Interfaces]: Collaborative computing, Evaluation/Methodology, Organizational design; H.5.2 [User Interfaces] INTRODUCTION

Over the years the World Wild Web (WWW) has evolved beyond being a platform for retrieving information only but has become a ubiquitous medium supporting various forms of communication, peer-to-peer interactions, shared collaboration, and the creation of user-generated content.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. NordiCHI 2010, October 16–20, 2010, Reykjavik, Iceland. Copyright 2010 ACM ISBN: 978-1-60558-934-3...$5.00.

13

Full Papers

Proceedings: NordiCHI 2010, October 16–20, 2010

We argue in this paper that mobile crowdsourcing offers great potential and new qualities when considering and exploiting the context of the user, e.g., his location. Mobile phones are ubiquitous in many parts of the world. Nowadays most devices provide not only means for communication and interaction, but they typically are enhanced with a range of different sensors (e.g., camera, GPS, accelerometer), hence making it possible to easily extract context information. When additionally considering that the WWW and data services are becoming more common on mobile phones, we envision such devices being the upcoming platform for crowdsourcing. We believe that mobile, and especially location-based, crowdsourcing has the potential to go beyond what is commonly referred to as “traditional” (digital) crowdsourcing by bringing it to the real world. Therefore we exploit both the seekers’ and the solvers’ physical location. We focus especially on tasks that go beyond the provision of digital content with no clear limitation on how they are being solved.

rely on the active contribution of the Web community are the center of the Web 2.0 phenomena. The contribution to services happens in different ways. In the following we discuss user-generated content and crowdsourcing in more details, as they have been the inspiration for the implemented platform. User-Generated Content

The creation of content can be discriminated in explicit and implicit content generation. It can be generated on the initiative of the contributor (e.g., adding a new entry in Wikipedia), based on a coordinated call (e.g., someone asks for the clean-up of an article, someone initiates a call to read chapters in librivox1), or on request from a potential web user (e.g., a request in a forum). Explicit content generation describes the process in which a number of web users individually produce content. The content production may be carried out independently or as a part of a coordinated effort. In both cases the central value is in the collective result. Wikipedia, an online encyclopedia, is an example, which is created based on entries added by a large number of web users. Similar examples are product reviews, experience reports, and recommendations provided by customers for others in online shopping platforms. Such collections are sometimes seen as making use of the wisdom of the crowd. There have been recent researches that assess how to best harness the wisdom of the crowds [9] [4]. Explicitly generated content requires effort by the user and typical incentives are peer recognition or immaterial or material benefits, such as payments or vouchers. In [12] the authors investigate how financial incentives impact the performance.

We conducted two field studies that focus on the evaluation of constraints and challenges that affect the crowd workers’ behavior. We found out that location (and hence the opportunity to retrieve tasks in the vicinity) has a crucial impact when it comes to assigning tasks by the crowd workers. Nowadays time is scarce. Thus most users preferred quickto-solve tasks in close proximity, which required minimal effort. This is backed up by the fact that crowd workers in the study tended to choose tasks, which could be solved through physical interaction, e.g., taking a photo. The contributions of this paper are as follows: (1) We introduce the architecture and implementation of a prototype system, which supports mobile crowdsourcing based on location information. (2) In a qualitative user study among 18 participants we explore novel aspects of crowd working and how location-awareness may facilitate and impact on the crowd working process. (3) We present design guidelines, helping developers of context-aware crowdsourcing applications to enhance functionality and uptake among potential crowd workers.

In contrast, implicit user-generated content describes content that is generated by implicit human computer interaction [16]. A prime example is a news website that provides a category “most popular articles” or an online shop with a top 10 of sold articles. Here users generate content (in these cases recommendations) by their actions (reading, downloading, and buying). What is interesting with regard to implicit user-generated content is that there is no extra effort required for the user in order to contribute this content, nevertheless there might be a cost associated (e.g., the loss of privacy). Looking to mobile technologies an example of implicitly generated content is the route someone takes from one location to another. If this is tracked, then the data can become a resource for others, as evident in OpenStreetMap 2. While collecting the tracks happens implicitly, the post-processing is an explicit wiki-based creation of street maps with meta-information. In this case, both an implicit and an explicit approach are used. For us these examples highlight the power of mobile crowds combined

This paper is structured as follows: we start with a brief overview of related works before presenting our concept in more detail. Then we explain the implementation and architecture of our prototype system. We describe the evaluation and report the results of our user study before finally outlining and discussing our design suggestions. BACKGROUND AND RELATED WORK

The Internet has become an essential platform for seeking and sharing information, communication, presentation, and collaborating for many users. This is facilitated by many applications, platforms, and services that are provided on the Internet. For many of these systems it is essential that Web users actively participate in generating content and providing services. Such applications and platforms that

1

www.librivox.org (accessed January 2010)

2

14

www.openstreetmap.org (accessed January 2010)

Full Papers

Proceedings: NordiCHI 2010, October 16–20, 2010

with the WWW to create new resources. Commercial systems like HD-traffic3 information use coarse location data from cell phones to provide high quality traffic information.

difficult to solve by computers and easily accomplished by humans. Examples of such tasks are the tagging of images, e.g., images of garments for an online catalog. Here the aim is to get a representative set of keywords so that users can find what they are looking for. Other domains are natural language processing, summarization, and translation. There is no clear limitation to what type of tasks can be solved through crowdsourcing, as long as they can be described in the system and the answer can be provided over the Internet. In most cases small amounts of money as compensation are provided to the users.

In our work we have looked at different types of usergenerated content and aimed at designing the platform to include as many types as possible. We see many crowdsourcing tasks as explicit requests for specific usergenerated content. Games and Content Generation There are several examples where games are successfully

Crowdsourcing on the World Wide Web

used to create content. Von Ahn et al. have shown that labeling images can be packed and provided to the users in a playful way [19] [18]. A side effect of playing the game is then the assignment of tags and labels to images. In this approach the game itself is already the incentive for contributing the content. We can imagine that it is feasible to create mobile games where users on the move through physical space would create meaningful and valuable information. An example could be running from location A to B with the condition to cross as few roads as possible.

Currently there are several websites available that are based on the concept of crowdsourcing. Amazon’s Mechanical Turk4 is a web-based marketplace for works requiring human intelligence in which anybody can post their tasks and specify prices for completing them. iStockPhoto5 is a webbased company offering huge collections of images uploaded and sold by photographers. Clients seeking stock images purchase credits and start buying the stock images they want. Another example is Innocentive6, which allows companies with specific R&D needs to share their challenges and specify awards among scientists dispersed all over the world. The solvers can submit their solutions through the Web, which go under review by the seeker. Also CambrainHouse7, built on crowdsourcing foundations, collects, filters, and develops the software ideas coming from the crowds. Artists or anyone with spare creativity can submit their T-shirt designs in Threadsless8, a clothing company collecting votes from the community and producing the top rated designs. In [7] it is explained how the power of Web 2.0 technologies and crowdsourcing approach are used to create new approaches to collecting, mapping, and sharing geocoded data.

Another popular type of game is Geocaching [13] [5]. Many of the motivations are not conflicting with the idea of exploiting the information people create while looking for Geocaches. In many cases players already provide photos of the location where they found the cache. This also coincides with our results discussed later where people favored photo-taking tasks. Smart Mobs and Ridesharing

Content generation and crowdsourcing tasks are so far restricted to the digital domain. From our perspective coordinated actions in the physical world such as Smart Mobs [15] or ride sharing supported by digital technologies hint a further direction of location and context-based crowdsourcing. The idea of a flash mob is that people use digital technologies and coordinate an action. If the action has a clear goal this is then considered as a smart mob. By bringing a number of people at a specific point in time to a certain location a political statement can be made, a street can be blocked, or an advertising campaign can be started.

Furthermore, there are researches that investigated various features of crowdsourcing systems. In [2] essential features of a crowdsourcing system and the precise relationship between incentives and participation in such systems are discussed. The authors reported that rewards yield logarithmically diminishing returns with respect to participation levels. In [1] authors studied Google Answer and found out that questions offering more money received longer answers. Yang et al. [20] explored the usage of the site “Taskcn”, a Chinese site where users submit solutions for various tasks and the winner earns a monetary reward. They found out that while new users are choosing unrealis-

With current mobile devices a new generation of ride sharing systems is investigated [8]. We see that ride sharing is essentially a crowdsourcing task in the physical world. It is context dependent (time and location) and may have a number of side conditions (e.g., only travelling with a person with more than 5 years driving experiences). Crowdsourcing

4

www.mturk.com (accessed January 2010)

Crowdsourcing on the WWW has gained popularity over recent years. There are several websites available serving as a platform to distribute crowdsourcing tasks. The characteristic of crowdsourcing tasks is that they are typically

5

www.istockphoto.com (accessed January 2010)

6

www.innocentive.com (accessed January 2010)

7

www.cambrianhouse.com (accessed January 2010)

8

www.threadsless.com (accessed January 2010)

3

www.tomtom.com/hdtraffic/ (accessed January 2010)

15

Full Papers

Proceedings: NordiCHI 2010, October 16–20, 2010

tically, those who are used to the site pursue a more profitable strategy by better balancing the magnitude of the rewards with the likelihood of success. Also Manson and Watts [12] investigated the effect of compensation and performance on Amazon’s Mechanical Turk platform and reported that increasing financial incentives increases the quantity of works done by participants but not necessarily the quality of them. In [11] a crowd translator is demonstrated that collects speech data from the crowd through the mobile phones, which is used to build a high-quality speech recognition system. In the described projects, tasks are location-independent and can be performed on any PC or mobile phone with Internet connectivity. However, there are certain situations where the problems are location-based and physical presence of a person for solving them is required. In our work we focus on location-based problems and on how crowdsourcing can be used to share and solve tasks that are inherently contextual. We fill in the gap between the seekers and a mobile crowd with a location-aware crowdsourcing platform and share tasks based on the solvers’ location. Mobile Crowdsourcing

Various research papers explored crowdsourcing based on the use of mobile phones. Eagle [3] developed txteagle, a mobile crowdsourcing system that enables people to earn small amounts of money by completing simple tasks such as doing translation, transcription, and filling out surveys by using their mobile phones. Askus is a mobile platform for supporting networked actions [10] that allows specifying tasks, which are then matched by the system to specific persons based on profiles. Such profiles may include geographic location. In contrast to our platform, Askus is pushing tasks actively based on a positive match rather than providing an open call addressed to crowd workers present at this location. Fashism9 is an online community that uses phones as a bridge between the physical and digital world. It provides an easy way for customers to get comments on their fashion style while doing shopping by sending a dressing-room photo to the community and getting votes and comments back from the crowds in real time. Google uses crowdsourcing to accumulate the road congestion data and provide the traffic conditions. A user’s phone running Google Maps10 for mobile phones sends bits of data back to Google anonymously, describing how fast he is currently moving. The combination of the data provided by the crowds supply a good overview of live traffic conditions. Ushahidi is an open-source platform from Kenya, which allows for crowdsourcing crisis information by letting par-

9

www.fashism.com (accessed January 2010)

10

www.google.com/mobile/products/maps.html (accessed January 2010)

16

ticipants submit information on violence through text messaging using a mobile phone, email, and the Web [14]. In contrast to these examples we deliberately aim at exploiting the users’ physical location and context. Additionally we investigate how the results of a crowdsourcing task can go beyond the provision of digital content. CROWDSOURCING BEYOND THE DIGITAL

When offering tasks to be solved by crowd workers, the person offering the task (in the following also referred to as the seeker) defines the product or result that is requested. Typically the seeker has little or no information about the potential crowd workers (solvers) that will carry out the task. Hence the description of the task is most critical to achieve the desired result. If tasks are beyond the digital domain and can only be solved in a certain situation it is crucial that the tasks are suggested to seekers who are likely to find themselves in such a situation. The system design needs to include means for matching potential crowd workers in the right location and at the right time with a described task. We believe that certain niches might evolve where crowd sourcing becomes remarkably relevant and successful, especially among certain communities. In the following we describe several scenarios that focus on tasks, which should be performed in a certain context in the real world and away from the desktop. Such tasks are characterized by the need to be performed in a specific location only, or require the presence of a certain context to solve it. Scenario 1: Recommendation on demand John is on his way home from work. On the train he sees an advertisement for a point-and-shoot camera he is interested in. The shop is on his way home but would require a short detour, so he uses mobile crowdsourcing to get information about the availability of the favored camera in this specific store. He is interested in some specific tangible qualities that can be easily assessed while holding the camera in his hands but would be hard to retrieve from a photo. As he trusts the judgment of a fellow customer more than the information he would get from the sales personal, he puts up the task for a crowd worker who just happens to be there. Scenario 2: Recording on demand Mary has a very important lecture today at university. The lecturer will discuss the sample questions for the final exam with the students. In the morning she sprains her ankle and hence she cannot go to university. As she knows that many students will attend the lecture, she puts out the task of recording the lecture into the crowdsourcing platform. She specifies the lecture, time, and location of what she would like to have recorded. A few minutes later she gets a positive feedback from Alex who has taken on the task and has a good reputation (completed many tasks and has been highly rated for them). Later, she receives audio and video files of the lecture as well as copies of Alex’s notes.

Full Papers

Proceedings: NordiCHI 2010, October 16–20, 2010

Scenario 3: Remotely looking around Sarah lives in New York and she is excited about a new job found in Munich. She will be there for six months and an estate agent has provided her some offers. On the Internet she finds the description of an apartment with some photos of its interior. Unfortunately, the offers did not include any photos and further information about the surrounding area. Using mobile crowdsourcing, she specifies a task and asks for more information on the area including photos. A nearby crowd worker who takes the task provides her a couple of pictures (captured with his mobile phone) of the street, some shops as well as a nice café. Scenario 4: Real-time weather information: Bob lives an hour’s drive from some great skiing resorts. As his meeting finishes before noon he decides to take the afternoon off in order to go skiing. He is not sure where to go – on the website all resorts state great conditions and perfect snow. From the webcams he cannot really see many differences. He decides to ask the crowds in his three most favorite places about the skiing conditions and crowdedness. Within a few minutes he gets back information from other skiers that provide him with the information.

Figure 1: System architecture Server

The server side is a PHP-based platform consisting of a MySQL database and several web services used by the web platform and the mobile client. The database includes a collection of all tasks defined in the system through the web platform. Each task is specified by a set of properties, which is then used to distribute it among the crowd. In Table 1 we provide a brief overview of the most important properties, which can be specified for each task. All tasks are associated with geographical information (Location), which is reused in the matching process to assign tasks to workers based on their current location.

Scenario 5: Translations on demand John is in China for two weeks. He decides to buy a very traditional souvenir for his parents. So he goes to downtown and finally finds one. But unfortunately the guys in the store cannot speak English and John does not speak Chinese and so he needs help. With the mobile crowdsourcing platform he searches for someone nearby who can help him in translating English to Chinese.

The Priority field of a task may be assigned to one of the three following priorities: priority 1 (which is the highest one) means that the task is time-critical and solutions have to be submitted within a pre-defined time period. This amount is being specified in the Expired property. Further, tasks with priority 1 are reserved for one solver only. Priority 2 tasks are not time critical (meaning that there is no specified deadline for submitting a solution), but the task is reserved to only one solver. Priority 3 tasks cannot be reserved and users should submit the solution when they sing up for them.

These scenarios show that crowdsourcing in the real world may offer a new quality. All tasks mentioned above are location-dependent and finding a person in close proximity is a crucial prerequisite. However, the tasks differ fundamentally in their time constraints, task duration, and in the way they are solved by the crowd worker. The time constraints range from minutes up to a day and similarly the duration of the tasks from seconds to several minutes. A LOCATION-BASED CROWDSOURCING PLATFORM

Web Platform

To investigate the potential of location-aware crowdsourcing we developed a prototype platform to easily create and distribute tasks among crowd workers. The platform consists of three components: (1) A web interface where seekers can upload arbitrary tasks associated with geographical information, (2) a server including a database for storing the tasks, which is responsible for distributing the tasks to potential crowd workers, and (3) a client application on mobile phones for the crowd workers, which pulls available tasks from the database based on the given location. Figure 1 depicts the system architecture.

To distribute tasks to the crowd we provide an AJAX-based web platform, which on one hand allows seekers to upload arbitrary tasks and distribute them in the community and on the other hand allows solvers to search and download tasks. Hence, we provide two different areas for seekers and solvers. The seekers’ area includes an overview of all tasks they previously specified (both solved and unsolved) where they can easily track their tasks and find out if a task was downloaded and when. We opted not to provide the names of the solvers in the prototype for privacy reasons – however we plan to integrate synonyms in the future to be able to provide a recommendation system. Further, seekers can create new tasks to be solved. All properties of the tasks

By providing such a platform we aim at bringing together all kinds of people regardless of their background, skills, and place of residence. So we are able to virtually offer seekers requesting any task at any time and anywhere.

17

Full Papers

Proceedings: NordiCHI 2010, October 16–20, 2010

Description

A detailed description of the required task sent to the crowd worker

Location

The geographical location (longitude, latitude) for the task, e.g., a shop

Vicinity

Specifies a radius around the task location in km. This indirectly influences the amount of crowd workers receiving this task

Reward

Specifies the incentive if a crowd worker accepts to solve this task

Priority

Specifies the priority of the task

Expired

Allows for specifying a time interval in which a crowd worker has to submit the task solution. This is important for time-critical tasks

Assigned

The time a user accepted to work on the task

Solution

The solution of the task as entered by the crowd worker

Submission

The time the solution was submitted by the crowd worker

have to be provided completely before they are stored in the database and made available to potential solvers. For simplicity, seekers are not required to enter the geographic coordinates of a task but can use a nearby postal address based on which the correct coordinates are calculated. It is also possible that seekers specify the task’s location on Google Maps from where the geographical data are extracted (Figure 2). The solvers’ area includes an overview of all tasks they assigned themselves (both solved and not yet solved tasks). Besides submitting a task via the mobile client, solvers may also enter the solution to their task via the web interface. However this feature was not used during the evaluation. Mobile Client

We developed a JME-based mobile application, which can be used both by seekers and solvers. For solvers, the application lets users retrieve tasks from the database based on given locations. The location can be retrieved either by using an (internal or external) GPS receiver or by providing a postal address (street and zip code/city), which is then converted into geographical coordinates on the server. Additionally a Google Maps view is provided to the users to specify their location and retrieve tasks. This setup allows for retrieving tasks not only based on the current location but also based on other locations, which crowd workers potentially might visit in the near future (e.g., on the way home). Additionally, users can use this feature indoors where GPS devices do not work properly.

Table 1: Task’s properties specified by the seeker

When it comes to working on a task, two different modes of the mobile application can be distinguished: the assignment mode and the solution mode. In the assignment mode the crowd worker can browse and preview available (location-based) tasks based on the given location. Once he decided to work on a task, he may assign himself the task. Based on the Priority property, the task may be unavailable to other workers for the period of time specified in the Expired property. If the user does not submit his task during this period, the task is released and become available again to all crowd workers. It is also possible to assign more than one task at a time (however we limited the amount of parallel tasks to two for the study). In the solution mode, workers can submit their solutions for assigned tasks. In the current version, the submission can be based on texts and pictures. During the design process of the user interface, we focused on keeping the interface simple. Hence, we use a wizard interface to guide users through the different steps of assigning, solving, and submitting the tasks. Figure 3 shows screenshots from the mobile clients.

Figure 2: The web client: seekers can define new tasks in database and use Google Maps to specify the location.

(a)

When using the mobile client as a seeker, the user may create new tasks and publish them to solvers by uploading them onto the server. This allows seekers to create tasks at any time, e.g., while being in transit. The location of the task can be set independent of the current location.

(b)

Figure 3: The mobile client screenshots: (a) Main menu where users can search tasks. (b) A sample task retrieved from the database.

18

Full Papers

Proceedings: NordiCHI 2010, October 16–20, 2010

USER STUDY

To gather data on the users’ crowdsourcing behavior, we conducted two user studies. To obtain reliable results we wanted users to act in their natural environment. Hence, we opted to do field studies in a familiar setting that is at home, at work, and in the city center of users’ hometowns. Initial Study

Due to the complex setup und amount of influencing factors, we decided to run a pre-study for gaining initial insight in potential challenges as well as a better understanding of the users’ crowdsourcing behavior.

Photo Task

Informative Task

Action Task

Take a photo of the closest mailbox

Check the price for a 8GB iPod Nano in the nearby electronic store

Send an email to the user study instructor telling him how many bottles of wine you have at home

Take a photo of your refrigerator

Check how many of the laptops on sale are left at the discounter market

Buy a small bottle of coke from the store around the corner

Take a photo of the coffee machine

Check the departure time of the next bus

Bring a cup of coffee to your colleague next door

Take a photo of your desktop

Count the number of colleagues currently at their desks

Call the office of the user study instructor

Take a photo of the central post office

Check the opening hours of the Apple store

Buy some milk chocolate

Home

Demographics

For the study we recruited nine volunteers via mailing lists, from our courses, and friends. In total, four males and five females participated in the study. Participants were employees (3), students (4), and interns (2), with an average age of 27 years. Prerequisites to participate were that the subjects went to office or the university campus minimum once per working day, and had to own a mobile phone.

Office

Study Setup

To simulate location-based crowdsourcing we asked the participants during the recruiting process to provide us their home and office address, which we used to define different tasks in the database. We divided the tasks into two different dimensions: location and type of tasks. For the location we had (1) tasks at/nearby the users’ home location, (2) tasks at/nearby the users’ office locations, and (3) tasks in the city center of their hometown. For the types of tasks, we distinguished between (1) photo tasks, which required using the phone’s camera, (2) informative tasks, which required the user to type in some information into the application, and (3) action tasks, which required the user to perform a certain kind of action (see Table 2).

City center

Table 2: A task table for one participant on the 3rd day

groups changed the retrieving mode, which could be verified based on the log data. After the two-week study, participants were asked to fill in a final questionnaire. Results

The experience we gathered during the initial study showed that location-based crowdsourcing has potential for many application areas. From 50 tasks in the database 30 have been accomplished in the end. Based on the study, the questionnaire, and the analysis of our log data we extracted the following key findings:

For each participant we then created two different tasks per location and per task type, resulting in an initial set of 6 tasks per user (54 altogether). Then we “refilled” the task list daily based on the tasks the participants opted to complete. There was no restriction for the participants solving tasks and they were free to solve as many as they wanted. We assigned geographical coordinates and a vicinity of 2km to each task based on the location where we wanted the task to be solved. Table 2 shows a sample set of tasks for one participant for a day. Based on the current location maximum just six tasks were visible at the same time.

Users prefer address-based task selection: Our questionnaire revealed that more than three quarters of the participants preferred to retrieve tasks using postal addresses. There are three potential reasons for this: First, both connecting an external as well as activating an internal GPS device puts a burden that many users are not willing to accept. Second, users were often indoors or in public transportations when selecting tasks using the GPS receiver does not work properly. Third, tasks were mainly situated in places people lived in and were familiar with, which made address-based selection an easier option.

For those participants who had a mobile phone equipped with GPS and compatible with our application we helped them with the installation and testing. The other participants were provided with Nokia N73 phones where we had preinstalled the application. Since our application allows for extracting tasks based on both geo-graphical coordinates and postal addresses, we divided the participants into two groups. We asked the first group to only retrieve tasks using the GPS receiver in the first week whereas the other group started with address-based selection. After one week,

Picture tasks are most popular: Interestingly taking pictures was the most popular task among the participants. Obviously most participants could easily handle using the camera of the mobile phone and preferred this task against

19

Full Papers

Proceedings: NordiCHI 2010, October 16–20, 2010

more time-consuming informative tasks (due to the need to type in the solution) and the action tasks.

Study Setup

The study ran for a week with a similar setup as the initial study, however we made minor changes. To provide an even more natural environment, we asked the participants to use their own mobile phone. We invited participants to our lab and after explaining the study to them we installed and tested the application on all phones.

Tasks were mainly solved at or close to home: Based on the results from the questionnaire and an analysis of the solved tasks we found out that the favorite location where users worked on their tasks was at or close to their homes (45% of the tasks).

The tasks used over the course of the study were similar to the initial study. However, since we found out that users completed most tasks in close proximity of their daily whereabouts, we asked the participants to provide us a frequent visited address in addition to their home and office addresses, e.g., their parents’ or friends’ addresses to better simulate a real-world scenario. The given addresses were used to define different tasks in the database (see Table 2). Since most of the tasks in the first study were solved after 3pm, we decided to refill the task tables for each participant in the evenings. As compensation we paid each participant 20 Euros.

Tasks are solved after work: When analyzing the times users assigned themselves tasks, we found out that there are peaks in the morning (25% of the tasks were assigned between 6am and 9am, most likely on the way to work), during lunch time (21%), and after 5pm (40%). Interestingly, tasks assigned in the morning were not solved immediately, but mainly after work (81% of all solutions were submitted after 3pm). Thus, we conclude that people tend to download tasks whenever they are free (on the way to work, during breaks, on the way home) and potentially plan to solve the tasks on their way home. This is also supported by the fact that tasks close to home are mainly solved in the study. Further, this might also explain why users favored addressbased selection, since retrieving tasks at locations different from the current one is only possible with address-based selection in our implementation.

In the first study we found out that users were hardly interested in using GPS data for retrieving tasks, hence we disabled this feature for this study and asked all the users to search for tasks by postal address only. Similar to the first study, there was no limitation on solving tasks and they could decide whenever they wanted to use the system. After a week we asked the users to fill in an online questionnaire, which included questions related to the crowdsourcing platform and a System Usability Scale (SUS) test.

Response times vary: When analyzing the response times (the time between assigning a task and submitting the solution), we found that 40% of the tasks were solved within the first 3 hours, 70% within 15 hours, and 90% within 20 hours. This implies that mobile crowdsourcing works very well within a time frame of one day – however for timecritical tasks, further investigation is necessary.

Results

During this study 55 out of 110 tasks we provided in the system beforehand were completed successfully (average per participant = 6.1, SD = 2.4). Based on qualitative and quantitative feedbacks from the questionnaire and the log files we derived the following results (results are based on a 5-Point Likert scale, 1 = don’t agree at all, 5 = fully agree; given percentages are based on ratings of 4 or 5):

Second Field Study

Based on the result of the pilot study where we mainly gathered qualitative feedback, we conducted a second field study with 9 participants, aiming at providing empirical evidence for our findings. The participants were recruited from mailing lists and none had participated in the first study. The participants used their own mobile phones and SIM cards. Their phones had an integrated camera (hence all participants could solve picture tasks), and we made sure that the application was compatible with each phone.

Informative tasks are as popular as Picture tasks: from all accomplished tasks, 23 were Picture tasks, 21 were Informative tasks, and 11 were Action tasks. The popularity of those two types of tasks is also verified by the participants’ answers in the questionnaire: 89% of all users agreed or fully agreed to prefer Picture tasks, 89% answered to prefer the Informative task, and 11% to prefer Action tasks. This shows that Informative tasks were equally interesting for the users even though they had to enter texts as solutions making those tasks more complex than Picture tasks.

Demographics

The participants of the second study were three females and six males with an average age of 26 years. Six participants were university students with various majors (computer engineering, economics, applied informatics, education science) and the other three were employees. Further, five participants had surfed the WWW via their mobile phones before and five had at least once installed an application on their mobile phones. Only one of the participants had experience with crowdsourcing platforms (Amazon Mechanical Turk).

Time-critical tasks are out of interest: from 55 completed tasks, 30 tasks had priority 3, 20 tasks had priority 2, and just one task had priority 1. This indicates that solving time-critical problems through the location-based crowdsourcing platform cannot be achieved easily because crowd workers prefer tasks without temporal constraints.

20

Full Papers

Proceedings: NordiCHI 2010, October 16–20, 2010

Solution should be achievable in 10 minutes: based on the questionnaire results the preferred amount of effort users were willing to take for solving a task is up to 10 minutes (88%). This also supports the previous finding since Picture and Informative tasks can, in general, be considered to be less time consuming than Action tasks. Put in other words, this is an indicator for time intensive tasks being less popular and it might be hard to find crowd workers for solving such tasks. Tasks are still solved after work: when it comes to the time the tasks’ solutions were submitted, we realized that 64% of the tasks were solved after 1pm (most likely after work). Additionally, based on the questionnaires’ results, 55% of the participants preferred to use the system in the afternoon and 11% at night (see Figure 4).

Figure 4: Tasks submitted over the course of a day. Challenges

We discovered the following challenges:

Midday breaks are good times to search for task: based on the questionnaire, 45% users tended to search for tasks during midday breaks such as lunchtime or between the lectures and 33% on the way home.

Users prefer to retrieve tasks by themselves: Users are willing to enter locations by themselves and having control over addresses used while interacting with the system. One main reason can be privacy issues, which is an important aspect in location-based systems.

Solving a task can take up to one day: the analysis of the response time (the time between assigning a task and submitting the solution) revealed that 57% of the tasks were solved within 10 minutes, 66% within two hours, and 95% within one day. This supports the finding of the initial study that mobile crowdsourcing works well within a maximum time frame of one day.

Provide means for easily choosing the task location: In many cases the locations where users assign themselves tasks are not necessarily the places where they solve them. Hence, an application should enable the user to choose any location, preferably close to their home. Here it might be helpful to provide a map of the surrounding area where users could easily click on a location. Additionally, the frequent locations users visit (e.g., parents’ or friends’ home, clubs...) have potential for distributing tasks. Hence, history of locations where users used the system and searched for tasks should be taken into account for dealing out tasks. Being able to assign oneself tasks over the course of a day seems to be a helpful feature with good uptake.

Home and surrounding areas are the most favorite places for solving tasks: interestingly, based on the results 66% of the accomplished tasks were retrieved at home and 61% of the solutions were submitted at home. Based on the questionnaire, 77% of the participants tend to solve tasks at home or close to it, 55% around the locations they visited frequently (e.g., downtown, clubs), and 44% around the location they daily went to for shopping.

Provide means for specifying priorities: Users prefer working on tasks in the after hours, although they tend to search for tasks during the midday breaks or on the way home. Hence, this is where the response time is likely to be short. However this means that seekers setting up tasks in the morning might have to wait the entire day for a result. We suggest using priorities and timer mechanisms for timecritical tasks.

Voluntary tasks have lower chance: the questionnaire revealed that 77% of the participants had done the task just because of the money, only 22% did it for having fun. Users search for tasks in their current location: we asked users if they searched for tasks in their current locations or locations which they plan to visit during a day. Based on the results, 88% wanted to search tasks in their current location and 22% also wanted to search tasks in locations where they are going to visit during the day.

Minimal effort for the crowd worker: We realized in the user studies that tasks requiring minimal efforts are in favor among users. Users mostly want to download a task and solve it afterwards and they tend to send up to 10 minutes to solve a task. Taking pictures was most popular, most likely due to the fact that no typing was required. Nevertheless, Informative tasks were also in the users’ favor, since they needed to type very short text. The same might be true for audio and video recording, when the application’s interface provides an easy way of using those features. Hence, it is a good strategy to break up tasks into minimal parts with short solutions.

The SUS score from the survey was 79.44, which indicates that users were comfortable with the application. DISCUSSION

Based on the user studies and the survey, the findings indicate that the design choices made for the types of tasks as well as for the means and time of delivery will impact how well location-based crowdsourcing will work. One shortcoming of the study was that tasks were not user-generated but self-designed. Yet, we envision only a minor impact on the solver’s behavior even for different tasks.

21

Full Papers

Proceedings: NordiCHI 2010, October 16–20, 2010

Privacy

[3] Eagle, N.: txteagle: Mobile Crowdsourcing. In Proc. of

Working on location-based tasks raises severe privacy concerns, since from both the location where a user assigns himself a task (e.g., an address) as well as from the task description (e.g., check the opening hours of the post office) the current location of the user can be derived. However, this is not an implementation issue but rather a conceptual problem, which cannot be entirely solved on the system side. Possible options are not to associate the solution submission-time with a task (which only allows to derive an interval in which a user was at a certain location) and to provide users a way to manually enter their current location in an abstract way, such as a zip code or an address in the proximity.

Human-Computer Interaction International (HCII), vol. 5623, pp.447-456, San Diego, CA, July 2009. [4] Fuxman, A., Tsaparas, P., Achan, K., and Agrawal, R.

2008. Using the wisdom of the crowds for keyword generation. Proc. of WWW, 2008, 61-70. [5] Gram-Hansen, L. B. 2009. Geocaching in a persuasive

perspective. Proc. Persuasive '09, vol. 350. [6] Howe, J. Crowdsourcing: Why the power of the crowd

is driving the future of business. Crown Business, 1st Edition August, 2008 [7] Hudson-Smith, A., Batty, M., et al. Mapping for the

Masses: Accessing Web 2.0 Through Crowdsourcing. Social Science Computer Review 27, 4 (2009), 524.

CONCLUSION

In this paper we investigated how crowdsourcing can be extended beyond the digital domain. Based on a discussion of different approaches for content generation, that is explicitly and implicitly, we have proposed an approach for location-based crowdsourcing. To explore the opportunities we created a prototype for location-based mobile crowdsourcing consisting of a web and a mobile client. Through these clients, people of the crowd (solvers) can search for tasks and submit solutions that have a link to the real world.

[8] Kamar, E. and Horvitz, E. Collaboration and shared

plans in the open world: studies of ridesharing. Proc. AI 2009 [9] Kittur, A. and Kraut, R. E. 2008. Harnessing the wis-

dom of crowds in wikipedia: quality through coordination. Proc. CSCW '08, 37-46. [10] Konomi, S., Thepvilojana, N., Suzuki, R., Pirttikangas,

S., Sezaki, K., Tobe, Y.: Askus Amplifying Mobile Actions. Proc. Pervasive 2009, pp. 202-219.

We evaluated the system in two field studies with 18 participants. The results show the feasibility of location-based crowdsourcing and highlight important aspects. In the discussion we addressed discovered issues and presented recommendations for design and improvement of a mobile crowdsourcing platform. When designing location based crowdsourcing systems and mechanisms for distributing tasks among the crowd the following aspects are crucial for its success and should be supported by the system: chosen location (at best close to the crowd workers home), the type of task (most favorite tasks are taking photos), and the time of day (preferably after work).

[11] Ledlie, J., et al. Crowd Translator: On Building Local-

ized Speech Recognizers through Micropayments. [12] Mason, W. and Watts, D. J. 2009. Financial incentives

and the "performance of crowds". Proc. HCOMP '09, [13] O'Hara, K. 2008. Understanding geocaching practices

and motivations. Proc. CHI '08, 1177-1186. [14] Okolloh, O.: Ushahidi, or'testimony': Web 2.0 tools for

crowdsourcing crisis information, published in Participatory Learning and Action, No. 59, 2009 [15] Rheingold H, 2002 Smart Mobs. The Next Social Rev-

olution (Basic Books, Cambridge, MA)

As a future work we plan to enhance our applications with video and audio features. Further, it might be interesting to take additional types of users’ context into account. We believe that “targeting” tasks might increase the potential of crowdsourcing if the client application is able to learn, e.g., routes the user takes as well as types of tasks and locations he prefers or frequently visit. Finally, we plan to explore how the crowdsourcing application impacts on the uptake and the user behavior among communities (e.g., people might agree to solve tasks without incentives, or provide higher quality results).

[16] Schmidt, A. 2000. Implicit human computer interac-

tion through context. Personal and Ubiquitous Computing. 4 (2) Springer. June 2000. pp 191-199. [17] Surowiecki, J. The Wisdom of Crowds: Why the Many

are Smarter than the Few and How Collective Wisdom Shapes Business, Economies, Societies, and Nations., Doubleday [18] von Ahn, L. and Dabbish, L. 2004. Labeling images

with a computer game. Proc. CHI '04. 319-326. [19] von Ahn, L., Liu, R., and Blum, M. 2006. Peekaboom:

REFERENCES

a game for locating objects in images. Proc. CHI '06

[1] Chen, Y., Ho, T.H., and Kim, Y.M. Knowledge market

[20] Yang, J., Adamic, L.A., and Ackerman, M.S. Crowd-

design: A field experiment at Google Answers. Working Paper, November 2008.

sourcing and knowledge sharing: Strategic user behavior on Taskcn. In Proc. of Electronic commerce,

[2] DiPalantino, D., Vojnovic, M. Crowdsourcing and all-

pay auctions. Proc. of Electronic commerce, 2009

22

Full Papers

Proceedings: NordiCHI 2010, October 16–20, 2010

Analysis of Precedent Designs: Competitive Analysis Meets Genre Analysis Mattias Arvola Department of Computer and Information Science Linköping University SE-581 83 Linköping [email protected] +46 13 285626

Jonas Lundberg Department of Science and Technology Linköping University SE-581 83 Linköping [email protected] +46 11 363452

ABSTRACT

Stefan Holmlid Department of Computer and Information Science Linköping University SE-581 83 Linköping [email protected] +46 13 285633

the landscape of other offerings. Numerous papers have reported on using competitive analysis in interaction design, but none of them provide any details on procedures of analysis [2, 3, 7, 9, 10, 16, 24, 25, 38, 44]. This paper will present two procedures for conducting analysis of precedent designs. The first procedure builds on a method presented by Brown [8] and the second procedure builds on genre analysis as it has been conceived in literature and film studies [26, 45]. The procedures are then put to test with students, and finally further developments of them are discussed.

Designers need to survey the competition and analyze precedent designs, but methods for that purpose have not been evaluated in earlier research. This paper makes a comparative evaluation between competitive analysis and genre analysis. A randomized between-group experiment was conducted where graphic design students were conducted one of the two analysis methods. There were 13 students in one group and 16 in the other. The results show that genre analysis produced more detailed descriptions of precedent designs, but its process was more difficult to understand. It is concluded that genre analysis can be integrated into competitive analysis, to make use of the strengths of both methods in the analysis of precedents.

Competitive Analysis

Brown [8] has given a practical account of how competitive analysis can be conducted in web design. He uses the example of pet-related websites that can be compared on their range of products, the design of search boxes, navigation, contact information, and shopping functions. The basic idea of competitive analysis is to line up competitors side-by-side and highlight similarities and differences on selected points of comparisons. This will disclose expectations from users who are used to other sites, and best practices in everything from interface design to offered features. Inconsistencies may indicate that some sites have innovative solutions, or that industry has not settled on any single approach. The result from competitive analysis is a strategy document that helps define the general direction for design without defining the design itself. Surveying the competition is, according to Brown, a good way to get ideas and establish a baseline.

Author Keywords

Analysis of precedents, Genre analysis, Competitive analysis, Interaction design. ACM Classification Keywords

H5.m. Information interfaces and presentation (e.g., HCI): Miscellaneous. INTRODUCTION

It is well known that designers learn from existing examples and precedent designs by surveying competing designs and seeking inspiration [23, 41, 46, 47, 48]. Experienced designers can also abstract from the specific cases to see design principles, patterns and schemes that are instantiated repeatedly [4, 5, 11, 27, 31]. Competitive analysis helps designers position their design in relation to

Procedure for Competitive Analysis

A competitive analysis can be broad and give a feel for the product landscape and identify best practices, or it can be narrow and identify how competitors have solved a particular design problem. The following steps are involved in a competitive analysis according to Brown: (1) Identify and describe the purpose of the analysis; (2) set up the competitive framework; (3) gather and present data; and (4) document the conclusions.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. NordiCHI 2010, October 16–20, 2010, Reykjavik, Iceland. Copyright 2010 ACM ISBN: 978-1-60558-934-3...$5.00.

23

Full Papers

Proceedings: NordiCHI 2010, October 16–20, 2010

1. Identify and describe the purpose of the analysis. A competitive analysis basically consists of two dimensions: a number of competitors and a set of criteria for comparison. This can often be visualized using a simple table where the competitors can be compared side-by-side and similarities and differences be highlighted. The difficult part of a competitive analysis is deciding what set of competitors to compare and what criteria of comparison to use. It is good to be clear on the purpose of the analysis to facilitate these decisions.

way is to score the competitors on every criterion, much like a restaurant review based on criteria like the quality of the food, the ambience, the service, and the expense. For such comparisons, you need to give an account of what, for instance, a high score on the service means. Descriptions is the most common form of data in competitive analyses and they can be explicit on how competitors stack up against each other, without risking potentially skewed numbers. 4. Document the conclusions. Data needs to be interpreted in relation to the client and/or the project at hand. The competitive analysis should project a direction for the future design work by stipulating conclusions based on data. Referring to clear design problems and a narrow analysis make it easier to draw explicit conclusions. If the purpose instead is to give a broad analysis and an overview of the product landscape, the analysis should provide a handful of consequences for design.

2. Set up the competitive framework. The framework, which consists of the two dimensions competitors and criteria for comparison, is often set up in a table. The competitors run along the top row and the criteria along the leftmost column. The criteria for comparison may be specific or general. Specific criteria for an online shop could be “Additional Item Categories”, “Account Management Links”, and “Offline Shopping Links”. A more general criterion for comparison could be “Home Page Navigation”. Another way to set up a competitive framework is to use a two-by-two, where competitors are placed along two dimensions. The criteria for comparison tends in such frameworks to be general since the number of criteria only are two. This type of visualization can be useful for identifying gaps in the product landscape that your product can fill. A third way to do a framework is to make a series of small images of the competitors that, for example, visualize their page layouts.

Genre Analysis

We will in this paper connect competitive analysis to the notion of genres. In, for example, literature and film studies it is common to use genre analysis in comparative analyses. We believe that this also may be a worthwhile approach in competitive analysis for interaction design. A common coding scheme in genre analysis consists of three points of comparison: form, content and purpose [45]. Think of a telephone bill, which can be described according to its layout (the form) that displays the phone calls and their respective cost (the content), to inform the recipient of how much to pay, when to pay, and what exactly it is that he or she is paying for (the purpose).

Very deep or very broad analyses risk becoming difficult to read. There are two ways to manage this: one way is to structure the document by criteria for comparison. This provides a collection of smaller competitions. You may, for example, compare a handful of sites focusing five groups of criteria for comparison: home page design, interior page design, search functionality, features offered, and navigation. This kind of organization makes it easy to identify the best competitors for a specific criterion and across all criteria, but it makes it difficult to get an overall picture of the user experience for each competitor.

A genre analysis describes the common features and differences between many similar objects, and sometimes how they change over time. Genre theory suggests that recurring forms of communication (e.g. the telephone bill) and interaction (e.g. how the telephone bill usually is delivered and paid) help participants understand the purpose of the interaction. The recurring and familiar forms also create expectations on form, content, and purpose of the interaction, and on the actors involved (e.g. for the telephone bill, that the sender is actually the telephone company and that the recipient is the one responsible for paying) [12].

The other way to manage larger analyses is to structure the document by competitor. That means there is one section for each competitor which gives each and one of them a profile. Every profile describes the criteria for comparison. This makes it easy to get an overview picture of the user experience for each competitor, but it makes it harder to compare the competitors with each other.

Genres are central to human cognition, and are based on the human ability to generalize common aspects of recurring situations and objects [35, 36, 37]. For people in everyday situations, genres help to find more of something one has experienced before, and to recognize what kind of situation one is in, for instance an argumentation or a negotiation [1, 12]. It is though, not obvious that more of the same means good design. For a design to be, for example, enchanting it must also evoke a sense of wonder at the newness and unexpectedness of the design [33]. Competitive analysis and genre analysis do, however, not drive design. They only provide a benchmark or basic understanding of the design domain at hand.

3. Gather and present data. When the framework is completed the analyst fills it with data from the analysis. The purpose of the analysis is to describe similarities and differences between products, and it is therefore important to find ways of highlighting them in the presentation of data. The data can be represented in different ways, with different fidelity. Yes/No values are good for comparing if competitors have or does not have a set of features. The drawback is that more subtle differences are lost. Another

24

Full Papers

Proceedings: NordiCHI 2010, October 16–20, 2010

For different genres of communication or interaction, people have expectations on the purpose, on the actors involved, and on the form of the genre object. If the form changes, for instance, by someone starting to fight during a negotiation, or by using inappropriate language, the deviation from the genre of negotiations re-defines the situation. Also, if the actors are seen as inappropriate, for instance a newspaper that apparently is not produced by a newspaper agency, then that deviation from the genre norm also re-defines the situation [12]. People draw upon genre rules, to re-create new situations according to their experiences and expectations [49]. Deviations from the norm can also change the genre, by altering the rules. Thus, genre is not just about repetition, but also about changes. Changes to a genre can divide it into sub-genres, appropriate for somewhat different purposes. Genres analysis can accordingly be used to describe and categorize phenomena into genres, sub-genres, and mix-genres, and to analyze what genre is appropriate for different ends. This has for instance been done for Swedish municipality websites [15]. Genre analysis can also be used to describe changes to a genre over time, which for instance has been done on cyber genres [43], on memos [49], and on web design styles [40]. Some designs are more generic, or more central to the genre, which has been shown in an analysis of personal homepages [13], and cultural aspects have also in earlier research been used as a basis for comparison [42]. Genre analysis has also been used to understand why some designs are better than other designs, by comparing the most successful genre variations to the less successful genre variations. Such analyses have been done on academic texts [45] and on online newspapers [21, 22].

of interaction design. In particular, when discussing aesthetics of interactive products and services we have to include not only characteristics of the artefact and user experiences, but also the characteristics of the interaction that takes place between artefact and user, and between users mediated by the artefact [28]. Procedure for Genre Analysis

The following description of how to conduct a genre analysis builds on our earlier work [14, 15, 19, 21, 22, 29, 30, 6], but is in this paper more comprehensively described. The following steps are involved: (1) Map out the contents of the products, what elements the contents consist of, and what the elements consist of; (2) identify the purposes of the whole products and the purposes of the elements; (3) describe the form of the whole product and of its elements; and (4) identify shared characteristics and differences between the products. 1. Map out the contents of the products, what elements the contents consist of, and what the elements consist of. The first step is to identify the most common elements of a product in the genre. This will provide a repertoire of design elements that can be used to compose genre typical products and to pinpoint characteristics that are unique to some products. Every design element can in turn be described in terms of its elements. This hierarchical breakdown will end at some level and the atoms of the design elements are, for example, lines, textures or specific behaviors. The contents of a product also have a structure: There are different ways to label elements and different ways to organize them in different products. It may also be relevant to analyze what design elements that appear together.

On a general level, different genre analyses largely share the same coding scheme, using form, content and purpose as points of comparison. There are, however, variations. Devitt [12], for instance, describes the characteristics of genres using a coding scheme consisting of a) characteristics of the communicators, b) situation with social and personal motives, c) form and content of communication. Swales [45] uses purpose, content, positioning, and form. Although some analysts use these abstract schemes, it is common to be more detailed. For instance, more detailed points of comparison for film analysis could be setting, characters, narrative, iconography and style [26]. For computer games a detailed set of points of comparison could instead be visual style, cut scenes, interface metaphor, pace, and control schemes [50]. Analyses made in information systems often concern digital media, its form in terms of positioning on screens, its contents in terms of services and information, and the purposes that the information system can fill for different actors [20, 32].

2. Identify the purposes of the whole product and the purposes of the elements. The next step is to find what purpose the genre as a whole fulfils, if particular products has partly other purposes (and may form sub-genres), and what different design elements are used for. You may speculate about the purpose, but the analysis needs to be complemented by interviews or surveys with users and producers in order to be really trustworthy. Such studies may show how important every element is to fulfill different stakeholders’ overarching motive. The composition of form, content and purpose of a product must be in line with the context in which it will be used. This is particularly important, when it comes to different stakeholders’ and users’ motives and experiences. Motives of different actors should match the purpose of the genre in order for the genre chosen to be appropriate. To give an example, a customer in an online store may want a specific item at a low price, while the owner of the store may want to sell as many items as possible to the same customer. The purpose of the online store should meet both these motives in order to be said to have an appropriate design. Experienced qualities like orientation, satisfaction, confusion or flow in the interaction may make the motives

The points of comparison we use in our approach have been developed over a few years. We have embarked from the notions of form, content and purpose, but have realized that they need to be interpreted somewhat differently in the area

25

Full Papers

Proceedings: NordiCHI 2010, October 16–20, 2010

easier or more difficult to achieve. It can, for example, be difficult for a user to reach the motive ‘shop food quickly’ in a confusing store.

A fifth form aspect is the sound that a product or design element has. Sound catches people’s attention, carries emotional content and provides an ambience.

3. Describe the form of the whole product and of its elements. When discussing form elements in interaction design we need to consider movement, space, time, appearance, texture, and sound [39].

4. Identify shared characteristics and differences between the products. Comparisons between the products are made by placing the analyses side-by-side and searching for shared characteristics and differences in terms of form, content and purpose of the products as a whole and of the design elements. Questions to ask include: What characteristics are necessary for a user to recognize the product as belonging to the genre? What characteristics are necessary for a user to know what to do with a product in this genre? What are the consequences of the variations in the genre for how the products are experienced and how they can meet peoples’ motives in different situations? Are there characteristics that make any product better adapted to various motives, experiences and situations? Are there mixgenres or sub-genres that meet partly different purposes and give partly different experiences? If so, what are the characteristics that differentiate the mix-genres or subgenres?

The first step of describing the form is to identify the positions of the design elements. The first form aspect is therefore space. A virtual space can also be described as a network of nodes where the user navigates using links. The concept of users’ space for action can be used to indicate actions that are close to hand and actions that are further away. Different kinds of spaces (physical, screen based, virtual) can also be mixed and combined. The spatial aspects of the form can be said to form a layout. When the layout has been described, the following step is to describe the second form aspect: movement. All interaction involves movement. It is about the movement of people and products, and how products and their design elements are transformed. Some elements are static, while other can change dynamically independently of the user’s actions. They can be passive, reactive interactive or proactive [17, 18]. Proactive elements perform actions based on predictions about the future. When users interact with an element its state also changes. An example is how the state of a web site changes for each step from page to page that the user make. This movement forms a history of where the user has been and where he or she is headed. The movement can be initiated by the user, the system, or be continuous. Continuous movement requires steering (as for example in a snowboard game where you ride down the slope). Movement is also effective for catching people’s attention.

Research Problem

Our practical experience of using genre analysis in interaction design research and education has indicated to us that it facilitates an awareness of the details in a design. The question for this study is whether this intuition is correct. Competitive analysis in Brown’s version may be equally effective. The aim is also to investigate what benefits and difficulties design students experience in competitive analysis and genre analysis. METHOD

A class of second-year graphic design and communication students (average age 23 years, 32% male and 62% female) were randomized into two groups to investigate if there is any difference in the level of detail between design descriptions produced using competitive analysis and descriptions produced using genre analysis. One group was assigned to do a genre analysis and the other group was assigned to do a competitive analysis. They had recently started on their first course in interaction design, after a year of graphic design studies. They were given written instructions in accordance with the earlier described procedures for the two methods. The assignment was to analyze and compare the start page, and the navigation to a specific movie using four web-TV applications (SVT Play, TV4 Play, TV3 Play and Reuters Video). They were told that they would later use their analysis as a pre-study for the design of a mobile web-TV application. They had one week to complete the assignment.

A third form aspect is the appearance of the product and its design elements. It gives clues to the behavior of the elements and how you interact with them. The appearance structures the interaction by presenting opportunities for interaction (i.e. affordances, [34]): If you look upon a door handle you may perceive that you can pull it or push it depending on your earlier interaction with the world. The presented structure may be static insofar that the same opportunities for interaction always are presented in the same way, or the opportunities for interaction may change continuously. A door handle cannot change its appearance but digital products can change their appearance completely from one moment to the next. Appearance also communicates emotional content. A fourth form aspect is the texture of the product or design element. It includes how a surface feels and looks. Just like appearance it communicates opportunities for interaction and emotional content. Other physical properties like weight also affect experienced qualities [31].

The alternative hypothesis was that there would be a difference in level of detail between the genre analysis and the competitive analysis. The null hypothesis was that there would be no difference. One could argue that a onedirectional hypothesis should be used given the literature

26

Full Papers

Proceedings: NordiCHI 2010, October 16–20, 2010

review and our previous work, but this would increase the risk of a type I error (rejecting a true null hypothesis).

Genre Analysis Benefit

The level of detail was operationalized as the number of observed features, where a feature was defined as a design element, or a description of a design element, alternatively an experience of a design element or the design in its entirety.

Gives interesting details Provides generally applicable criteria

13 completed genre analyses (3 male and 10 female) and 16 competitive analyses (5 male and 11 female) were handed in. Two uncompleted genre analyses were also handed in, but were not included in this study. The Mann-Whitney (Utest), which is a non-parametric test for independent samples, was chosen since the assumptions underlying the t-test (normality and homogeneity of variance) could not be met.

Number of features

Rank 3

102

1

160

7

125

2

168

9

146

4

194

11

147

5

256

16

159

6

271

17

166

8

324

19

177

10

325

20

202

12

370

25

227

13

420

26

229

14

430

27.5

239

15

430

27.5

283

18

576

28

327

21

338

22

351

23

360

24

R1 =

236

R2 =

198

Median for Group 1 =

324

Median for Group 2 =

214.5

Median for both groups combined =

Difficulty

Easy to understand procedure

Difficult to set up criteria

Gives broad picture

Uncertain what you miss when you use the criteria you have set up

their experiences and reflections was held. That discussion was documented by taking notes that subsequently were sorted into categories based on benefits and drawbacks. RESULTS

The median for the genre analyses was 324 features, and for the competitive analyses it was 214,5 features. The collected data is presented in Table 1. The study showed that there was a significant difference, to the advantage of the genre analysis, between the two methods (MannWhitney U(13, 16) = 63, p < 0.05, two tailed).

Rank

138

Difficult to know where to start and what to look for

Benefit

Table 1. Results from group discussion on benefits and difficulties of methods for design descriptions.

Competitive Analysis (Group 2, n2 = 16) Number of features

Unclear procedure

Difficult to know when to stop

The participants were, after the assignment, grouped in pairs or groups of three, to discuss what they had done and seen in their analyses. They were also instructed to discuss benefits and drawbacks of the methods, and what was easy and difficult in using them. Finally, a joint discussion of Genre Analysis (Group 1, n1 = 13)

Difficulty

Competitive Analysis

The group discussion with the participants after the assignment provided insights to strengths and weaknesses of the methods. These results are summarized in Table 2. The participants reported that they were unclear about how they would use the information in their future design of the mobile web-TV application. They thought it would have been easier to focus their description if they had more insight into that project. They were also asked to reflect on what would have happened if they had not been given any procedure and only the assignment to analyze precedents and competitors to get inspiration for their design. They answered that it would not had been as thorough and that there would had been a risk that they would only have clicked and played around. One participant thought that would have been better since he/she then only would collect all the good things and annoyances from different sites to know what to build on and what to avoid. Looking at what the participants actually wrote about in their analyses we can see that the genre analyses were in line with the framework given to them to follow. This means that they reported the elements that the web sites contained, and described their form and purposes. The competitive analyses varied more in the their focus, and could for example highlight more technical aspects and did

269.25

Table 2. Data on the level of detail in design descriptions.

27

Full Papers

Proceedings: NordiCHI 2010, October 16–20, 2010

more often line up features that were either present or not present among the competitors. They were also less exhaustive on form aspects.

4.

This procedure needs to be iteratively developed and resulting descriptions of precedent designs as well as impacts on the design process need to be evaluated. This was a study with second year graphic design students, and future research also needs to test the method with experienced design students and professional designers.

DISCUSSION

The results show that it is likely that genre analysis provides more detailed descriptions than competitive analysis using Brown’s [8] procedure. There were two methodological issues in the study. Firstly, it was difficult to decide what should count as a feature. Secondly, there were repetitions of the same feature in every description. These potential sources for error are, however, assumed to be evenly distributed in the genre analyses and the competitive analyses.

Conclusion

Interaction designers need to conduct analysis of precedent designs, but methods for that purpose have not been evaluated in earlier research. The results of this study indicate that genre analysis produces more detailed descriptions of precedent designs than competitive analysis does, but it is also more difficult to understand. Genre analysis can however be integrated with competitive analysis to make use of the strengths of both methods.

A plausible explanation to the higher level of detail in the genre analysis is that it offers points for comparison in an observation protocol (i.e. form, content and purpose), and that the analysis is hierarchically conducted. Such a framework is completely lacking in the competitive analyses. This means that the product in its entirety, its elements, and the elements of the elements, are analyzed in the genre analysis. This also means that the analysis may continue ad infinitum, which must be considered a risk.

ACKNOWLEDGMENTS

We wish thank the students who participated in this study. The research has been sponsored by Östersjöstiftelsen, and Santa Anna IT Research Institute.

The advantage of the competitive analysis in Browns version is that the instructions are easy to understand, and the importance of the purpose as a delimiter of the analysis is stressed. It can, in a procedure for analysis of precedents, be beneficial to merge the two methods.

REFERENCES

1. Agre, P. E. 1998. Designing genres for new media: Social, economic and political contexts. In CyberSociety 2.0: Revisiting Computer-Mediated Communication and Community, S. G. Jones, Ed. Sage Publications, Thousand Oaks, CA, 69-99.

Procedure for Analysis of Precedents

2. Alben, L. and Rigsby, M. 2000. Envisioning the equarium: strategic design planning for the Monterey Bay aquarium's web site. In Proceedings of the 3rd Conference on Designing interactive Systems: Processes, Practices, Methods, and Techniques (New York City, New York, United States, August 17 - 19, 2000). D. Boyarski and W. A. Kellogg, Eds. DIS '00. ACM, New York, NY, 452-454. DOI= http://doi.acm.org/10.1145/347642.347813

The genre analysis can in fact be integrated with a competitive analysis as in the procedure described below. This will not make the genre analysis any simpler to perform, but it will provide a context and purpose, which can make it easier to delimit the genre analysis. 1.

Identify and describe the purpose of the analysis (step 1 of competitive analysis). Decide also what usage task to analyze to limit your analysis (e.g. ’go from start page to specific movie clip’).

2.

Set up the competitive framework by placing the competing products side by side, and analyzing their form, content, and purpose:

3.

Document the conclusions (step 4 of competitive analysis).

3. Armitage, J. 2003. "And another thing..the current site is in German": the final project in an international digital business consultancy. In Proceedings of the 2003 Conference on Designing For User Experiences (San Francisco, California, June 06 - 07, 2003). DUX '03. ACM, New York, NY, 1-15. DOI= http://doi.acm.org/10.1145/997078.997107

a.

Map out the contents of the products, what elements the contents consist of, and what the elements consist of (step 1 of genre analysis).

b.

Identify the purposes of the product and the purposes of its elements (step 2 of genre analysis).

c.

Describe the form of the whole product and of its elements (step 3 of genre analysis).

d.

Identify shared characteristics and differences between the products (step 4 of genre analysis).

5. Ball, L, Ormerod, T., and Morley, N. 2004. Spontaneous analogising in engineering design: A comparative analysis of experts and novices. Design Studies 25, 5, 495-508.

Gather and present data (step 3 of competitive analysis).

6. Blomkvist, J., and Holmlid, S. 2009. Examples in Service Design: Tools for Communication and Setting

4. Arvola, M. 2006. Interaction Design Patterns for Computers in Sociable Use. International Journal of Computer Applications in Technology, 25 2/3, 128-139.

28

Full Papers

Proceedings: NordiCHI 2010, October 16–20, 2010

for Stories. In Proceedings of the First Nordic Conference on Service Design and Innovation (Oslo, Norway, November 24 - 26, 2009). ServDes ’09. Arkitektur- och designhøgskolen, Oslo, Norway.

16.Forrest, M., Forlizzi, J., and Zimmerman, J. 2008. Driving the family: empowering the family technology lead. In CHI '08 Extended Abstracts on Human Factors in Computing Systems (Florence, Italy, April 05 - 10, 2008). CHI '08. ACM, New York, NY, 2913-2918. DOI= http://doi.acm.org/10.1145/1358628.1358783

7. Brinck, T., Ha, S. S., Pritula, N., Lock, K., Speredelozzi, A., and Monan, M. 2003. Making an iMpact: redesigning a business school Web site around performance metrics. In Proceedings of the 2003 Conference on Designing For User Experiences (San Francisco, California, June 06 - 07, 2003). DUX '03. ACM, New York, NY, 1-15. DOI= http://doi.acm.org/10.1145/997078.997084

17.Holmlid, S. 2000. Adapting Users: Towards a Theory of Use Quality. Doctoral Thesis. Linköping Studies in Science and Technology, Dissertation No. 765. Linköpings universitet, Linköping, Sweden. 18.Holmlid, S. 2002. Selecting the abstract: When you get what you want. In Proceedings of the Ericsson Conference on Usability Engineering (Copenhagen, Denmark, 2002). ECUE ‘02.

8. Brown, D. M. 2007. Communicating Design: Developing Web Site Documentation for Design and Planning. New Riders, Berkeley, CA.

19.Holmlid, S., Arvola, M., and Ampler, F. 2000. Genres and Design Considerations of iTV Cases. In Proceedings of the 1st Nordic Conference on HumanComputer Interaction (Stockholm, Sweden, October 23 - 25, 2000). NordiCHI 2000. STIMDI, Stockholm, Sweden, CD-ROM.

9. Burke, P., Nguyen, S., Sun, P., Evenson, S., Kim, J., Wright, L., Ahmed, N., and Patel, A. 2005. Writing the BoK: designing for the networked learning environment of college students. In Proceedings of the 2005 Conference on Designing For User Experience (San Francisco, California, November 03 - 05, 2005). Designing For User Experiences, vol. 135. AIGA: American Institute of Graphic Arts, New York, NY, 32.

20.Holmlid, S., and Lantz, A. 2006. Developing e-services in a government authority: Different views on design in procurement and system development. Position paper at the NordiCHI 2006 Workshop on User Involvement and Representation in e-Government Projects (Oslo, Norway, October 14 - 18, 2006).

10.Chand, A. and Dey, A. K. 2006. Jadoo: a paper user interface for users unfamiliar with computers. In CHI '06 Extended Abstracts on Human Factors in Computing Systems (Montréal, Québec, Canada, April 22 - 27, 2006). CHI '06. ACM, New York, NY, 1625-1630. DOI= http://doi.acm.org/10.1145/1125451.1125747

21.Ihlström, C. and Lundberg, J. 2003. The Online News Genre through the User Perspective. In Proceedings of the 36th Annual Hawaii international Conference on System Sciences (Hicss'03) - Track 4 - Volume 4 (January 06 - 09, 2003). HICSS. IEEE Computer Society, Washington, DC, 103.1.

11.Cross, N. 2004. Expertise in design: An overview. Design Studies 25, 5, 427-44. 12.Devitt, A. J. 1993. Generalizing about genre: New conceptions of an old concept. College Composition and Communication 44, 4 (Dec, 1993), 573-586.

22.Ihlström, C., and Lundberg, J. 2004. A genre perspective on online newspaper front page design. Journal of Web Engineering 3, 1, 50-74.

13.Dillon, A. and Gushrowski, B. 2000. Genres and the Web: Is the personal home page the first uniquely digital genre?. Journal of the American Society for Information Science, 51, 2, 202-205.

23.Keller, A. I., Pasman, G. J., and Stappers, P. J. 2006. Collections designers keep: Collecting visual material for inspiration and reference. CoDesign 2, 1, 17-33.

14.Edvardsson, A., and Lundin, J. 2003. Fotboll med Internet som spelplan: en metod för stt skapa designmönster för ett digitalt medium [Football with Internet as Game Field: A Method for Creating Design Patterns for a Digital Medium]. Masters Thesis. Department of Computer and Information Science, Linköpings universitet, Linköping, Sweden.

24.Kim, E., Koh, B., Ng, J., and Su, R. 2006. myPyramid: increasing nutritional awareness. In CHI '06 Extended Abstracts on Human Factors in Computing Systems (Montréal, Québec, Canada, April 22 - 27, 2006). CHI '06. ACM, New York, NY, 1843-1848. DOI= http://doi.acm.org/10.1145/1125451.1125800 25.Kim, J. and Zimmerman, J. 2006. Cherish: smart digital photo frames for sharing social narratives at home. In CHI '06 Extended Abstracts on Human Factors in Computing Systems (Montréal, Québec, Canada, April 22 - 27, 2006). CHI '06. ACM, New York, NY, 953958. DOI= http://doi.acm.org/10.1145/1125451.1125635

15.Eliason, E. and Lundberg, J. 2006. The appropriateness of Swedish municipality web site designs. In Proceedings of the 4th Nordic Conference on HumanComputer interaction: Changing Roles (Oslo, Norway, October 14 - 18, 2006). A. Mørch, K. Morgan, T. Bratteteig, G. Ghosh, and D. Svanaes, Eds. NordiCHI '06, vol. 189. ACM, New York, NY, 48-57. DOI= http://doi.acm.org/10.1145/1182475.1182481

26.Lacey, N. 2000. Narrative and Genre: Key concepts in media studies. St. Martin's Press.

29

Full Papers

27.Lawson, B. 1980. How designers Architectural Press, London, UK.

Proceedings: NordiCHI 2010, October 16–20, 2010

think.

New York, NY, 898-903. http://doi.acm.org/10.1145/506443.506649

The

DOI=

39.Saffer, D. 2007. Designing for Interaction: Creating Smart Applications and Clever Devices. New Riders, Berkeley, CA.

28.Lim, Y., Stolterman, E., Jung, H., and Donaldson, J. 2007. Interaction gestalt and the design of aesthetic interactions. In Proceedings of the 2007 Conference on Designing Pleasurable Products and interfaces (Helsinki, Finland, August 22 - 25, 2007). DPPI '07. ACM, New York, NY, 239-254. DOI= http://doi.acm.org/10.1145/1314161.1314183

40.Schmid-Isler, S. 2000. The Language of Digital Genres. A Semiotic Investigation of Style and Iconology on the World Wide Web. In Proceedings of the 33rd Hawaii international Conference on System Sciences-Volume 3 - Volume 3 (January 04 - 07, 2000). HICSS. IEEE Computer Society, Washington, DC, 3012.

29.Lundberg, J. 2004. Shaping online news: Genre perspectives on interaction design. Linköping Studies in Science and Technology Dissertation No. 918. Linköping University, Linköping, Sweden.

41.Schön, D. A. 1987. Educating the Reflective Practitioner. Jossey-Bass Publishers, San Fransisco, CA.

30.Lundberg, J., Arvola, M., and Holmlid, S. 2003. Genres, Use Qualities and Interactive Artifacts. In Proceedings Volume 2 of the Conference HCI 2003: Designing for Society (Bath, UK, September 8 - 12, 2003). HCI 2003. Research Press International on behalf of British HCI Group, Bristol, UK, 165-166.

42.Segelström, F. and Holmlid, S. 2009. Online services and cultural diversity: Applying Hofstede’s framework to predict potential issues. In Proceedings The 11th International Research Symposium on Service Excellence in Management (QUIS) (Wolfsburg, Germany, June 11 - 14, 2009). Print Services, Volkswagen AG, Wolfsburg, 604-612.

31.Löwgren, J. 2007. Inspirational patterns for embodied interaction. Journal of Knowledge, Technology & Policy 20, 3, 165-177.

43.Shepherd, M. and Watters, C. 1998. The Evolution of Cybergenres. In Proceedings of the Thirty-First Annual Hawaii international Conference on System SciencesVolume 2 - Volume 2 (January 06 - 09, 1998). HICSS. IEEE Computer Society, Washington, DC, 97. DOI= http://dx.doi.org/10.1109/HICSS.1998.651688

32.Marriott, K., Moulder, P., and Hurst, N. 2007. Automatic float placement in multi-column documents. In Proceedings of the 2007 ACM Symposium on Document Engineering (Winnipeg, Manitoba, Canada, August 28 - 31, 2007). DocEng '07. ACM, New York, NY, 125-134. DOI= http://doi.acm.org/10.1145/1284420.1284455

44.Sood, S., Gilligan, R., Chandler, C., and Slack, S. 2004. Disneyworld.com redesign. In CHI '04 Extended Abstracts on Human Factors in Computing Systems (Vienna, Austria, April 24 - 29, 2004). CHI '04. ACM, New York, NY, 869-884. DOI= http://doi.acm.org/10.1145/985921.985946

33.McCarthy, J., Wright, P., Wallace, J., and Dearden, A. 2006. The experience of enchantment in human– computer interaction. Personal Ubiquitous Comput. 10, 6 (Sep. 2006), 369-378. DOI= http://dx.doi.org/10.1007/s00779-005-0055-2

45.Swales, J., M. 1990. Genre Analysis: English in academic and research settings. Cambridge University Press, Cambridge, UK.

34.Norman, D. A. 1999. Affordance, conventions, and design. interactions 6, 3 (May. 1999), 38-43. DOI= http://doi.acm.org/10.1145/301153.301168

46.Sökjer, P. 2007. Interaction designers' use of their repertoire in meetings with clients. In Proceedings of the 21st British HCI Group Annual Conference on HCI 2008: People and Computers Xxi: HCI...But Not As We Know It - Volume 2 (University of Lancaster, United Kingdom, September 03 - 07, 2007). British Computer Society Conference on Human-Computer Interaction. British Computer Society, Swindon, UK, 257-258.

35.Paltridge, B. 1995. Working with genre: A pragmatic perspective. Journal of Pragmatics 24, 4, 393-406. 36.Rosch, E. 1973. On the internal structure of perceptual and semantic categories. In Cognitive Development and the Acquisition of Language, T. E. Moore, Ed. Academic Press, New York, NY, 111-144. 37.Rosch, E. 1978. Principles of categorization. In Cognition and Categorization, E. Rosch and B. Lloyd, Eds. Erlbaum, Hillsdale, NJ, 27-48.

47.Sökjer, P., Holmlid, S., Tholander, J., and Lantz, A. 2007. The dynamics of objects in client-designer communication. In Proceedings of The Virtual ‘07, (Almåsa, Sweden, September 20 - 22, 2007). M3, Södertörn University, Haninge, Sweden.

38.Rosenbaum, S., Wilson, C. E., Jokela, T., Rohn, J. A., Smith, T. B., and Vredenburg, K. 2002. Usability in Practice: user experience lifecycle - evolution and revolution. In CHI '02 Extended Abstracts on Human Factors in Computing Systems (Minneapolis, Minnesota, USA, April 20 - 25, 2002). CHI '02. ACM,

48.Winograd, T., Bennett, J., and Young, L. D., Eds. 1996. Bringing Design to Software. Addison-Wesley, Reading, MA.

30

Full Papers

Proceedings: NordiCHI 2010, October 16–20, 2010

Extended Abstracts on Human Factors in Computing Systems (Vienna, Austria, April 24 - 29, 2004). CHI '04. ACM, New York, NY, 773-774. DOI= http://doi.acm.org/10.1145/985921.985929

49.Yates, J., and Orlikowski, W. J. 1992. Genres of organizational communication: A structurational approach to studying communication and media, The Academy of Management Review 17, 2 (Apr. 1992), 299-326. 50.Ye, Z. 2004. Genres as a tool for understanding and analyzing user experience in games. In CHI '04

31

Full Papers

Proceedings: NordiCHI 2010, October 16–20, 2010

Exploring Distance Encodings with a Tactile Display to Convey Turn by Turn Information in Automobiles Amna Asif1 , Wilko Heuten2 , Susanne Boll1 University of Oldenburg Escherweg 2 26121 Oldenburg, Germany {amna.asif—susanne.boll}@informatik.uni oldenburg.de 1

OFFIS Escherweg 2 26121 Oldenburg, Germany [email protected] 2

systems to support the driver. A car navigation system is one of the promoted and preferred information systems for cars. Visual and auditory displays in the present car navigation systems are successful to provide spatial information. However, the car navigation system increases a demand of the driver’s visual and acoustic attention while driving and are subject to distract the driver. The visual attention of the driver is essential in the primary task of driving, which is for example steering the car, using brakes and controls as well as observing the traffic. Pauzie and Marin [10] investigated that aging drivers spent 6.3% and young drivers spent 3.5% of their driving time glancing at the screen. Auditory displays on the other hand are challenging in a noisy environment. The driver performs a multiple number of primary and secondary tasks on visual and auditory displays that can impose mental workload [4] and distraction [12], which is harmful for safety of driving.

ABSTRACT

Visual and auditory displays successfully complement each other presenting information in car navigation systems. However, they distract the visual and auditory attention of the driver, which is needed in many primary driving tasks, such as maneuvering the car or observing the traffic. Tactile interfaces can form an alternative way to display spatial information. The way of how exactly information should be presented in a vibro-tactile way is explored rarely. In this paper we investigate three different designs of vibro-tactile stimulation to convey distance information to the driver using a tactile waist belt. We explore the tactile parameters intensity, rhythm, duration, and body location for encoding the distance information. We conduct a comparative experiment on a real navigation scenario in an urban environment to evaluate our designs. In our study we discovered that rhythm and duration are suitable parameters to generate tactile stimulation for encoding distance information. In this way the driver perceives countable vibro-tactile pulses, which indicate the distance in turn by turn instructions. The approach is found be simple way of encoding complex navigational information.

A tactile display can be used as an alternative interface for the car navigation system to reduce the mental workload and the distraction, following the Multiple Resource Theory (MRT) [21]. Two important parameters of the turn by turn navigation are direction and distance [2, 19]. In previous studies car simulators have been used to evaluate the vibro-tactile distance encoding. However, the investigation of the precise approach of vibro-tactile distance presentation in a real environment is missing. We investigated a number of the encodings based on rhythm, intensity and duration to discover an appropriate approach for the vibro-tactile distance encoding. We evaluated three vibro-tactile distance encodings in an experiment: (1) Only rhythm based distance encoding (2) Rhythm and intensity based distance encoding (3) Rhythm and duration based distance encoding. In our study, we focused on comparing different methods of conveying the distance with the vibro-tactile feedback in real urban environments. The study shows success of duration in combination with rhythm based distance encoding in the car navigation systems.

ACM Classification Keywords

H.5.2 User Interfaces: Haptic I/O; 1.3.6 Methodology and Techniques: Interaction techniques General Terms

Human Factors, Experimentation Author Keywords

Car navigation system, Tactile interface INTRODUCTION AND MOTIVATION

The activity of driving is multi-tasking and complex [12]. Vehicles are equipped with a number of distinct information

In the remainder of this paper, Section 2 introduces the reader with the state-of-the-art tactile interfaces and related approaches of the vibro-tactile distance and direction encoding in automobiles. The design space and the vibro-tactile distance encodings are described in Section 3. The experiment details are presented in Section 4. In Section 5 we report the find-

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. NordiCHI 2010, October 16-20, Reykjavik, Iceland. Copyright 2010 ACM ISBN: 978-1-60558-934-3...$5.00.

32

Full Papers

Proceedings: NordiCHI 2010, October 16–20, 2010

ings. In Section 6 we discuss the answers to our research question and further findings. We close the paper with a conclusion in Section 7.

vibrators in an 8 x 8 matrix [6] is used to evaluate the ability of drivers to distinguish up to eight different directions. The distance information is presented by van Erp and van Veen [17] with vibrocon (vibro-tactile icon) for three distance steps of 250 m, 150 m, and 50 m. The information is presented to the users by activating the four tactile actuators either under the left or right leg of the driver. The results of an evaluation in the simulator show that the tactile interface helps to reduce the visual burden of the drivers. We take the opportunity to explore the tactile encoding for spatial information in the real driving environment. van Erp and van Veen encoded a distance information in the form of rhythm patterns though it is possible to explore more methods of distance encoding. The proof-of-concept study shows that using vibro-tactile displays in the car is useful for presenting distance information.

RELATED WORK

Previous research has shown that the tactile displays were effectively used to provide navigation aids to the pedestrians and blind users. McDaniel et al. [9] present a scheme for using tactile rhythms to convey intimate, personal, social interpersonal distances to blind users. ActiveBelt [16] consists of a number of vibration components integrated into a belt or a vest. The vibration components are equally distributed around the person’s body and activated for showing him a direction. Tactile Wayfinder [5] was evaluated for the task of pedestrian route guidance. The tactile Wayfinder supported the pedestrian in orientation, choice of route, keeping through track, and recognition of destination. PeTaNa [20] a torso based wearable system was evaluated to present direction and distance information to soldiers in the field. The direction information was presented on the respective location of torso of the soldier. Distance was coded with the temporal rhythm on the vibration. van Erp et al. [19] investigated different distance encoding schemes with pedestrian participants. The vibration rhythm was used to code the distance and the body location was used to code the direction. An additional experiment investigated usefulness of tactile display with a helicopter and a fast boat. Straub et al. [14] used a vibro-tactile waist belt to encode distance for the pedestrian. They used four distance encodings based on the parameters of intensity, frequency, position (which tactor), and patterns. Pielot et al. [11] presented a position and spatial distance of several people with the help a tactile torso based display in fast paced 3D game. The results showed that the location of the team members can be effectively processed with the help of the tactile display in high cognitive demand. The team showed a better team play and higher situation awareness. The findings of the previous studies encourage the fact that it is possible to encode distance with tactile displays. The effectiveness of the tactile display for presenting direction and distance information to blind and pedestrians motivate the idea to discover the approach to code distance in vibrotactile signals in car navigation systems.

In the previous research, torso based tactile displays are successfully employed to present navigation information to the blind users and pedestrians but parameters still require exploration in the automobiles. In the previous studies [9, 11, 14, 19], a number of vibro-tactile encoding schemes are compared to display distance information to the team players, pedestrian and blind users. Besides previous proof-ofconcept studies [17] the tactile parameters are still required to explore to discover an appropriate approach of encoding distance with vibro-tactile signals in the car navigation system. In this paper a similar comparative approach is used to explore the tactile parameters to encode distance information in the car navigation system. EXPLORING DISTANCE ENCODING WITH VIBRO-TACTILE SIGNALS

Car navigation systems provide route guidance to the driver towards a destination [15]. In our study, we used a tactile waist belt to provide turn by turn directions and distance information to the driver. Figure 1 presents example cues that Distance Turn-now

Furthermore, the tactile interface is effectively used in Advanced Driver Assistance Systems on commercial scale e.g. in Citroen1 and Audi2 in the seat and steering wheel, respectively. Similarly, the previous studies investigated the feasibility of the tactile channel in vehicle information systems e.g. in car navigation systems. A vibro-tactile seat is developed with the characteristics of a matrix with 6 x 6 elements with interaction area of a 430 by 430 mm with dynamically modifying haptic feedback based on the driving sitting posture [13]. A prototype steering wheel [7] is integrated with six tactile actuators. The steering wheel is used to display the direction in a car simulator. The best driving performance is attained by combining tactile display with a visual display or an auditory display. A seat fitted with 24 1 http://www.citroen.com.hk/tech/sec

Direction Left

Near

Left

Far

Left

Very-far

Left

Figure 1. The direction and distance instruction presented by the tactile feedback, as the car approaches the crossing.

04.htm

2 http://www.audiworld.com/news/05/naias/aaqc/content5.shtml

are conveyed to the driver through the tactile belt while ap-

33

Full Papers

Proceedings: NordiCHI 2010, October 16–20, 2010

proaching the actual turn. In order to convey the direction left, the tactile belt indicates left by activating vibrators on the left side. The vibro-tactile signals provide information about the four categories of distance as proposed by Fukuda [3]: (1) Very-far, (2) Far, (3) Near, and (4) Turn-now. The driver make turning decisions by following the vibro-tactile instructions concerning the direction and distance while approaching the crossing. This section describes our vibrotactile distance encoding in the car navigation system and the practical approach that we adopted to explore the design space.

the study helped us in the design of vibro-tactile distance encodings for a further comparative evaluation. In addition, the pilot study was conducted to provide the proof-of-concept of displaying distance information with only tactile feedback on the real road. Participants, apparatus and Procedure

One female participated in the pilot study. She had 25 years of driving experience. The vibro-tactile signals are used to present distance information to the participant. The participant tested six options of vibro-tactile distance encoding in seven design solutions. We adapted the approach of thinking aloud to collect the participant’s comments and observations. We made videos for data collection. We measured the vibro-tactile signal perception and distance categories.

Vibro-tactile Distance Encoding

There are many potential ways to use vibration for encoding distance. According to [1, 18] the tactile parameters that can be modified to encode information in general are frequency, amplitude, waveform, duration, rhythm and body location. The frequency is the number of cycles per second [8]. Vibration stimuli will be detected when the amplitude exceeds a threshold which is dependent on several parameters including frequency [18]. The parameters of frequency and amplitude are not well suited to encode information [20]. The change in voltages results in change in frequency and amplitude simultaneously. The change is almost linear given in Precision Microdrive3 . So we can treat the frequency and amplitude as one parameter of intensity by following the approach in [11]. Furthermore, we cannot modify the waveform of the signal as its manipulation would require specific hardware [18]. Further parameters for encoding distances are rhythm and duration. The rhythm is created by grouping a number of pulses in a temporal pattern. Varying the duration of the vibro-tactile signal means to vary the length of a single pulse.

The distance signals were controlled by an experimenter on the backseat. The participant was trained on all the designs before going to the driving sessions. The participant drove in multiple sessions. The participant drove at least two times for each design. The participant was speaking whatever she thinks about the vibro-tactile signal perception and categories of distance. The videos were analyzed post-studies to collect the results. Results and discussions

In summary, we found that the following designs are acceptable according to the participant’s comments: rhythm based distance encoding, intensity and rhythm based distance encoding, duration and rhythm based distance encoding. The respective category of distance relative to the quantitative range of distance is given in Table 1. In the following, we will discuss our observations and the driver’s comments of the pilot study in detail.

We can design different vibration patterns by modifying the intensity, rhythm, duration, and the body location. In this study the body location of vibro-tactile cues is already used for presenting direction as the vibro-tactile waist belt uses the left and right vibrators to encode the direction of the upcoming turn. Thus, it cannot be used further for distance encoding.

Categories of distance: In distance encoding, the signal of turn-now is required to be comparatively intense and easy to identify to make sure that the driver has to take turn at that moment. According to observations, the vibration signal for turn-now will be few meters (i.e. 10m) before the next crossing while driving on the street.

In summary, encoding distance by vibration can be conducted by altering the parameters intensity, rhythm, and duration. These parameters can be altered individually or in combination. Following options are available for the adjustment: intensity only, rhythm only, duration of the signal, intensity and rhythm, intensity and duration, and rhythm and duration. In order to validate, whether all of these options were easy to perceive, to distinguish and to interpret by the driver, we investigated them in a first pilot study.

Vibration patterns in automobiles: The driver commented that ”I get strong sensations with less number of pulses, and while increasing the number of pulses feeling-wise it is less intense”. The driver gets smooth sensations with a higher number of pulses in a small time interval (e.g. minimum six pulses per second). The vibration pattern sensed intense and distinct if less pulses take place in a longer time interval (e.g. 2 pulses per second). The participant commented that ”The vibration is not easy to feel and I am undecided between different levels of intensity” for only intensity-based distance encoding. We observed that the participant annoyed by the intensity only vibro-tactile signal. Furthermore, it was difficult for the participant to understand the distance information. We discovered that rhythm is an important parameter for vibro-tactile information encoding in the car navigation system. We discovered that it is possible to get appropriate rhythm by changing a number of pulses while keeping the constant level of the intensity. Consequently, it is difficult to

Pilot study

The overall aim of the pilot study was to explore the usefulness of the different ways of encoding distance information with vibro-tactile signals. We conducted the pilot study to select the possible tactile parameters for encoding the distance information in the real driving scenarios. The results of 3 https://secure.precisionmicrodrives.com/product info.php? products id=92

34

Full Papers

Proceedings: NordiCHI 2010, October 16–20, 2010

Category of Distance Very Far Far Near Turn Now

Rhythm based encoding Rhythm & intensity based encoding

Distance (meters) 200-150 100-80 50-30 10

Rhythm & duration based encoding

Very-far

Far

Table 1. The quantitative categories of four distances.

select among only rhythm, intensity and rhythm, and duration and rhythm based vibro-tactile distance encoding. We concluded that all designs in combination with rhythm are successful for vibro-tactile distance encoding.

Near

Turn-now

Although we conducted this study with only one participant, the observations and the driver’s comments helped us a lot to improve vibro-tactile distance encodings for further comparative experiment. The pilot study provided a proof-ofconcept that the tactile feedback is successful to present the distance and direction information to the driver on the real road without support of visual and auditory modalities.

Figure 2. Rhythm based, rhythm & intensity based, and rhythm & duration based distance encoding consists of different patterns of pulses with different intensity levels.

In the following duration, intensity and rhythm are used to encode distance. The driver recognizes the information in the form of countable pulses, the distinct rhythm patterns. Distance encoding in vibro-tactile cues

We discovered three promising vibro-tactile distance encodings based upon our pilot study and observations. The designs are composed of vibration duration, intensity, and rhythm. In the three designs we encode the categories of distances (see Table 1). The first design utilized the parameter rhythm, the second one utilized the parameters rhythm and intensity, and the third design made use of rhythm and duration. In the first two designs, the driver perceives a very smooth vibration at the distance of very-far and a very intense vibration at the distance of turn-now. The third design – rythm and duration encoding – conveys countable pulses to the driver. The three designs are explained with the help of graphical representation (see Figures 2), where the vertical axis presents the level of intensity and the horizontal axis presents time in milliseconds (ms). Furthermore, the four categories of distance are presented from the top to the bottom in the diagrams. The technical characteristics of the three designs are outlined in the following: Rhythm based distance encoding:

encoded in pulses per second with variable intensity. A pulse is composed of its activation and sleeping states. If we look from top to bottom in the diagram an event of very-far is composed of 25 pulses with 70% of intensity level. On the second place the event of far is composed of 10 pulses with 70% of intensity. On the third place the event of near is composed of 6 pulses with 80% of intensity. And in the bottom the turn-now event consists of 2 pulses with maximum intensity. Rhythm and duration based distance encoding:

The graphical visualization of the duration and rhythm based distance encoding is presented in Figure 2. In this design the pulses are more intense and less in a number. The distance is presented with a number of pulses in the given time along with its intensity level. A pulse is shown with its activation and deactivation states. The first event is composed of 4 pulses of the length of 2.5 seconds. The inter-stimulus interval between the pulses is 313 ms. The second event consists of 3 pulses of the length of 2 seconds. The inter-stimulus interval between the pulses is 333ms. The third event is encoded as 2 pulses of length of 1.5 seconds. The interstimulus interval between the pulses is 375ms. The fourth event is encoded as one pulse in a second with 500 ms interstimulus interval. On the basis of the pilot study, we proposed the three different methods of distance encoding based on: only rhythm, intensity and rhythm and duration and rhythm. In the following section, we present the design of the experiment in which we compared the methods of distance encoding.

The graphical visualization of only rhythm based distance encoding is shown in Figure 2. The encoding of a distance signal is presented with its intensity and length. The event is composed of pulses per second. A pulse is composed of its activation and deactivation states. The intensity is same for all the four events. If we observe from the top to the bottom the design consists of 25 pulses for the very-far. The far, near and turn-now consist of 10 pulses, 6 pulses, and 2 pulses respectively.

EVALUATION

The goal of the study is to compare the different approaches to encode distance information with vibro-tactile signals. We aim to investigate the most simple vibro-tactile distance encoding for the driver that leads to successful task completion. The comparative evaluation investigates the specified questions:

Rhythm and intensity based distance encoding:

The graphical visualization of the intensity and rhythm based distance encoding is presented in Figure 2. The distance is

35

Full Papers

Proceedings: NordiCHI 2010, October 16–20, 2010

Q1: Will the participants perceive turn by turn distance information without support of visual and auditory modalities in the car?

Apparatus

We used a fully automatic Volkswagen Touran car in all sessions. The tactile belt is used to provide navigational aid in all three driving sessions (see Figure 3). The tactile belt is integrated to be worn around the waist of the driver. The fixed circumference of 90 cm is made up of a fabric in the shape of a flexible tube. The tactors are sewed into the fabric tube. These tactors are vibrating components of Samsung SGH A400, size of 11 mm. The six tactors are distributed equally around the waist of the participant. We have used the questionnaires, video recording, and screen captures for the collection of data.

Q2: Which one of the encodings – rhythm, rhythm and intensity, rhythm and duration – is the most usable encoding? Q3: Which distance encoding helps the driver to make the least number of information perception / interpretation errors? Q4: What are common errors that participants make to interpret distance information by using the tactile display? Experiment design

Our experiment consists of one independent variable. The vibro-tactile signal presents the direction and distance information to the participants. We compared the three vibrotactile distance encodings while driving in real urban environments. The approach of the experiment was within subject, with 3 different routes and 3 different vibro-tactile encodings of distance. In the within subject approach the different experimental conditions are the three vibro-tactile distance encodings and the same participants have been used in all the conditions. Dependent measures

Figure 3. The driver wearing the tactile belt.

Considering the questions for the evaluation Q1 to Q4, we selected the measures of usability of the designs, information perception errors and cognitive workload as dependent measures.

Questionnaire

Usability of the designs: We measure usability in the terms of information perception, learnability, length of vibro-tactile signal, ease of use, and user’s judgment rating on the design. We asked the participants to rate the design’s usability on a questionnaire after completing each route.

The questionnaire consists of questions regarding dependent measures like information perception and usability of the designs. The users’ responses are gathered on a 10 point scale. The scale is numbered from 1 to 10 and presents negative to positive response respectively.

Information perception errors: Information perception errors are the number of errors made by the participant in identifying the category of distance. We measure the number of participants’ wrong responses on the categories of distance for quantifying how many errors they made in perceiving the distance information. Furthermore, we analyzed how they interpreted the categories of distance.

Interviews

Distraction: We analyzed the drivers’ responses and videos to measure the distraction.

Screen captures and videos

In the end of the driving sessions, the participants were interviewed. They were asked about their impression about the vibro-tactile distance encodings, all driving sessions, tactile display preference over other displays, distraction and any further comments.

Video recordings are used for the data collection. Considering ethics, we have taken participants’ signatures for their willingness to record their videos. The open source software CamStudio is used for capturing the laptop screen of the experimenter and the voice of the participant. The participants were asked to announce the category of distance every time when they feel vibration. Different categories of distance pointed by experimenter are captured from laptop screen. Afterwards, the participant’s oral response on category of distance is compared with screen captures. A custom built application is used to analyze data from the videos and CamStudio. In the custom built application the keys of keyboard contained meaningful information related to correct

Participants

We evaluated a total of 13 participants, 1 female and 12 male in our experiment. The participants were between 2040 years old. All participants voluntarily participated in the experiment. Each participant completed all three sessions of the experiment. The participants have in the average 12 years of driving experience. We conducted (3*13) driving sessions with all participants in the whole study. We evaluated 3 participants after sunset and 10 in daylight. All participants completed assigned tasks easily in both conditions.

36

Full Papers

Proceedings: NordiCHI 2010, October 16–20, 2010

and incorrect responses of the driver e.g. the participant’s response on very-far as far can be considered as meaningful information. On sensation of the vibration signal the driver guesses the category of distance by thinking aloud. Later, an analyzer watched the screen capturing segment with video. The analyzer presses the keys according to the response of the participant about the information signal. The information produced in the procedure is saved in a text file. The videos are observed two times by the analyzer.

Route-1 Route-2 Route-3

Procedure

The distance signals are controlled by the experimenter. The experimenter is seated next to the driver. During the driving session the four categories of distances are presented to the next crossing. We presented the categories of distance according to the length between two crossings e.g. all four categories of distance are displayed to the participant, if the length (in meters) is in the range of very-far distance (see Table 1). Furthermore, we displayed the 3 categories of distances if the length is in the range of the far distance and so on. We have created random sequences of the designs to balance any learning effect.

Figure 4. Three routes are used for driving in the experiment.

ilarly, we collected additional observations and the participants’ comments regarding the vibro-tactile distance encodings in the qualitative measures. In the following, we will discuss the quantitative results of our evaluation. Quantitative results

Before going to the venue, we explained the whole process of the experiment and trained each participant on all designs. Each participant was trained on the distance designs in five minutes sessions before leaving for real driving. The training was repeated before the beginning of every session.

In the following we present the quantitative results of our evaluation, structured with respect to the dependent measures (see Section ). We applied a non-parametric Friedman’s ANOVA test on data collected from the questionnaire to measure usability of the vibro-tactile distance encodings. Furthermore, to know the difference between individual conditions we applied Wilcoxon tests on each pair of conditions. We applied a simple descriptive statistics on the quantitative data collected from the videos and screen captures to measure information perception error. The results are presented with the help of boxplot and simple bar charts.

The participants were relying on the tactile belt to drive from origin to the destination. Turn by turn direction is displayed by tactile feedback. Each participant drives according to a random sequence of designs in the experiment. The participant is again trained on the first design in the sequence until he will be able to confidently recognize the signals presenting the categories of distance. In addition, the graphical presentation of the different designs – as shown in Figures 2– have been provided. After completing the first route the driver is asked to park the car on a safe place and to fill out the questionnaires. Then the driver is trained on the second design in the sequence until he will be able to confidently recognize the signals presenting the categories of distance and so on.

Usability of the distance encodings

The participants ranked the designs as most preferred, preferred, and least preferred. Figure 5 presents the participants’ ratings for the three vibro-tactile designs. The rhythm and duration based distance encoding is the most preferred design according to the statistics. The rhythm and intensity based distance encoding is rated as preferred. The rhythm based distance encoding is rated as the least preferred design by the participants.

Figure 4 presents the three different routes the participants had to drive. The destination of a route represents the start of the next route. The lengths of route 1, route 2, and route 3 are 1.3 km, 1.2 km and 1.4 km respectively. In the whole experiment the participants drive on every route at a minimum of 4 times for each design. The order of the designs was changed amongst the participants. However, we kept the same order of the driving routes.

Figure 6 presents the participants’ response on the learnability of the designs. The Friedman’s test shows that learnability of vibro-tactile distance designs are significant different (Chi-Square=9.19, p<.05). Further, the Wilcoxon text depicts that the learnability score of duration and rhythm based distance encoding is significantly higher than intensity and rhythm based distance encoding, p<.025. Similarly, learnability score of duration and rhythm based distance encoding is significantly higher than rhythm based distance encoding, p<.025.

RESULTS

All the participants were able to complete all the driving sessions. In the following, the results are divided into quantitative and qualitative results. The quantitative results provide the dependent measures like usability of the vibro-tactile distance encodings, and information perception errors. We applied statistical test to analyze the quantitative results. Sim-

In Figure 7 the boxplot presents the participants’ responses about information perception. The Friedman’s test shows that the participants perceive information significantly different in all designs (Chi-Square=10.286, p<.05). Further,

37

Full Papers

Proceedings: NordiCHI 2010, October 16–20, 2010

Figure 7. Ease of information perception results

Figure 5. Preference ratings for the different tactile designs.

the Wilcoxon test shows that the information perception in duration and rhythm based distance encoding ranked significantly positive than intensity and rhythm based distance encoding, p<.025. There is no difference in information perception of duration and rhythm based distance encoding and rhythm based distance encoding, p=.044. Figure 8 present the participants’ responses on the length of vibro-tactile signal. The participants’ acceptance of the length of vibro-tactile signals in the three designs is significantly different (Chi-Square=6.884, p<.05). The Wilcoxon test shows no significant difference exists in participants’ preferences on the length of the vibro-tactile signal between duration and rhythm based distance encoding and intensity and rhythm based distance encoding, p=.034. Similarly, no significant difference exists in participants’ preferences on the length of the vibro-tactile signal between duration and intensity based distance encoding and rhythm based distance encoding, p=.067.

Figure 8. Time lengths of signals results

Information perception errors

The three designs have been compared by the number of errors made by the drivers in recognizing the category of disThe Friedman’s test does not reveal any significant differtance. The percentages in Figure 9 show that the participants ence in the rating of the design judgment in all three vibromade the least number of mistakes in recognizing all catetactile distance encodings, (Chi-Square=4.478, p=.107). gories of distance in the duration and rhythm based distance encoding. In the rhythm based distance encoding the drivers According to the Friedman’s test there is no significant difmade most of the errors in perceiving a very-far distance, ference in the ease of use of the three designs, (Chi-Square=2.600, 35.7%. We observed the maximum error rate in recognizp=.273). ing far, near, and turn-now distances in the intensity and rhythm based distance encoding, 25.5%, 26%, and 7.2% respectively. Figure 10 shows that regarding the rhythm based distance encoding the drivers made most of mistakes in recognizing very-far as far, 33.3%. Furthermore, the drivers made equal amount of mistakes in recognizing far as very-far and near, 7.7%. The participants made mistakes in recognizing near as far, 17.5%. Figure 11 illustrates that most of the times the drivers made mistake in guessing the very-far distance as far, 12.8% in the rhythm and intensity based distance encoding. The drivers made higher number of errors in recognizing near as far, 26%.

Figure 6. Learnability results

A few mistakes of the participants were observed in rhythm

38

Full Papers

Proceedings: NordiCHI 2010, October 16–20, 2010

and duration based distance encoding according to Figure 12. Turn-now Near Far Very-far

1.5

3.4

Observations and drivers’ comments

The comments and responses have been collected with the help of video recordings. In general most of the participants liked the vibro-tactile display in the car navigation system. The participants stated that the tactile vibration is sensed more effectively in sitting position inside the car. In the following we structured qualitative findings by the aspects of engagement with the environment, visual and auditory modalities, driver distraction and categories of distance.

7.2

0.0

19.3

0.0

15.4 4.7 12.8

26.0 25.5 35.7

Engagement with the environment Duration and rhythm

Only rhythm

We observed from the videos that the vibro-tactile feedback did not cause any visual distraction. In each driving session the participants were freely commenting about the houses and shops on the way. The participants stand no obstruction in thinking aloud and using the tactile display simultaneously. The drivers discovered the vibro-tactile feedback to be quite helpful while talking to the other passengers and concurrently searching a specific location in the city. The participants do not need to slow down the speed of the car to recognize and understand the vibro-tactile feedback. In total the participants missed out 3 out of 221 turns in the whole experiment. A participant commented that ”I have enjoyed this drive, because the system was not grabbing my attention at all”.

Intensity and rhythm

Figure 9. Error rates in recognizing distance categories in relation to all three designs

Visual and auditory modalities

In the responses to questions regarding the overall impression of the prototype 85% of the participants told that they like the vibro-tactile signals for displaying direction and distance. The idea of replacing the auditory feedback with the tactile feedback is preferred by 93% of participants. The participants commented that the tactile feedback is less irritating than the auditory feedback. The tactile interface is preferred for directional information by the participants. Two participants told that they would prefer to have a visual display with the tactile display. So, the visual displays can help in such situations when participants are not confident about the tactile signal.

Figure 10. Drivers’ recognition of distance categories in the rhythm based distance encoding

Driver’s distraction

No visual distraction is evident from the video analysis and the experimenter observations. We observed that in the rhythm based distance encoding the maximum intensity used to present all categories of distance was irritating for the drivers of the car. In the rhythm and intensity based distance encoding a lower intensity to indicate vary-far and far is liked by the participants. One participant told ”I will prefer lower intensity to present longer distances because it’s not disturbing”. Another participant commented ”It is good to present veryfar and far with low intensity signal to get warning before the next turn”.

Figure 11. Drivers’ recognition of distance categories in the rhythm and intensity based distance encoding

Categories of distance

The participants told that they can easily differentiate the vibro-tactile signals of very-far and far with turn-now. The participants preferred to get the signal of turn-now very close to the turn. We observed that the smooth signals for the distance very-far and far are less irritating for the participants. One participant did not felt the very-far signal in the rhythm

Figure 12. Drivers’ recognition of distance categories in the rhythm and duration based distance encoding

39

Full Papers

Proceedings: NordiCHI 2010, October 16–20, 2010

and intensity based distance encoding twice. The participants told that they are certainly interested in the distances near and turn-now. In addition, the participants responded that it is good to have very-far and far signals in advance, so they will get attentive for upcoming turn.

drivers to easily recognize the meaning of the signal. The two designs, rhythm based distance encoding and rhythm and intensity based distance encoding, are composed of similar rhythm patterns with different levels of intensity. In both designs the results differ insignificantly in terms of mistakes of participants in perceiving distances. So, we conclude that modification in the level of intensity does not impact significantly on perception of distance in the designs. We also observed from our qualitative results that the lower intensity is less irritating for longer distance. In summary the approach of rhythm and intensity based distance encoding can be beneficial in some aspects (e.g. for longer distances lower frequency is not disturbing) but the approach of intensity based distance encoding is not mainly contributing. The results support the concept of quiet communication in the car proposed by [17]. The results encourage the use of tactile display for complex information in the car navigation systems besides the fact that the drivers need backup support of visual and auditory modalities [7].

DISCUSSION

The tactile interface is certainly helpful in the situations when it is very risky to put eyes off the road. The participants found the tactile feedback useful, less irritating and safe for the car navigation systems. The rhythm and duration based distance encoding, which supports the concept of countable pulses, is most successful among all tested designs. The results show that the rhythm and duration design is significantly more usable than the other two designs. The driver made considerably less errors in perceiving distance information. Therefore, it is discovered as an appropriate approach to encode distances through vibro-tactile signals. In the following the results regarding our research questions for this study (Section ) are discussed.

Q4: What are common errors that participants make to interpret distance information by using the tactile display? The drivers tended to make the least number of errors while they get opportunity of counting the number of pulses. In general, we can observe from the quantitative and qualitative results that most of the times, the participants face difficultly in distinguishing vibro-tactile signals for the two adjacent categories of distance. Most of the errors are evident in the rhythm based distance encoding. Similarly, the qualitative results show that the participant was unable to feel the vibro-tactile signal with lower intensity. The participants sometimes do not perceive the vibro-tactile signals of lower intensity because of vibration of the car.

Q1: Will the participants perceive turn by turn distance information without support of visual and auditory modalities in the car? The participants missed the crossing only 3 times in the whole experiment which supports the argument that vibro-tactile signals can be used to present distance information for the whole route while driving. We concluded from the qualitative results that the participants asked for visual display in case they were not sure regarding the meaning of the vibro-tactile signal. The findings like engagement with environment and the participants’ preferences to the tactile display support the fact that tactile feedback with no visual and auditory attention is beneficial for providing navigation information in many situations. So we determined that the vibro-tactile signals can be qualified to present spatial information in the car navigation system.

From qualitative results we can conclude that the tactile display does not imply any visual distraction, thus, the drivers are able to fully concentrate on the primary driving task. Our findings support the fact that the tactile displays in the car navigation systems cause less amount of distraction to the driver [17].

Q2: Which one of the encodings – rhythm, rhythm and intensity, rhythm and duration – is the most usable encoding? The results show that the rhythm and duration based distance encoding is proved to be more usable among all tested designs. In addition, it is easier for the driver to learn the distance information in rhythm and duration based distance encoding as compared to the other encodings. The concept of countable pulses for encoding distance information is quite distinct from previously proposed schemes [17, 19]. The results demonstrated that the countable pulses encoding is usable in the car navigation system. Furthermore, the participants believe that afterwards they can also gain confidence on the vibro-tactile rhythm based distance encoding approach with more practice.

The results of the experiment can only be applied to the street scenarios. The distances are decided for categories of very-far, far, near, and turn-now while driving in a residential area. The vibro-tactile signals were controlled by the experimenter, so we cannot neglect the chance of human error in the experiment. The gender aspect is also limitation of the study because mostly male participants have taken part. The results cannot be generalized for old drivers.

CONCLUSION

Q3: Which distance encoding helps the driver to make the least number of information perception / interpretation errors? The participants made the least number of errors on the rhythm and duration based distance encoding compared to the other two designs. Besides rhythm, the duration of the vibro-tactile signal plays an important role in the vibrotactile distance encoding in the car. The simplicity of the vibro-tactile information encoding information allowed the

From previous studies [6, 17] we may assume that tactile feedback can be used as an alternative way to present distance information to the driver without support of the visual or auditory displays. We investigated in this paper three vibro-tactile distance encodings based on rhythm, intensity and duration. To investigate the designs we conducted an evaluation on a real navigation scenario in an urban environment.

40

Full Papers

Proceedings: NordiCHI 2010, October 16–20, 2010

The study shows that the tactile feedback is quite helpful for the navigation in the environment without support of visual or auditory displays. The tactile design which modifies the parameters rhythm and duration of the signal is proved to be most successful among the three tested designs. By this design the driver receives a tactile stimulation through countable vibration pulses, which encode the distance information. We can conclude that the approach can be applied to encode distance information in the most complex road scenarios. However, the present study is carried out in the urban environments and it is not clear, whether the results can be generalized for other environments, such as motorways or highways and how the distance categories have to be adapted to other scenarios. This issue remains for following analysis.

6. J. H. Hogema, S. C. D. Vries, J. B. F. V. Erp, and R. J. Kiefer. A tactile seat for direction coding in car driving: Field evaluation. IEEE Transactions on Haptics, 2 (4):181–188, 2009. 7. D. Kern, P. Marshall, E. Hornecker, Y. Rogers, and A. Schmidt. Enhancing navigation information with tactile output embedded into the steering wheel. Pervasive’09, 7:42–58, 2009. 8. N. J. Mansfield. Human Response To Vibration. CRC Press, 2005. 9. T. McDaniel, S. Kirshna, and D. Colbry. Using tactile rhythm to convey interpersonal distances to individuals who are blind. In CHI-2009, 4669-4674.

In the near future, further investigations will be conducted, in particular, to find out if there are any gender or age differences regarding the acceptance and usability of tactile presentations while driving. For our long term research, we have identified several research questions regarding tactile presentations for people on the move, such as:

10. A. Pauzie and C. Marin-Lamellet. Analysis of aging drivers behaviors navigating with in-vehicle visual display systems. In In-vehicle Navigation and Information Systems, 1991. 11. M. Pielot, O. Krull, and S. Boll. Where is my team? Supporting collaboration and situation awareness with tactile displays. In CHI ’10, 2010.

1. What other information besides turn by turn instructions can be presented with tactile feedback to the user, in order to improve safety and comfort?

12. M. A. Regan, J. D. Lee, and K. L. Young. Driver Distraction: Theory, Effects, and Mitigation. CRC Press Taylor & Francis Group, 2009.

2. Is it possible to transfer the findings also to cyclists, motorcyclists, and pedestrians?

13. A. Riener and A. Ferscha. Reconfiguration of vibro-tactile feedback based on drivers’ sitting attitude. In International Conferences on Advances in Computer-Human Interactions, 2009.

Information presentation in cars is currently a topic of high interest in the public and transportation industry. On the one hand, there is a huge demand from the end users’ point of view to ease the ability to move from one location to another with more safety and comfort. On the other hand, new technical developments allow advanced and innovative interaction techniques in cars, which will be an important argument for buying cars in the future. Tactile displays will have a great impact on future developments in human machine interface design of automobiles.

14. M. Straub, A. Riener, and A. Ferscha. Distance encoding in vibro-tactile guidance cues. In MobiQuitous 2009, 2009. 15. K. Tanaka, Y. Kishino, T. Terada, and S. Nishio. A destination prediction method using driving contexts and trajectory for car navigation systems. In ACM symposium on Applied Computing, 2009. 16. K. Tsukada and M. Yasumura. ActiveBelt: Belt-type wearable tactile display for directional navigation. In Ubiquitous Computing, 2004.

REFERENCES

1. S. Brewster and L. M. Brown. Tactons: structured tactile messages for non-visual information display. In AUIC ’04, 2004.

17. J. van Erp and H. van Veen. Vibro-tactile information presentation in automobiles. In Eurohaptics 2001, 2001.

2. G. E. Burntt and J. M. Porter. An empirical comparison of the use of distance versus landmark information within the human-machine interface for vehicle navigation systems. In Human Factors in Transportation, Communication, Health, and the Workplace, 2002.

18. J. B. F. van Erp. Guidelines for the use of vibro-tactile displays in human computer interaction. In Eurohaptics 2002, 2002. 19. J. B. F. van Erp, H. A. H. C. van Veen, C. Jansen, and T. Dobbins. Waypoint navigation with a vibrotactile waist belt. ACM Transactions on Applied Perception, 2:106–117, 2005.

3. H. Fukuda, T. Inoue, Y. Sato, and Y. Hayashi. Study on level crossing design and evaluation method based on cognitive model. Quarterly Report of RTRI, 40:26–31, 1999. 4. M. R. Grech, T. Horberry, and T. Koester. Human Factors in the Maritime Domain. CRC Press Inc, 2008.

20. H.-J. van Veen, M. Spape, and J. V. Erp. Waypoint navigation on land: Different ways of coding distance to next waypoint. In EuroHaptics 2004, 2004.

5. W. Heuten, N. Henze, M. Pielot, and S. Boll. Tactile Wayfinder: A non-visual support system for wayfinding. In NordiCHI 2008, 2008.

21. C. D. Wickens. Processing resources in attention. In Processing Resources in Attention. London:Academic, 1984.

41

Full Papers

Proceedings: NordiCHI 2010, October 16–20, 2010

Facilitating Continuity: Exploring the Role of Digital Technology in Physical Rehabilitation Naveen Bagalkot, Elena Nazzi, Tomas Sokoler IT-University of Copenhagen Rued Langgaards Vej 07, Copenhagen nlba, elna, sokoler, @itu.dk ABSTRACT

INTRODUCTION

In this paper we report our early experiences on exploring if, and how, digital technology can help facilitate a stronger sense of continuity in the physical rehabilitation process, as experienced by the therapists and the senior citizens. We recognize four aspects of the design space offered by the notion of continuity, and present two design explorations: MagicMirror, exploring the design for self-monitoring and collaborative articulation; and Walky, exploring the design for self-monitoring and the integration of rehab activities with other everyday activities. Taking a research-throughdesign approach we have used sketching in a co-design setting with senior citizens and therapists, as the main vehicle for our research. With our MagicMirror and Walky design explorations, we aim to contribute on two levels: firstly, by providing two proof-of-concepts of designing digital technology for the physical rehabilitation of senior citizens. Secondly, and more importantly, we provide a pointer towards a new practice of physical rehabilitation: a practice where the role of digital technology is to support, through the facilitation of continuity, the collaboration between therapist and senior citizens across the private home and the rehab clinic. We speculate that, digital technology, in this way, can become a constructive part of the general move towards successful and more efficient rehabilitation processes for the ever-growing number of senior citizens in need of rehab.

Most of the developed world is facing a demographic challenge, of providing better welfare for an increasing population of senior citizens. This has triggered a host of national and pan-national initiatives in exploring the use of digital technology, aiming to enable assisted but independent living for senior citizens in their homes. A white paper from the Danish Government [8] states how a successful rehabilitation of senior citizens is central in facing this demographic challenge and identifies four basic aspects of a successful rehabilitation. In this paper we take the definition of successful rehabilitation from the Danish whitepaper as our starting point to explore the role of digital technology when aiming for more successful and efficient rehabilitation processes. In general, the Danish white paper [8] recommends more active involvement of senior citizens in the rehab process emphasizing that this process should be one of collaboration between therapist and senior citizens. Also, the white paper suggest the need for a rehabilitation process where the rehabilitation activities (i.e. physical exercises) are to be better integrated with the everyday mundane activities that constitute everyday life of senior citizens. This includes considering the rehab process as an everyday social phenomenon opening up opportunities for social wellbeing through social interaction within groups of senior citizens engaged in rehabilitation.

Author Keywords

Rehabilitation, interaction design, aging, continuity, everyday activities, physical sketches, research through design, self-monitoring

Put together, we read these recommendations as the overall need for a stronger sense of continuity that will render the rehabilitation process more coherent and more conducive towards collaboration involving all parties. We take this as our point of departure and present our explorations as part of negotiating the design space that emerges as we move forward.

ACM Classification Keywords

H.5.m: Information interfaces and presentation (e.g. HCI): miscellaneous, J.3. Life and Medical Science.

In particular, we present two such explorations: MagicMirror, exploring the design for self-monitoring and collaborative articulation; and Walky, exploring the design for self-monitoring and the integration of the rehabilitation activities with other everyday activities. Through these explorations we argue for a new role of digital technology in the physical rehabilitation of senior citizens: of enabling a successful and more efficient rehabilitation process by

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. NordiCHI 2010, October 16–20, 2010, Reykjavik, Iceland. Copyright 2010 ACM ISBN: 978-1-60558-934-3...$5.00.

42

Full Papers

Proceedings: NordiCHI 2010, October 16–20, 2010

facilitating a stronger sense of continuity in this process, as experienced by senior citizens as well as therapists.

technology we design to become part of physical rehabilitation processes:

In the following section we describe briefly the background and the motivation of our explorations. Before describing each of our explorations, we emphasize our methodology of researching through design. We then describe each of our explorations emphasizing the core scenario, the process of co-exploring in-situ, and the underlying aspects of continuity they explore. We then conclude by summarizing the explorations, highlighting our two core contributions and discussing future work.

Self-monitoring

Our design should help the citizen become more aware of his/her own condition and the effects of rehabilitation in general reinforcing the role as being an expert on his/her own condition. Collaborative articulation

Our design should encourage a process of collaborative articulation between citizens and professionals in the continuous negotiation and re-negotiation of rehab goals that drives the rehab process forward.

BACKGROUND

The national Danish whitepaper [8] defines a successful rehabilitation as follows:

Integration with everyday activities

Our design should support the integration of the rehabilitation process with the citizen’s everyday activities beyond the clinic for an independent and meaningful life.

"A goal-oriented, cooperative process involving a member of the public, his/her relatives, and professionals over a certain period of time. The aim of this process is to ensure that the person in question, who has, or is at risk of having, seriously diminished physical, mental and social functions, can achieve independence and a meaningful life. Rehabilitation takes account of the person's situation as a whole and the decisions he or she must make, and comprises coordinated, coherent, and knowledge- based measures."

Rehabilitation as social phenomenon

Our design should incorporate the notion that a rehabilitation process, though anchored within physical exercises, can be viewed as an everyday social phenomenon with inherent openings for social interaction across citizens, their relatives, professionals and the local community. We find that these four points encourage us to address an overall notion of facilitating a sense of continuity in the rehabilitation process, as experienced by the therapists and the senior citizens. We will talk about this as exploring four aspects of this same notion.

We read this definition as bringing forward four points for reflection when considering the role of the digital

Figure 1: Map showing the Design space offered by the notion of facilitating Continuity in Rehab process

43

Full Papers

Proceedings: NordiCHI 2010, October 16–20, 2010

Figure 1 is a map showing examples of the way the four aspects may play out as part of a rehab process. Also, the map shows how the type of rehabilitation processes, that we seek to design in support of, cuts across time and place— across exercising at home and exercising at the rehab center.

rehab activities like exercising at home or walking in the park have the potential to act as ‘tickets-to-talk’ providing openings for social interaction. Digital technology, in turn, may help make an individuals rehab activities noticeable to a group of relevant friends/rehabbuddies. Piggybacking on everyday activities as openings for social interaction has been suggested as a way to circumvent the stigma of loneliness when making initial contact [10]. Hence, though the rehab exercise primarily was meant to increase physical wellbeing it may, as a positive side effect, be a catalyst for social interaction possibly leading to increased social wellbeing within a group of senior citizens undergoing rehabilitation.

• Enabling senior citizens to bring information back and forth, between the home and rehab center, may induce a process of collaborative articulation during therapists-tocitizen meetings: a two way consultation between experts of each their kind. The senior citizens can let expertise on their own everyday lives meet with the expert knowledge on rehabilitation held by the professional therapist. This will inform and help to drive the negotiation process of planning the particular rehab activities relevant for the senior citizen in question. Previously, Sokoler et al [9] explored this aspect of collaborative articulation within the world of hand rehabilitation.

Hence, as the Figure 1 lays-out, these are the four different aspects of the notion of continuity, which become more prominent when the rehab process is explored beyond the boundaries of the rehabilitation centre. The overall objective of our research initiative is to explore how to design digital technologies for the rehabilitation by exploring the design space offered by the notion of continuity in the physical rehabilitation process. However, in this paper we limit our selves to explore the design space offered by three aspects: self-monitoring, collaborative articulation and integration with everyday activities. While, we believe that, the social interaction aspect is an important and promising avenue to explore, it is left for future work.

• Walking in the park or going to a nearby shopping mall are instances of everyday activities of senior citizens that may offer opportunities to integrate rehab activity in everyday activities, contributing to an ongoing rehab process. Providing senior citizens with their therapist’s recommendations of the ‘correct’ way to walk, as they walk is an example of this integration. Furthermore, enabling the senior citizen to bring information about everyday walking activities to the next meeting with the therapist may (as described above) inform the planning of rehab activities.

In the remaining sections we present the two explorations of the overall design space that we have conducted so far. The two explorations in question are: “MagicMirror” and “Walky”. The MagicMirror explores designing for the aspects of self-monitoring and collaborative articulation while “Walky” explores designing for the aspects of selfmonitoring and integration of the rehabilitation process with everyday activities.

• Inherent to the above two aspects is the need for the senior citizens to monitor the progress of their rehab process, as they are performing dedicated rehab exercises at home or going about their everyday activities. Thereby, self-monitoring is an integral part of the above two aspects.

In the next section we highlight our methodology of researching through design and then detail out the two explorations.

• Finally, as the rehab process is taken beyond the rehab clinic to the realm of the everyday life of senior citizens, the social dimension becomes more prominent. Everyday

Figure 2: Research through Design as a methodology of MagicMirror and Walky explorations.

44

Full Papers

Proceedings: NordiCHI 2010, October 16–20, 2010

Here we have the opportunity to work with an interesting group of occupational therapists, very much involved into exploring new solutions for enhancing the rehabilitation process. Our collaboration started 2 years ago and includes different projects, among others, the use of the Wii platform and physical gaming for rehabilitation.

METHODOLOGY

We follow a research through design approach as a methodology for doing research. Research through design has gained importance in the recent years within the fields of HCI and Interaction Design research. As a still evolving form of research methodology, there have been many definitions of what it means to contribute to research through design [6, 13, 15, etc.]. However, the common understanding of researching through design is the ability to contribute to ongoing research discourses through designing for a particular design situation. As prerequisites of exemplary research through design, there has to be an evident expansion of the conceptual and philosophical framework [3], brought about through a reflective discourse built around a set of design exemplars [5]. The works presented in this paper follow this emergent way of contributing to research.

Citizens living in the local neighborhood visit the center. They undergo rehabilitation from simple aging problems, to mild or more severe impairments. We had the opportunity to work with two elderly women during our two explorations. While the first lady has problems with her back, the second lady has a balance problem and uses a four-wheeled walking aid since four years. EXAMPLE SKETCH: MAGIC_MIRROR

MagicMirror is a sketch inspired from the settings of a gym, where wall-wall mirrors provide a self-referential image of ones progress of muscle building. Here the MagicMirror takes multiple forms in different situations: a big screen at the rehab centre, and a TV at home, for instance.

As mentioned above, the larger framework explored with MagicMirror and Walky is, if and how digital technology can facilitate a sense of continuity in the physical rehabilitation process. This investigation is driven by the two concrete design explorations, where we explore the design space offered by three aspects of continuity: selfmonitoring, collaborative articulation, and integration of rehab activities in everyday life (see Figure 2). Thereby, we contribute to the fields of HCI and Interaction Design research, through these two design explorations, on two levels: firstly, by providing concrete proof-of-concepts of digital technology designed for the specific physical rehabilitation activities of senior citizens. Secondly, and more importantly, we provide a pointer towards a new practice of continuous, ongoing and collaborative physical rehabilitation process, informed and facilitated by digital technology.

Recently initiatives in ‘WiiHabilitation’ have gained ground in the world of physical rehabilitation [11]. With the Wii platform introducing a more physical gaming experience, the idea of using this technology as part of physical rehabilitation is straightforward. Hence, experiments with ‘WiiHabilitation’ are gaining ground throughout the world of geriatrics and physical rehabilitation [for instance, 2, 7, 11]. The Wii platform allows us to look into the general notion of integrating play, and the experiential qualities belonging to the gaming universe, with the tough everyday challenges of going through physical rehabilitation. With MagicMirror, we explore beyond what is offered by the immediate features of Wii and games for specific kinds of rehabilitation. Here, we move towards exploring the design of digital technology for facilitating the aspects of self-monitoring and collaborative articulation in the rehab process, beyond the rehab centre.

The MagicMirror and Walky design explorations lie in the design exploration vertex of the Interaction Design Research triangle [6], with sketching in a co-design setting as the central driving engine. Inspired from Bill Buxton [4], by sketch we mean early manifestations of design ideas in different forms, not the least in software/hardware, video and photomontages, etc., that are available for people to experience, and hence be part of the design exploration in early stages. The process of sketching and negotiation of the design space was situated in-situ, with the therapists and the senior citizens taking part in the process through enacting and experiencing. Thereby the evaluation of the sketches and the specific features of the design concepts was an ongoing and inherent part of the process. Additionally, the co-design process facilitated a move forward in co-articulating what it means to design, and use, digital technology to facilitate three aspects of continuity in rehab process, as experienced by the therapists and the senior citizens.

Specifically, we explore the possibilities of tracking and video recording the exercises the senior citizen performs under the supervision of the therapist, at the rehab centre. The senior citizen takes home this recorded video and uses it as the ‘reference’ exercise to monitor self while exercising at home, as it plays out on the TV. Simultaneously, the MagicMirror tracks the body movements, records and overlays it on the instructional video, thus giving the senior citizen a self-referential video for exercising. This recorded video is then taken back to the clinic where the senior citizen and therapist discuss the progress in detail, thus enabling a more collaborative rehabilitation process. A Scenario

Ole is a 70-year-old man living alone. He has developed knee pain due to a biking accident and is undergoing a rehab process at the rehab centre. He meets his therapist

SETTING: TÅRNBY REHABILITATION CENTER

The setting of our explorations is Tårnby Sundhedscenter, a local rehabilitation center of the Copenhagen Municipality.

45

Full Papers

Proceedings: NordiCHI 2010, October 16–20, 2010

and they record the video of Ole doing the exercises at the rehab centre. This video is carried onto Ole’s cell phone to his home, where he plays the video on his television, while exercising. The MagicMirror tracks his movements and overlays this feed with that of the recorded version, thus giving Ole a direct reference to exercise correctly. On his next visit to the rehab centre, the therapist looks up the recorded home videos of Ole’s exercises and both of them discuss his progress and next steps.

being captured. Our sketch of MagicMirror is still work-inprogress and a little away from this technological possibility. Here, we simulated the tracking and overlapping by using a combination of Adobe Premier and Flash. The exercises were captured from the Adobe Premier suite through the webcam of a MacBook Pro, which were then taken to Adobe Flash and the body joints were tracked frame by frame (see Figure 3). This video was taken to the home of the senior citizen and played as the reference video. The new exercises were again recorded through Adobe Premier and played alongside the reference video during the exercise at home. This home video was taken back to the design studio, to be tracked and laid over the reference video. Finally we took the overlaid video to the rehab centre again to simulate and enact the discussion of progress.

Sketching and co-exploring MagicMirror

Currently in a physical rehabilitation process, the therapists prescribe exercises for the senior citizens to do at home, but don’t have much information of how the senior citizens managed to do these exercises. At the same time, unlike while exercising with their therapists, the senior citizens don’t have much clearer instructions during exercising at home. A case in point is an 82-year-old lady doing pushups at home on a soft mattress, while her therapist having no clue that his instructions will be carried out in such manner. This was observed during one of the field studies done by a master student working on the project. The MagicMirror scenario is an example of a way digital technology can facilitate rehabilitation process to go beyond the rehab centre by enabling information exchange between the rehab centre and home. As the scenario depicts, the recorded exercises at the rehab centre become the ideal exercise while exercising at home. The live tracking and overlaying of the current exercise over the ideal exercise gives a self-referential monitoring of the progress to the senior citizen. With this recorded home exercise available at the rehab centre, the senior citizen can discuss with the therapist the various aspects of the progress of the rehabilitation process. As the senior citizen and therapists manage this exchange of rehab information, they prepare themselves for a more collaborative articulation of the rehab process.

Figure 3: The body joints tracked for the particular exercise Co-exploring through enactments

For this initial exploration, we selected one form of exercise for back training: lifting both the hands from side of the body to the top and back. The vertical body position, and the straightness of the joints were the main features to be tracked in this exercise. As shown in figure 3, the MagicMirror sketch tracks the vertical position of the spine with respect to the hand movements. Also the senior citizen was asked to wear wrist, arm and head bands with IR emitting LEDs, which were tracked by a webcam ensemble, for later processing in an image processing software. For the purpose of this sketch, the black bands augmented the visual reference provided by the lines.

Sketching MagicMirror

The intended technology behind the MagicMirror is motion tracking through an Infrared enabled webcam, which will then be processed by an image processing software for tracking and overlay. This process happens in real-time, with the processing done by the software as the video is

Figure 4: A Three stage Co-designing of MagicMirror - at rehab center, at home and back at rehab center

46

Full Papers

Proceedings: NordiCHI 2010, October 16–20, 2010

During, and after, the process of sketching we involved two therapists from the Tårnby rehabilitation centre and a senior citizen with back problems. As an early exploration of the possibilities, we enacted a three stage cycle with the senior citizen and the therapists: recording of the reference video at the rehab centre, exercising at home using this reference video, and finally, the meeting back at the rehab centre with the therapist and senior citizen discussing over the home video (see Figure 4).

recording and tracking of exercises done, both at the rehab centre and at home. This feature addresses facilitating a more collaborative articulation by enabling both the therapists and the citizens to meet with richer information and discuss the progress of the rehab process. Facilitating the Aspect of Self-monitoring

• The senior citizens themselves are recorded as they exercise with the therapist at the rehab centre. While exercising at home looking at this reference video, the senior citizens themselves are their own ideal, rather than an animated avatar. This provides a more self-referential and immediate monitoring of the rehab progress.

Co-discussing the enactments

During the discussion after the enactments, the therapists immediately realized the importance of the video material, as any discrepancy was visually evident with the tracking of the body joints.

• MagicMirror intends to track the relevant body joints and draws a line-diagram (see Figure 3) in real-time as the exercises are being recorded. As the current video with this information is overlapped over the reference video, the senior citizen gets an immediate visual feedback of the correctness of the exercise.

Meanwhile, the senior citizen was enthusiastic of seeing herself as the focus of the reference exercise video. She also mentioned that the overlaying of her home exercise video with the reference video was helpful to encourage her, as she gets to see the extent she stretched her arms today, compared to her previous day’s effort.

These two features address specifically the possibility of self-monitoring as, and when, the senior citizens exercise at home.

During the third stage in the cycle, the senior citizen and therapist looked at the video of the exercise done at home on a big screen at the rehab center (see Figure 4). Here the line-diagram and overlaying of the videos gave a clear indication of both, the degree of progress made by the senior citizen and the discrepancies in the exercise. Though the senior citizen was well advanced in her rehab process, both the therapist and the senior citizen immediately sensed the possibilities of tracking and overlaying, in facilitating self-monitoring of the rehab process. The overlaid videos gave a direct visual evidence of to what extent the senior citizen could stretch her arms on this day as compared to the effort she put in the reference video.

EXAMPLE SKETCH: WALKY

Walky is a sketch that explores walking, as a rehab activity as well as an activity of daily living. It is driven by the idea of augmenting everyday objects of rehabilitation with digital technology to provide feedback for self-monitoring and to communicate with other objects of daily living. Walky is an augmented four-wheeled rollator providing the senior citizens opportunities of self-monitoring of their walking as they go about their everyday activities. Recently, assistive technologies have been developed to support the walking activity of senior citizens with mild and severe impairments. Passive robotics and intelligent environments have been studied to provide the support for autonomous object detection and avoidance, for navigation, and for breaks and speed control [e.g.13]. Sensors, for instance pedometers and accelerometers, are largely used to monitor walking activity. Flowie [1] is a recent example of self-monitoring applied to the walking exercise. It connects a pedometer and a digital picture frame in the home and creates a persuasive virtual coach that aims to motivate people to walk more.

During the enactment and later, the therapists were of the opinion that the video material from the home of the senior citizen was valuable information on the everyday realities of the world of the senior citizen, which will help them in fine-tuning the rehab process better for the senior citizen. Furthermore, the senior citizen, when discussing with the therapists, imagined that with this video material she could have a more central role in articulating the next stages in the processes. Design Rationales: Facilitating Self-monitoring and Collaborative articulation

With Walky we explore beyond passive robotics and concentrate on the design of digital technology for facilitating the aspect of integrating rehabilitation activity into everyday life of senior citizens, and the aspect of selfmonitoring. We specifically, concentrate on walking as an activity of everyday life.

In the next paragraphs we reflect on how MagicMirror facilitates the aspects of collaborative articulation and selfmonitoring. Facilitating the Aspect of Collaborative Articulation

• MagicMirror provides both, the therapist with the knowledge of how the exercise happened at home, and the senior citizen with the knowledge of how the exercises are done correctly as the senior citizen is exercising at home. It enables information exchange between the rehab centre and the home, through video

Walky consists of an augmented four-wheeled rollator, combined with a small movable device, called PocketWalky. The rollator is augmented with sensing, actuating and communication technologies. PocketWalky

47

Full Papers

Proceedings: NordiCHI 2010, October 16–20, 2010

communicates with the rollator, displays information about walking, and allows senior citizens to exchange progress information with the therapists in order to set and reset together the rehabilitation goals.

As a starting point we formulated three questions about walking: “how much I walk” to understand, and keep track of, the distance covered while walking; “how I walk” to provide clues about posture and balance for correct usage of the rollator; and “where I walk”, to get a sense of being part of ones rehabilitation friends through visual notifications of others’ activities. The third question reflects the fourth aspect of continuity: of facilitating the opportunity for social interaction through making rehabilitation exercises visible for within the local community of senior citizens. It has been just briefly touched during this first sketch of Walky and will be part of its future explorations.

Specifically, we explore the possibility to make the use of everyday objects a sources for: self-monitoring, and communication with the therapists and with the local community. In doing so, we aim to integrate rehabilitation exercises for better walking into other everyday activities of senior citizens involving walking. A Scenario

Anne is an 80-year-old woman living alone for 15 years. She has been using a four-wheeled rollator for 2 years due to a balance problem, and she is going to the rehabilitation center twice a week. The therapist introduces her to Walky and PocketWalky, a rollator that gives her feedback about her balance while walking and its companion movable device. This is done through mild vibrations on the handle of the side she is exerting more pressure. Anne and her therapist set her walking goal of the week in terms of 6 times the distance to the local shop from her home, and 3 times the distance to the park near her home (corresponding approximately to 3 km). During the week, after going to the market she sees her progress on PocketWalky, of how much she has been walking. During the next visit to the rehab center, her therapist takes a look at her progress and together they discuss to reset or maintain the goals.

In summary, with Walky we explore the design of an augmented walking aid that: • Enables senior citizens to self-monitor their walking activity (“how I walk” and “how much I walk”); • Makes the walking goals visible as the senior citizens go about their everyday activities; • And enables the therapist to view the report of how the walking went in everyday life when the citizens come back to the rehabilitation center. Sketching Walky

We sketched Walky using the Arduino1 platform. We augmented a common rollator used in Denmark with simple sensors: a pressure sensor on each handle, an ultrasonic distance sensor, and a switch to count the number of rotations of the rollator’s wheel. The first sketch included tactile (a vibration motor on each of the handles) and visual actuators (a couple of LED lights on the rollator frame). The sketch gives feedback on posture, balance, distance, and speed. We added a ZigBee2 module for the communication with PocketWalky, the movable device, to transmit information about walking activity, and to receive commands and calibration details. To sketch PocketWalky we used simple cardboard sketches and a more realistic mock up realized with a 3D printer. In this early exploration we gathered the information that PocketWalky would display with another Arduino+ZigBee sketch connected to a portable pc used to displaying the sensors data. Finally we used other cardboard sketches of digital picture frames to represent one of the everyday objects with which the movable device can communicate in the home environment. Figure 5 shows these sketches at work in our co-design sessions.

Sketching and co-exploring Walky

Currently the senior citizens undergoing mobility rehabilitation, with the help of walking aids, visit the rehabilitation center once or twice a week. They undergo an hour of exercises with the supervision of their therapists. An example of a common exercise is to walk as much as they can, back and forth, on a 20-meter stretch of corridor. The therapists correct small details of the usage of the aid; depending on the specific problem and severity, the therapists remind the senior citizens to keep their back straight, or to concentrate on keeping their balance in the center. The therapists suggest the senior citizens to exercise by going out for a walk everyday, when not visiting the rehab center. Some of the citizens fix their goals in terms of meters to walk in a week, others in terms of going to the nearby shop everyday. However, neither the therapists have any means to figure out how the senior citizens walked, nor the senior citizens have any means of verifying if, and how much, their everyday walking is helping them towards progress, or whether their posture and balance is right while walking.

These sketches were realized to answer the questions of how much I walk and how I walk. • “How much I walk?” While walking with Walky, the senior citizen can access and monitor the distance

As any other aid, rollators have a proper way to be used effectively: walk within the line joining the front two wheels (closer to the person walking), the back has to be upright, and the balance has to be kept by maintaining the body weight in the center of the imaginary line that divides the rollator in two symmetrical parts.

1

Arduino is an open-source electronics prototyping platform. See http://arduino.cc. 2

ZigBee is a standard-based communication technology. See http://en.wikipedia.org/wiki/ZigBee.

48

Full Papers

Proceedings: NordiCHI 2010, October 16–20, 2010

covered, and the duration of the current walking activity. Walky gives feedback of distance covered in terms of meters, which can be translated in terms of landmarks reached (e.g. how many times this distance equals the distance from home to my everyday shop). During the walk, this feedback is displayed on PocketWalky. Through this device the senior citizen can visualize the walking goal, track the progress made, and reset the weekly or daily goals.

walk. During the walk we gave her feedback on how much she was walking in meters. She liked the idea and actually discovered that she had walked much more than she thought she could. We used the cardboard sketches of PocketWalky to visualize the feedback, and to show her possible interactions. We prepared some paper based transparent screens to overlay to simulate the updating of the information. Co-discussing the experiences

• “How I walk?” Walky reminds the senior citizen to keep the proper posture and the right balance while walking. If the senior citizen is walking too far from the structure of Walky, the PocketWalky displays either graphical animations or colored LED lights. This will remind the senior citizen to walk within the rollator frame (closer to it) and keep the back upright, re-establishing the right posture. If the senior citizen is exerting more weight on the rollator handles, and/or the weight is not evenly distributed between the two handles, a mild vibration— on either handle—indicates the side on which the senior citizen is putting more weight.

In general the senior citizen felt the haptic feedback more direct, and discreet, as compared to the acoustic or visual one. Which highlights the fact that, while integrating rehabilitation as part of everyday life, we need to design the sensing and actuating technologies that don’t provoke embarrassment or invade privacy. For example, providing tactile feedback showed to be non intrusive, effective, and intuitively preferable to audio or visual feedback. The senior citizen said that she was engaged in keeping her correct balance by trying to make the rollator’s handle not vibrate. The therapists recognized the opportunities in receiving direct feedback from the aid. They recognized how a direct haptic feedback can communicate what is wrong, and produce an effective body reaction for correcting the usage of Walky.

Co-exploring through Experiencing the Sketch

During, and after, the process of sketching, we involved the therapists and an elderly woman to co-explore the possibilities offered by Walky. To begin, we organized two sessions: an indoor session at the rehabilitation center, and an outdoor session in the local neighborhood.

The senior citizen experienced direct feedback of selfmonitoring in the rehabilitation environment and she recognized its usefulness in this setting. During the first visit the senior citizen mentioned she wouldn’t need a direct feedback during her everyday walking outside the rehab center. However during the second visit outdoors, she appreciated how the feedback, about how much she has walked, could give her a more precise translation from her perception of distance to the actual meters covered. After this observation we could imagine a future scenario where self-monitoring has different levels and each of these levels can be turned on and off by the senior citizens according to the current stage of their rehab process. The therapists also recognized a good potential in PocketWalky, as a dedicated device to link rehabilitation and everyday activities, and the possibility it offers them to get informed about the progress made by the senior citizens outside the rehab center.

In the rehabilitation center, we introduced the senior citizen to the project through the Walky sketches with the collaboration of the therapists (see Figure 5a). The senior citizen was asked to walk with Walky, and to experience the different direct feedbacks about posture and balance. Both the senior citizen and the therapists had the chance to walk with the sketch and comment on the perception of the feedbacks. In the second outdoors visit (see Figure 5b) we focused on the questions of “how much I walk?” We organized a short walk outside the rehabilitation center. The rehab center is part of the senior citizen’s neighborhood and hence familiar for her. We let her decide the direction and the distance, and she indicated a white house, whose distance she thought was a good approximation of how much she could

Figure 5: Co-exploring Walky indoors (a), outdoors (b), Sketches of PocketWalky (c,d).

49

Full Papers

Proceedings: NordiCHI 2010, October 16–20, 2010

The indirect feedback accessible from PocketWalky (e.g. distance covered) offered a more objective measurement of “how much” the senior citizen has walked. We realized that absolute feedback in terms of meters loses meaningfulness in relation to everyday life. Being able to walk four times the distance from home to the nearby shop in two days has much more meaning than walking 800 meters. Together we recognized that the possibility of setting goals, and being able to self-monitor the progress of these goals, in terms of local landmarks and icons encountered in everyday activities (shop, park, friend’s house, post office), could be very helpful. Finally, the therapists recognized the possibility that Walky offers in motivating senior citizens towards getting better, by offering them real-time feedback on how they are walking, and also by the possibility of treating their walk to a nearby shop as a rehabilitation activity.

for the senior citizens to exercise the correct way of walking, improving their rehabilitation. • These sketches work on everyday rehabilitation tools and everyday objects: PocketWalky, for example, enters the home and combines with common objects. It may take a form of a digital picture frame, a TV screen, or simply stay on the fridge, and support the visibility of goals for the rehabilitation of walking. CONCLUDING REMARKS.

In the above sections we detailed how we explore the design space offered by the 3 aspects of continuity in the physical rehabilitation process of senior citizens. We took our motivation from the definition of the successful rehabilitation by the Danish white paper [8]. We discussed how the recommendations of the white paper represent a call for facilitating the notion of continuity in the rehab process, as experienced by the therapists and the senior citizens. Then we detailed, and discussed, our early experiences in co-exploring MagicMirror and Walky, highlighting how the designs facilitate the aspects of selfmonitoring, collaborative articulation of the rehab process, and integration of the rehab process as part of everyday activities. We finally conclude in reflecting on our contributions: firstly, by providing concrete proof-ofconcepts in the forms of MagicMirror and Walky; and secondly, by opening up the design space for facilitating continuity and articulating a new way of digital technology in support of physical rehabilitation.

Design Rationales: Facilitating Self-monitoring and Integration of Rehab with everyday activities

In the following paragraphs we reflect on the features of Walky that facilitate self-monitoring, and integration of the rehab process with other everyday activities. Facilitating the Aspect of Self-monitoring

Walky provides, both direct and indirect, feedback for facilitating self-monitoring. Because of the ability to monitor their own walking activity, senior citizens can learn to be experts of their walking outside the rehabilitation center. • The haptic feedback from Walky initiates an immediate reaction from the senior citizen and correction of the walking posture. Here we piggyback on everyday artifacts (the walker handle) to give immediate feedback as facilitating self-monitoring.

MagicMirror and Walky: Two Proof-of-Concepts

As described above, we went about developing the sketches in-situ, with the therapists and the senior citizens as stakeholders and co-designers. The design of the sketches evolved from paper sketches to hardware manifestations in a co-design setting, where senior citizens and therapists evaluated the ideas and specific features as the process moved forward. In addition, the process was set in the actual site of possible usages: the rehab centre, the living rooms of the senior citizens, and outdoor parks, thereby bringing forward a more direct and relevant evaluation of the concepts. Thus we contribute to the field of HCI and interaction design research with two proof-of-concepts, in the forms of MagicMirror and Walky. They represent ways of digital technology facilitating concrete rehab exercises of upper body rehabilitation, and walking, respectively.

• Walky offers an opportunity for the senior citizens to set their rehabilitation goals in a more relevant manner, and together with the therapists, reflect and decide how much they would like to walk per day to achieve this goal. Having a visual reminder of their goal can be a stimulus and a motivation. The walking progress can be quantified using the progress towards the goal as a tool to exchange information between the therapist and the user. Through facilitating the aspect of self-monitoring in the above ways, Walky aims to add meaning while keeping the original meaning of walking to the nearby shop, for instance, intact.

Digital Technology facilitating continuity in rehab process

Furthermore, the MagicMirror and Walky both point towards a new role of digital technology in the rehabilitation of senior citizens. By facilitating information exchange between home and clinic, MagicMirror enables both the therapist and the senior citizens to engage in a more collaborative articulation of the rehab process. The Walky enables information exchange between everyday activities of walking, the senior citizen, and the therapist, thereby, integrating the rehab activity as part of everyday activities involving walking. Both the explorations facilitate the aspect of self-monitoring by providing immediate

Facilitating the Aspect of Integration of the rehab process with other everyday activities

Walking is in itself an activity of everyday life. Age and impairments often reduce the confidence and the ability of the walking activity. Walky and PocketWalky are focused on integrating the rehabilitation exercises of walking as part of the walking one does in everyday life, outside the rehab center. • While walking to the shop, or going to the park, Walky, by providing real-time feedback, gives the opportunity

50

Full Papers

Proceedings: NordiCHI 2010, October 16–20, 2010

possibilities for the senior citizens to monitor their progress, giving them a more direct control and active role in their rehab process. Thereby with these explorations we move towards articulating a new role for digital technology in the rehab process: to facilitate the notion of continuity in the rehabilitation process, as experienced by the therapists and senior citizens. This is realized by designing digital technology to address the facilitating the aspects of: selfmonitoring, collaborative articulation of the rehab process, and integration of rehab activities with everyday activities.

4. Buxton, B. (2007) Sketching User Experiences: Getting the Design Right and the Right Design. Morgan Kaufmann Publishers Inc , USA 5. Ehn, P. (2002) Neither Bauhäusler nor Nerd: Educating the Interaction Designer. In Proceedings of the 4th Conf. On Designing interactive Systems: Processes, Practices, Methods, and Techniques (London, England, June 25 28, 2002). DIS '02. ACM, New York, NY, 19-23. 6. Fallman, D. (2008). The Interaction Design Research Triangle of Design Practice, Design Exploration, and Design Studies, Design Issues, Vol. 24, No. 3, p. 4-18, MIT Press.

Future Work

While in this paper, we specifically addressed how digital technology can facilitate the first three aspects of continuity, we in future, plan to address the fourth aspect of facilitating social interaction. Further explorations are planned to explore the role of technology in providing openings for social interaction amongst a local group of senior citizens, by taking advantage of the social character of rehabilitation activities, both, within and beyond the rehab centre.

7. Fergus, P. Kafiyat, K. Merabati, M., Taleb-bendiab, A. El Rhalibi, A. (2009) Remote Physiotherapy Treatments using Wireless Body Sensor Networks. IWCMC2009, Leipzig, Germany, June 21-24, 1191-1197. ACM, USA. 8. National whitepaper on rehabilitation: http://www.marselisborgcentret.dk/fileadmin/filer/hvidb og/hvidbog.pdf. 9. Sokoler, T., Löwgren, J., Eriksen, M. A., Linde, P., and Olofsson, S. (2007). Explicit interaction for surgical rehabilitation. In Proceedings of the 1st international Conference on Tangible and Embedded Interaction, Baton Rouge, Louisiana, February 15 - 17, pp117-124, TEI '07. ACM, USA.

With Walky, and “where I walk?”, we have started exploring the possibilities of “notice and be noticed”: have the feeling of what’s happening in the local community of senior citizens, by receiving notifications when there is a friend out walking, for instance, and sharing small bits of information about walking activity with the local community. With MagicMirror, we aim to explore the idea of MagicWindow, where senior citizens can notify their friends when they start exercising using the MagicMirror, thereby inviting implicitly their friends to be part of it.

10. Svensson, M. S. and Sokoler, T. (2008). Ticket-to-talktelevision: designing for the circumstantial nature of everyday social interaction. In Proc. of the 5th Nordic Conference on Human-Computer interaction: Building Bridges (Lund, Sweden, October 20 - 22, 2008). NordiCHI '08, vol. 358. ACM, New York, NY, 334-343

ACKNOWLEDGEMENTS

We would like to thank the therapists of Tårnby rehabilitation center, specifically, Hannes and Majbritt, and the senior citizens, Borgild and Anita, for their time, enthusiasm, and precious contributions during the coexplorations. We thank two master students, Louise Miller (IT University of Copenhagen) and Pragati Kapur (IDC, IIT Bombay), for their contributions in designing and sketching Walky. We also thank the reviewers for their constructive criticisms on the paper.

11. Tanner, L. (2008) Doctors use Wii games for rehab therapy. Associated Press, 2 2008 12. Weiss, P. Rand, D., Katz, N. Kizony, R. (2004) Video Capture Virtual Reality as a Flexible and Effective Rehabilitation Tool. Journal of Neuro Engineering and Rehabilitation. Volume 1, issue 12. 13. Wolf, T. V., Rode, J. A., Sussman, J., and Kellogg, W. A. (2006) Dispelling "design" as the black art of CHI. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (Montréal, Québec, Canada, April 22 - 27, 2006). R. Grinter, T. Rodden, P. Aoki, E. Cutrell, R. Jeffries, and G. Olson, Eds. CHI '06. ACM, New York, NY, 521-530.

REFERENCES

1. Albaina, T. Visser, C. A.P.G. van der Mast, M. H. Vastenburg (2009). Flowie: A Persuasive Virtual Coach to Motivate Elderly Individuals to Walk. In Proc. Of 3rd Int. Conf. On Pervasive Computing Technologies for Healthcare (PervasiveHealth). London, UK, April 1-3, 2009.

14. Yu, H., Spenko, M., and Dubowsky, S. (2003). An Adaptive Shared Control System for an Intelligent Mobility Aid for the Elderly. Auton. Robots 15, 1 (Jul. 2003), 53-66.

2. Alankus, G. Lazar, A. May, M. Kelleher, C. (2010) Towards Customizable Games for Stroke Rehabilitation. CHI2010, Atlanta, USA, April 10-15, 2113-2122. ACM, USA.

15. Zimmerman, J., Forlizzi, J., and Evenson, S. (2007) Research through design as a method for interaction design research in HCI. In Proceedings of the SIGCHI Conf. on Human Factors in Computing Systems (San Jose, California, USA, April 28 - May 03, 2007). CHI '07. ACM, New York, NY, 493-502.

3. Binder, T, and Redström, J. (2006) Exemplary Design Research. Accessed on February 17, 2010, from: http://eprints.sics.se/920/

51

Full Papers

Proceedings: NordiCHI 2010, October 16–20, 2010

4Photos: A Collaborative Photo Sharing Experience Martijn ten Bhömer1, John Helmes2, Kenton O'Hara2, Elise van den Hoven1 Department of Industrial Design1 Microsoft Research Ltd.2 Eindhoven University of Technology 7 JJ Thomson Avenue Den Dolech 2, 5600 MB, Eindhoven, CB3 0FB, Cambridge, the Netherlands United Kingdom [email protected], [email protected] [email protected], [email protected] ABSTRACT

offer new ways for people to store, organise, tag and manipulate photographic content. Digital photo frames and photo printers create different opportunities for the way photographic content is presented in domestic settings. Of significance too are the increased possibilities for sharing our digital photos with others through established CMC channels such as email and perhaps more importantly through the appropriation of online social networking sites such as Facebook. Alongside these shifts, there has been a growing interest among the HCI and design community in emergent practices and values arising from the adoption of these new photographic technologies within domestic settings (see Lindley et al. [13] for an overview). Curiously though, as a number of authors observe, there is a disparity in the focus of this work whereby technologies and behavioural practices around photo displays have been relatively underexplored compared to other aspects of photowork [5,19,20]. Of particular significance to their arguments are concerns with the material properties of photo displays that form the essential components through which social practices, values and meanings are realized. Understanding this relationship between material properties and behavioural practices with respect to photo displays can then provide us with a resource for design. By manipulating the material characteristics of photo display and their setting we should be able to introduce new dynamics into the discussions and other behaviours that happen around these.

In this paper, we describe the iterative design and user study of “4Photos”, a multi-screen table centrepiece allowing media content to be shared and enjoyed in a social setting. It was our intention to design an object with the purpose to gather qualitative data concerning the social effects of new ways of democratic, serendipitous and playful photo sharing. To facilitate this we used online photo repository content that most often gets experienced in an individual setting. Using 4Photos we positioned this content within a social setting and observed how the presentation of these images enabled new ways of „phototalk‟ to arise. We describe the design process, the final concept and reflect upon observed practices that emerged from people‟s usage of 4Photos. We then present several design implications and discuss future directions for continuation of this research. Author Keywords

Information interfaces, Photo sharing, Interaction, Serendipity, Collaboration, Screens, Iterative design. ACM Classification Keywords

C.0. Computer Systems Organization: General – Hardware/software interfaces. H5.m. Information interfaces and presentation (e.g., HCI): Miscellaneous. H.5.2 [Information Interfaces and Presentation (e.g. HCI)]: User Interfaces - Input devices and strategies (e.g., mouse, touchscreen), User-centered design. INTRODUCTION

With the major proliferation of digital photo technologies, we have seen gradual and significant shifts in domestic photographic practices. Digital capture technologies in the form of dedicated devices and mobile phone based capture have opened up photographic capture to a much wider range of the population [5,18]. Other forms of photoware [6]

With this in mind we present an example of a new form of photo display, 4Photos, designed to be the centrepiece for a dinner table. In designing for this context, we make a number of design choices intended to introduce different social dynamics into the talk constructed around displayed photos. In particular, through our manipulations of display form factor, interaction access points and mechanisms for sourcing content, we hope to affect some of the subtle control dynamics of phototalk achieved with other forms of photographic presentation (e.g. paper or laptop). In this paper we present a discussion of the 4Photos design and then examine the impact of these factors through behavioural analysis of the system in use.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. NordiCHI 2010, October 16–20, 2010, Reykjavik, Iceland. Copyright 2010 ACM ISBN: 978-1-60558-934-3...$5.00.

52

Full Papers

Proceedings: NordiCHI 2010, October 16–20, 2010

new display technologies. Most commercial digital photo frames, for example, cycle through a selection of images meaning that particular location for display within the home need not be so constrained as by their physical counterparts.

Related work

Before discussing the design of the system, we first wish to build up the context for our arguments with reference to the literature. Work by Chalfen [2] has highlighted ways that domestic photography is a socially engaged practice through which family norms, traditions and values can be expressed and maintained. Such practices are reflected in activities such as the display of photographs in the home and the talk that is performed around photographs. These ideas are seen too in some of the foundational work in HCI and CSCW on photographic practices such as the work of Crabtree et al. [4] and Frohlich [6] on phototalk during the collocated sharing of photographs within domestic settings. Crabtree‟s work is particularly important in demonstrating the interactionally embodied nature of phototalk around printed photographs as revealed through ethnographic observations and analyses of photo sharing practices. Significant to our concerns in this paper is how Crabtree demonstrates the collaborative construction and coordination of phototalk as a practical achievement, realised through the way social groups gather round the photographic representation, the way the physical printouts are oriented to group members, passed around and gestured towards. While this study focused on the paper-based photographic representations predominant at the time, it is nevertheless significant in the ways it relates articulation and interaction work taking to the material properties of the representational artefacts. A key theme from this work is the issue of conversational control and its relationship to the representational artefact. For example, ownership of the physical photograph bestows a prominent role within the phototalk that occurs around it. In part this is about content ownership but also in the timing of photo distribution and physical orientation of the photos to others in the conversation [4].

Related to this and also the issues of control discussed above are notions of curatorial control over what is displayed. Durrant et al, among others, argue that curatorial control over placement of photos has traditionally resided with the family member in charge of the photographic technology [2,5,16]. However, with the apparent democratization of domestic photography in terms of capture and online sharing we again see shifting dynamics of curatorial control throughout the rest of the family. While there is increased democratization in display activity here, content of the photos displayed in the home remain closely bound to the owners of the space – namely the immediate family members that reside there. This too affects the dynamics with which such photos come to be used in phototalk on particular social occasions. Authors such as Kim and Zimmerman have highlighted alternative scenarios that may be of interest too [9]. For example, their work considers the scenario of visitors coming into the household. In response to this, they discuss a display concept that enables context sensitive presentation of photographic materials in response to particular visitors leading to potentially new dynamics in phototalk on such occasions of a visit. In this paper we want to explore other ways of shifting curatorial control over content to be more democratic (e.g. among a group of visitors to the home). Where we have seen interesting innovations in this area of photo display e.g. [9,20] what is curious is the relative lack of exploration in terms of their emergent behavioural properties arising from deployment in real world setting. A notable exception here is the recent work of Leong [11] which explores the use of a digital photo frame within a domestic setting. The particular concern of Leong was with the notions of serendipity arising from the random presentation of multiple images from a domestic photo collection. Leong demonstrated that abdicating choice of photographic presentation in this way, and the uncertainties associated to this, leads to new values and meanings relating to photographic display. We revisit some of these themes of choice abdication, uncertainty and serendipity in the discussion of our own fieldwork later in the paper and in particular how it relates to collaborative photo sharing experiences as opposed to the individual experiences that were the focus of Leong‟s enquiry.

The material properties of photo display in relation to behavioural practice and values are considered further in [19,20]. Again while this work was conducted with more traditional paper based photographic displays, it draws out key themes useful to our arguments. First is how social meaning is created through the placement of photographs in particular spatial locations around the home (a theme that is also developed in Crabtree‟s discussion of coordinate displays within the home [3]). Different meaning becomes associated with different household locations. Different locations too affect how photos are available to use in particular social circumstances, shaping the opportunities for use in photo talk – their availability to hand is bound up in the material qualities of their display. Critical to this meaning making with printed photographic materials is that physical display space is a finite resource. If a printed photo occupies a particular location, that location is no longer available for the display of another photo. As a consequence, the choice of photo for a particular location within the home acquires particular significance. Yet such dynamics potentially are shifting with the introduction of

Generally speaking, where innovation has occurred in the area of photo displays, there is often a strong grounding in the form factor of traditional displays. Many are designed to be positioned on wall, sideboards, shelves and mantelpieces in much the way that traditional photo frames are. The aim here is to augment the traditional with digital capabilities in terms of the way content is sent to be managed and displayed. As such, while location is a strong theme to emerge from the photographic literature little attention is

53

Full Papers

Proceedings: NordiCHI 2010, October 16–20, 2010

given over to consideration of alternative locations of display in design. This then is something we aim to address in this paper where we present a concept specifically for a different location, namely the dinner table.

fits neatly to a group defined by a location where the photographic display is placed. Indeed in light of different public private boundaries in different user segments raised by Miller and Edwards [14] with respect to online photosharing practices with sites such as Flickr and Facebook, it is likely that related concerns might be apparent in the linking of photo displays to online photo repositories in social networking sites. This is a question we consider further through our design and fieldwork with 4Photos.

This leads us to consider explorations of tabletop displays for presenting and managing photographs e.g. [1,10,17]. While these have yet to be adopted in any meaningful sense in domestic settings, they at least point to new ways of configuring people around photographic arrangements that can impact on the social context in which phototalk is conducted. This is due to the way the table arranges people around the photo display but also in the way that it opens up access points to all around the table through the use of multi-touch input mechanisms. These factors relate to our design concerns in this paper though we take a somewhat different approach to these issues. Tabletop computing typically treats the whole surface as the focal point for interaction and in this respect the photowork conducted is something that is a focal activity rather than just a component in a social gathering. Our approach, then, is to create a photo display to be situated on a tabletop where it is but one component in the assembly of artefacts making up the social setting, such as a dinner party. That is, it is designed as a tabletop ornament. In this respect, our aims are to exploit the ways a table physically and socially configures people for a particular social gathering but without requiring photowork to become dominant over other aspects of the social occasion and over other artefacts assembled on the table to support these occasions. This approach is in line with arguments about appropriate balancing of interactive and non-interactive aspects of tabletop assemblies [15].

Having discussed some of the factors shaping phototalk and display behaviour such as form factor, interaction access points, location, curatorial control, and mechanisms for sourcing content we now move on to a discussion of the 4Photos concept and ways in which we have manipulated these factors to achieve a more democratic phototalk appropriate to a social gathering. DESIGN APPROACH AND CONCEPTS

As mentioned within the previous sections it was our intention to design a research vehicle for exploring photoviewing activities in social contexts and locations not currently supported by traditional displaying devices. To prevent ourselves from exploring an unlimited amount of different social settings we decided to specifically focus at the social context of having dinner. We were particularly interested in this social context as it offers family and friends to come together for social interaction and presents interesting challenges in relation to social dynamics of talk. To this end, we initially explored a variety of different ideas and concepts. For example, we explored a concept called “media jar” which stores pictures and sound clips captured by mobile devices carried around by family members. This media is triggered on lifting the lid providing a resource for conversation. In a second example concept, the “media vase” mobile phones are docked in the “vase” to create a multi-view synchronised slideshow on the phones‟ displays. A third related concept we explored was called “photo centrepiece” in which photo viewing takes place on a dedicated device in the middle of the dinner table (see Figure 1).

A final line of discussion we want to introduce concerns the sourcing of content and how this potentially relates to shifting dynamics of phototalk. We have already seen how new photowork mechanisms such as online sharing are altering the dynamics of participation in the domestic use of photographic representation [5]. Given this more democratic participation enabled through these mechanisms (e.g., social networking sites such as Facebook or dedicated photo sharing sites such as Flickr), of interest for innovation and behavioural understanding concerns the relationship between these new resources and repositories for photowork and subsequent display behaviour. Indeed how can such democratic participation in online photo sharing be exploited in more democratic photo displays within particular social contexts, as repositories for digital photo displays. That is how can people‟s online photo resources be combined together for presentation on a photo display to shift dynamics of curatorial control and phototalk. There are of course now commercially available photo display systems that can be linked to online photo sharing and social networking. But we have very little understanding of the behavioural consequences and social values of this. For example, it is yet to be determined whether content designed for a particular online audience

Figure 1. The photo centrepiece concept. Apart from presenting media content it could also have a decorative function.

54

Full Papers

Proceedings: NordiCHI 2010, October 16–20, 2010

Design goals

within our iterative design process. After several design iterations, mainly focused on its interaction features and form, the process led us to the final prototype; “4Photos”. In the following sections we describe the iterative design process from the photo centrepiece towards 4Photos.

Through these conceptual explorations we were able to identify and refine particular design goals pertinent to our focal scenario. In light of these goals, it was felt most suitable to pursue the photo centrepiece concept in more detail. We discuss this further below.

Photo centrepiece

Non-obtrusiveness

The photo centrepiece prototype was designed to be positioned on a dinner table. A dinner table is designed to naturally configure people to gather for social occasions like dinner and drinks. The aim of the photo centrepiece was to exploit this natural configuration and provide a way for everybody around the table to view and interact with the artefact while maintaining natural gaze and interpersonal awareness (following recommendations of Lindley and Monk [12] whereby displays for photo viewing should afford good interpersonal awareness). To support this, the photo centrepiece prototype was composed of 4 vertical sides, each side framing a small portrait oriented display and an IR distance sensor (see Figure 2).

The aim was to let the technology not become dominant in the social context, but rather blend in the social dynamics of the moment. Especially given the fact that we focused on the social context of people having dinner, it was important to carefully take family values and dinner rituals in consideration. It was important too not to interfere with the physical assembly of artefacts found in this scenario (e.g. food dishes, plates). In fact, it was our intention to create new possibilities in order to enrich the social experience and band between people. Democratic control and participation

Embedding the artefact in new social contexts required a different approach to the form and interaction design in contrast to traditional photo-displaying products. We wanted the artefact to enable multi-user viewing and participation for family and friends and provide focus on the social enjoyment of the experience. As mentioned before, to offer new ways for democratic participation.

The artefact is to be placed in the middle of the table as one component in the artefact assembly making up a dinner table – much in the same way one might use a candelabra or flower arrangement as a dinner centrepiece. In this respect it needed to be a light and mobile artefact that would integrate easily with this artefact assembly. With this in mind we adopted a vertical display arrangement, which is more natural for people‟s gaze and mutual awareness during the social context of a dinner compared to a horizontal tabletop setup in which people look further down. Bearing in mind the additional criteria for flexible and rapid iteration, the photo centrepiece was developed as shown in Figure 2.

Easy access to new content

Based on insights from Kirk et al. [10] it can be concluded that people like to use recent content to spark conversation and storytelling. However, this work also reflects issues concerning the high threshold for downloading new content from a capture device. By focusing on an existing media repository, in our case Facebook, we prevented the introduction of yet another object to „maintain‟ or collect content for.

Interaction mechanisms

We implemented two interaction mechanisms in the prototype. The first used IR proximity sensors. Being positioned on all four sides of the object, these sensors could detect gesture and movement towards the displays from each side. The second interaction mechanism was a rotatable part on top of the prototype, the “head” (see Figure 2).

Random but related

Using online photo repositories we did not merely want to present content in a similar fashion as pre-arranged on these services. For this reason the object had to stimulate a more serendipitous experience, as also described in the work of Helmes et al. [7] and Leong [11]; wherein serendipity contributes to the quality of the experience. In order to achieve this experience but still maintain the possibility for „story telling‟, the artefact had to pick random sets of related images from any album within the repository. DESIGNING 4PHOTOS

Based upon the design goals and early physical explorations we iteratively designed towards a prototype for the photo centrepiece concept. This initial prototype was meant for exploration of the interface and a basic interaction with materials chosen to speed up prototyping rather than focussing on the necessary aesthetics for fit within the domestic environment. It maintained some flexibility to support exploration of different sensors and several interfaces and was robust enough for early user evaluation and allowed us to gather valuable input for the next steps

Figure 2. Picture of the photo centrepiece prototype with a rotatable part on top, called the “head”. Approx. size: 12x12x25. cm

55

Full Papers

Proceedings: NordiCHI 2010, October 16–20, 2010

spark curiosity and conversation amongst users (see Figure 5). Interaction was limited to a zoom function triggered by moving a hand towards the IR sensors (the spinning top part was not used). In an early test, during a family dinner, the object was perceived as very unobtrusive and passive as desired. At the same time though it was considered little more than a picture frame with moving images. One articulated reason being the fact that the random presentation of photos lacked any coherence to facilitate story telling or meaningfulness. A second reason was that the zooming function was not perceived as very useful - the size of all photos on each side of the object increased simultaneously, which simply made it impossible to highlight or talk about one particular photo. Finally, it was evident users wanted control in order to refresh the object‟s content.

Figure 3. Viewing angles of the object, top view 1 shows the original layout of the photo centrepiece. View 2 shows the new layout in which the object is stretched and displays rotated. 4PHOTOS FINAL DESIGN

Through initial testing of the photo centrepiece in tabletop configurations we discovered viewing angles of the displays were not optimal from a number of different directions. The unsymmetrical distribution of the viewing angle, did not lend itself well to typical configurations of people around the dinner table. To overcome this we changed the shape of the prototype (by slightly stretching the original shape) which led to the final form factor of 4Photos. We also rotated and slightly tilted the displays to create a homogenous viewing field (see Figure 3). In iterating towards a new prototype we also wanted to create a more suitable aesthetic for the domestic environment, drawing inspiration from other products and concepts fitting within this setting. This resulted in the final form factor prototype displayed in Figure 4.

Figure 5. Illustration of the first interface. Random photos move around across the displays Iteration 2: Moving and horizontal photostrip

With these issues in mind we introduced several changes. First, in order to allow users to select between content from different people they were able to rotate the head of the device to change the album from which photos were drawn. Second, we changed the zooming function to result in the image, on the side of the triggered IR sensor, to zoom in and push the same image to the other three screens– providing users with a coordinating function to highlight and talk about a photo. Third, we introduced an additional feature by maintaining a more structured and meaningful presentation of the photos within a “photostrip”, clustering multiple images from a single person. With this photostrip the four displays create a linked display surface which the photostrip moves around, facilitating storytelling by automatic and continuous cueing [8] (see Figure 6).

Figure 4. Photo of the 4Photos prototype with the rearranged displays, domestic aesthetic and rotatable top part. Interface design

We designed the interface of 4Photos in parallel with the physical form. The next sections describe the iterations and choices we made that lead to the “photostrip interface”. Iteration 1: Random photos floating around

The goal of this initial interface was to create a serendipitous photo viewing experience, displaying random photos of people gathered around 4Photos. These random photos appeared on the sides of the object and started to float around – the photos being different on all four sides to

Figure 6. The photostrip variation of the interface. The photostrip rotates around the 4 sides of the object.

56

Full Papers

Proceedings: NordiCHI 2010, October 16–20, 2010

Iteration 3: Final improvements for testing

Group 4: Gathering of three female friends aged 40-50. The daughter and son of the host are also present, aged 12-14. The meal is held at the host‟s home.

After implementing the previously described features and adding the „spinning‟ interaction (using the rotating top part), we again tested the object within a similar social setting as during our first user involvement. Based on the results from the user test we decided to maintain the interaction with the photostrip. We tweaked the interface to facilitate a better photo viewing experience. For example, we changed the order of the photos in the photostrip. In the previous iteration, after a spin, the photostrip would start again from the beginning of an album. We observed that people simply did not want to wait each time for 4Photos to go through the same images again before seeing new ones. In the improved version, the photostrip comprises a new set of photos each time a different person is selected creating a more random selection of new images.

Group 5: Group of student friends aged in their 20s. One female and four males having lunch at their university department.

Furthermore, as we so far tested with pre-uploaded images; which quite often lead to a lack of content, we decided to source the object with content from Facebook. Apart from having a large collection of available photos from this source, the Facebook API also allowed us to setup a special friends list to allocate the object to the photo repositories of the participants in the evaluation. As mentioned in the beginning of this paper, using Facebook to source content into 4Photos seemed a plausible implementation. Not solely to prevent people from having an additional device to populate with media, but also given the fact that currently Facebook content often gets viewed in an individual setting. We were quite interested as well in the social implications when projecting this content back into a social setting. Amongst other findings we will go into more depth about this topic within the next section.

Figure 7. Group 2 gathered around the 4photos at mealtime.

For each occasion, one of the attendees would create a Facebook friends list consisting of those people who would be present at that particular meal. The online albums of these people from this friends list would source the device with photos for that meal. The device would be placed on the dinner table among the other dinner paraphernalia and food. After a brief instruction on the basic interaction mechanisms, the groups would simply get on with their meal. All sessions were video recorded for later analysis. Results

Based on our analysis of the video recordings and guided by the design goals described earlier, we divided the results into six themes. We will illustrate these themes using examples and quotes.

USER EVALUATION

In order to further our understanding of behavioural practices around the device we conducted a series of qualitative tests in real world settings and contexts. The device was deployed on five occasions with different groups who were getting together socially for a meal. We deliberately chose to conduct these tests with different groups where different social relationships might play a role, such as, for example friends, couples, work colleagues, parents and children.

Display as a conversational resource

Situating the display within the social context of these social gatherings indeed provided a significant conversational resource for those present. SY “It‟s such a conversation starter.” The nature of these conversations varied and was dependent upon the make-up of social relationships in the group. We saw examples of shared reminiscing among the close groups of friends and examples where the photographs were used as a means to get-to-know about less well-known members, e.g. the work colleagues on the following conversation snippet:

Group 1: Colleagues from a research department having a meal together after work. Five males and one female all aged between 20 and 30. Meal is held in the kitchen area at the office. Group 2: Gathering of friends all aged 20-30. Three are males and two females. Two of the group are married and live in a shared house with another of the males. The other two are a couple who are visiting. The meal is in the home of the married couple and third male (see Figure 7).

P “Ah – there’s my daughter” [one of P‟s pictures is displayed up] J “Oh I didn’t realize you had a daughter.” P “Yes I have two. X is one and a half and Y is three and a half.”

Group 3: Family meal with husband, wife and two young children plus one female visitor. The husband and wife are in their 30s, the children are aged four and six. The visitor is in her 20s. The meal is held at the family home.

In some sense, these are fairly typical examples of phototalk that might be elicited through other forms of

57

Full Papers

Proceedings: NordiCHI 2010, October 16–20, 2010

SY “you pick up on random things like fish – you would never talk about fish in normal conversations.”

photographic display and presentation. However, there are subtly different properties and values that arise from the particular form factor and contextual situation in which the object is immersed that seemed to facilitate this. Of particular importance to people was the fact that the photo pool comprised photos from all present and not just the host.

As well as using individual photographs as triggers, there were times too when conversations made reference to multiple photographs from one particular photo strip. The clustering of the photos from a single album, then, allowed a more linked storytelling to take place

T “It’s taking it to a different level – it’s about everyone rather than one person’s photo.”

Coordinating views: physically, verbally, interactionally

While having different views on the screens provided more opportunity for triggering conversation, there were times too when participants wanted to see the same thing. A number of compensatory behaviours were observed to support this and allow smoother conversation. First, people would adjust their bodily position and gaze to view an adjacent screen. These movements would be in response to conversational or gestural based references to particular photographs on other screens. Typically, the nature of the viewing angles meant that people could only view at most two screens through small adjustments in body orientation (e.g. through leaning over); though occasionally more exaggerated body orientations were used to see additional screens. When bodily adjustments were difficult, participants also engaged in verbal based coordination behaviours.

This shifts the dynamics of control in significant ways, distributing both responsibility and opportunity for conversational control away from a single person. The photographs offered a means for everyone at certain points to make contributions to the conversation. But it also offered a means for people to invite others into the conversation through their questioning about the photographs – such as those who may have been shy and less able to get involved. Intrusiveness of the display

It is important to understand that conversation at the table was not simply about the photo content. Often, what started as a particular reference to a photograph would then evolve into its own conversational strand. People could exchange similar experiences and anecdotes with both weak and strong associative ties to the content. Conversations took on their own momentum where attention would shift from looking at the display to more “normal” face-to-face conversational gazes. That is, the device shifted in and out of conversational context throughout the meals, which shows that it supports storytelling. These dynamics are an important consideration for assessing whether the device is intrusive to the setting or shifts dynamics to facilitate the setting. In our observations, there were times when the displayed photos were very much a focal point of conversation – in particular when the phototalk had strong links to the content. But, equally, there were times when the conversation was less closely tied to the displayed photograph – either because of loose initial associative triggers or because of the natural progression of the conversational thread. In these instances the device seemed more unobtrusive and merged within the background.

C “Oh look at the turtle.” CO “Oh I haven’t got the turtle.” C “The turtle is coming round now.” CO “I’m not on the turtle. I’m on the bubbles.” Here, the group work around the “what you see is what I see” principle, using verbally mediated pauses in more of a “What you are about to see is what I see” principle. People also used the zoom function to coordinate conversation and shared views. This paused the content momentarily allowing more elaborated stories or closer inspection and reference to specific photographic. But it was also done explicitly to simultaneously push a particular piece of content to all four displays. Typically, the initiator of the conversation, being in front of the relevant display when the triggering photo was visible, performed this action. The timing of the interaction was used to draw attention to a particular photograph. But, it was also conducted in response to “questioning” looks or bodily gestures (e.g. leaning) from other participants.

People‟s orientation to the acceptability of this conversational influence was not consistent among groups or the different social contexts. All groups enjoyed conversations arising from the device placement and felt it was a great social facilitator. A couple of participants, though, made reference to the potential of the device to dominate talk. For informal settings and relationships this was not typically an issue but one participant expressed it may be too distracting for a more formal dinner party because of over distraction. Similarly, there were times in the family meal where the children were felt to be paying too much attention to the device and not focusing on meal time. Also valued was how the device created opportunities for conversational topics that would otherwise be unlikely.

Another factor that facilitated coordination of conversation across multiple views was that photos shown at any one time were from the same Facebook album. This provided a common contextual relationship for the photos imposed within the particular Facebook profile (e.g. from the same event or holiday). So, even when a conversation referred to a photo not immediately visible to all parties, this was mitigated by the common context of the photostrip.

58

Full Papers

Proceedings: NordiCHI 2010, October 16–20, 2010

Playing with visibility and invisibility

men to continue looking at the same picture set. A few minutes later when the men appear to have finished their discussion of the bike-related photographs SY asked: “Can we spin it or are you still…?”. M replied, “No no I think it’s done now”. SY responded, “Are you sure?”.

While having different images on each screen led to a need for coordination work, there were also playful and humorous behaviours arising from this. For example in Group 2, all the males were arranged on one side of the table with two females on the opposite side. Participant M said “This is where it pulls in content you didn’t want to show.” A picture of M then came up and male participant, T, pretended it was showing embarrassing content of M M‟s girlfriend, SY, being seated opposite. The men started teasing M and SY playing on the fact that SY couldn‟t see what they are viewing. “Who’s that you are with M – who’s that girl... Is that a strip joint you are in?”

In most cases, coordination of these multiple access points was successfully achieved and negotiated. Where it did present particular problems was when younger children were present. For example, the two young children in Group 4 were particularly taken with the interaction mechanisms themselves. As a consequence they continually tried to interact with the device with little awareness of its role within the conversation. Consider the following intervention by L, the mother of participant O:

In a second example from group 5, we see humour arising from ambiguity of conversational reference in relation to the displayed content. In this example, participant C pointed at the screen facing her, displaying a picture of her brother and said “Ah that’s my brother. Oh this is a really crap photograph of him really.” At this point, participant CO bursts out laughing and makes a reference to the image on her own screen saying “We’ve got Hugh Jackman round here. I should point out that’s not her brother.”

L “Don’t touch it because we haven’t seen these ones yet… You’ve got to stop!” O “But I like it” L“I know you like doing it but we want to see the photos… Was that a glass of cream or something?” K “It was a coconut.” We see here the repair work arising from O‟s inappropriate interaction, with references being made to an object that has gone by. So while the parents work around these issues through verbal references, there is some suggestion that more support for this repair work here could have been useful, the ability to bring a photo back that has passed by.

In these social contexts, then, issues of shared visibility go beyond matters of coordination, highlighting alternative sources of value and meaning making to consider in design. Managing multiple access points

The ability to interact with the device from all directions, be that spinning the head or the zooming sensors, opened up control to all. In all groups we saw interactions with the device by all present. The device was more inclusive in terms of availability of actions and resources participants had to steer and control the conversation. This differs from photo slideshows viewed on a laptop where affordances of the interaction mechanisms lend themselves to control by individuals in charge of the technology. Within this context of shared interaction and of concern to us here, were the ways access and control was managed through social protocols and also the conditions under which this broke down. With different images being viewed by multiple people at any one time, any change of view had to be negotiated with others for whom the photos were potentially a resource. Cues in conversational content and tone and attentional orientation to or from the device provided important cues in the timing of these interactions such that they could be performed with minimal social disruption. What was striking is how this mutual access and control was often conducted with seemingly minimal effort. But there were times, too, when more explicit attempts were made to negotiate access and control when there was conflict or insufficiently unambiguous cues from others. For example, in Group 2, the men were discussing a photo from a mountain bike-centric photo album. While looking at the photos, one of the women, SY, moved her hand to interact with the device. As she approached the device, her intentions became apparent prompting T to say „whoah whoah‟ - to stop her moving the photo on, as he wanted the

Serendipity and goal driven control

The random aspects of photo presentation provided important user value. People expressed joy and surprise as new photos came up. The uncertainty about which and whose photos would be next created anticipation among participants that was a source of fun. Indeed, a number of the groups compared the interaction to the game of “spin the bottle [a truth or dare type game]” and playful purposes. SY “I like the way it’s a lottery – who’s it going to land on. You could actually have drinking games with it couldn’t you. Every time it lands on you, you have to take a drink” Handing over some control over content to the system provided the kinds of values of serendipitous presentation highlighted in the works of Leong [10]. People were reminded of things they had forgotten: CO “It makes me think of all those albums that I forgot were on there.” Or things they had forgotten to tell people about: CO “It reminds you of things you forgot to tell each other” Giving control to the device also allowed the device to remain active during periods when participants were engaged in other conversation threads or activities such as eating. There were times, too, when people were more deliberate and purposeful in their interactions. In contrast to the large playful spins used to achieve random content

59

Full Papers

Proceedings: NordiCHI 2010, October 16–20, 2010

selection, slower controlled rotation of the device head was used to select specific people‟s photos or even a specific photo.

For example, in Group 2, one of the men had an album of mountain biking related photos. While these were of interest to the men, they were not for the women present. SY commented: “Why does it end up on bike porn?” Again, of significance is not that this caused any observed problems or frustrations, but how it reveals potential differences in uses and purposes of content in Facebook versus in the collocated setting. As such, Facebook needs to be considered in more subtle ways than simply a low effort way of sourcing content for the device.

ST “Can I find the one with the Octopus?” At other times, this deliberate control was to create more even distribution of content if it was felt someone‟s content was represented enough – thereby trying to include him or her in the conversation more. In one episode, SY spun it and it comes up with the same photo again – SY says, “Oh S you are popular” – then one of the men on the other side intervened and spun the top again to get new content. A short while later, S‟s photos appeared again so she nudged the device head to move to the next album.

At times the relationship between Facebook and the device became two-way. In general, people were happy with the low effort option of utilising existing Facebook albums. However, there were instances where participants put new content on their Facebook accounts specifically because they were going to be using the device. The device and scenario, then, created a new context of usage for Facebook content that shaped Facebook content management. People also spoke of content they would have liked to appear. One couple mentioned they would like to have seen their wedding photos on there as that would have been interesting for the gathered group. These early indicators of preparation activities for these settings suggest more imaginative and creative uses of Facebook might arise with longer-term usage. Of interest here was how people spoke of tagged photographs of themselves or the albums of others as a resource for their photographs. In this sense, people‟s photograph collections within Facebook were not seen as simply their own immediate collections. Rather Facebook was more of a distributed repository of photographs. As a work around, some participants suggested adding other people to the Facebook “dinner” friends list who were not present but who had relevant photo collections for those attending. For example, T said, “we should have put Millsy up there.”, making reference to the fact that “Millsy” had lots of good photos of their wedding.

Facebook content

In using Facebook photo albums as a photo source, our aims were to provide a low effort way of providing content; exploiting and re-using people‟s existing photo organisation work. While this was born out, a number of additional issues became apparent about the link with Facebook. One issue of note here is how the device changed the setting of consumption and the potential audience. These issues came back in user behaviour and conversation. For example, there were comments relating to uncertainty about which photos might pop up and if they would be appropriate. J “I hope there is nothing embarrassing on there.” The concerns arise, in part, because control settings placed on photos in Facebook do not always neatly map onto the collocated setting. The device is set up using a host member to whom the other guests are connected to on Facebook. But other members of the collocated setting may not necessarily be connected to each other via Facebook. Therefore, the audience for the photos in the collocated setting is potentially outside the bounds of control settings used for a particular Facebook friends list. These concerns of course are dependent upon the particular relationships that make up the social setting. With the above quote, the social gathering was with work colleagues where identity management may differ from that with well-known friends. We saw these concerns raised, too, in social groups with young children. The father in that Group 4 expressed a similar hope that no “dodgy” content would come up. The young children were not Facebook users - it is entirely plausible this would not be an uncommon feature of other social gatherings around the device. While there was little in the way of actual incidents of embarrassment during our observations, what is significant is how people revealed their orientation to these concerns.

CONCLUSIONS

In this paper, we described the iterative design and use of 4Photos, a collaborative photosharing device. Our aims with the device were to explore a new scenario of use of photo displays beyond the more traditional “wall and mantle piece” approaches of many current digital photo frames. We focussed on the creation of a table centrepiece that would be used during social gatherings around mealtimes. Within the context of this particular scenario, we wanted to explore a different set of social dynamics around phototalk by manipulating the material properties of the display device. In essence, we wanted to democratise the way that photos could be used in the construction of phototalk as part of a social gathering. We approached this issue in several key ways: making the device visible from multiple directions; by making interactions with the device available from all directions and available to all present at the table; and finally by making curatorial control over the content on the device more of a shared experience. Furthermore we wanted a device that augmented the

A related issue is with interest in particular content presented in the collocated setting. Within Facebook, content is sometimes posted that is of interest to a limited subset of people rather than a more general audience. As a solitary viewing experience, people choose which albums to look at in detail according to interest. However, the dynamics of such use changes in the collocated viewing.

60

Full Papers

Proceedings: NordiCHI 2010, October 16–20, 2010

mealtime experience and that fitted both with the natural assembly of other artefacts on the table and the natural configuration of people around the table. With these issues in mind in the design of 4Photos, we iterated through several different physical instantiations exploring different key aspects of the form factor. These aspects included the object‟s aesthetic qualities, its size, mobility and screen orientation. Similarly, we iterated through several implementations of the interface, focusing on key issues such as levels of control (active, passive use), structured vs. random presentation, playfulness, and the extent to which views were coordinated across the device. We further explored implications of sourcing content in different ways, in particular by exploiting photos from Facebook.

situated displays: Social and Interactional Aspects of Shared Display Technologies. Kluwer Academic Publishers, 2003, 422. 4. Crabtree, A., Rodden, T., and Mariani, J. Collaborating around Collections: Informing the Continued Development of Photoware. In Proc. CSCW 2004, ACM Press (2004), 396. 5. Durrant, A., Taylor, A.S., Frohlich, D., Sellen, A., and Uzzell, D. Photo displays and intergenerational relationships in the family home. In Proc. HCI 2009, British Computer Society (2009), 10. 6. Frohlich, D., Kuchinsky, A., Pering, C., Don, A., and Ariss, S. Requirements for photoware. In Proc. CSCW 2002, ACM Press (2002), 166. 7. Helmes, J., Hummels, C., and Sellen, A. The other brother: Re-experiencing spontaneous moments from domestic life. In Proc. TEI 2009, ACM Press (2009), 233. 8. Hoven, E. and Eggen, B. Informing augmented memory system design through autobiographical memory theory. Personal and Ubiquitous Computing 12, 6 (2008), 433. 9. Kim, J. and Zimmerman, J. Cherish: Smart digital photo frames for sharing social narratives at home. Work in progress CHI 2006, (2006), 953. 10. Kirk, D., Sellen, A., Rother, C., and Wood, K. Understanding photowork. In Proc. CHI 2006, ACM Press (2006), 761. 11. Leong, T. Understanding Serendipitous Experiences when Interacting with Personal Digital Content. PhD Thesis, University of Melbourne, 2009. 12. Lindley, S. and Monk, A. Social enjoyment with electronic photograph displays: Awareness and control. International Journal of Human-Computer Studies 66, 8 (2008), 587. 13. Lindley, S., Durrant, A., Kirk, D., and Taylor, A. Collocated social practices surrounding photos. In Proc. CHI 2008, ACM Press (2008), 3921. 14. Miller, A.D. and Edwards, W.K. Give and take: A study of consumer photo-sharing culture and practice. In Proc. CHI 2007, ACM Press (2007), 347. 15. O'Hara, K. Interactivity and non-interactivity on tabletops. In Proc. CHI 2010, ACM Press (2010), 2611. 16. Rose, G. Family photographs and domestic spacings: a case study. Transactions of the Institute of British Geographers 28, 1 (2003), 5. 17. Shen, C., Lesh, N., and Vernier, F. Personal digital historian: story sharing around the table. Interactions 10, 2 (2003), 15. 18. Shove, E., Watson, M., Hand, M., and Ingram, J. The Design of Everyday Life. Berg Publishers, Oxford, 2007. 19. Swan, L. and Taylor, A.S. Photo displays in the home. In Proc. DIS 2008, ACM Press (2008), 261. 20. Taylor, A., Swan, L., and Durrant, A. Designing Family Photo Displays. In Proc. ECSCW 2007, Springer London (2007), 7.

We conducted a qualitative user test with five different groups that provided more insight into how 4Photos was used in a variety of different social settings. It was evident that 4Photos functioned as a conversation starter, either because of the enthusiasm it evoked for people to tell about photos or the curiosity of others to find out more about them. It prompted people to find out about each other and exchange stories and also encouraged shared reminiscing among friends and family. Often the content also inspired people to talk about things they would never have talked about before; we ob-served how very mundane content sparked fun conversations. Apart from functioning as a conversational resource its interaction possibilities enabled a shared responsibility. By distributing its control and content sourcing, 4Photos shifted away from a more individualistic, single person interaction setup. This was further emphasized by the verbal and physical interactions that emerged around the different content on each display leading to alternative sources of value and meaning for the participants (e.g. jokes referencing content about to appear on other people‟s screens). The serendipitous presentation of content was also a source of fun for participants. We also started to identify interesting social issues regarding the relationship with Facebook as a source for content. It was apparent that using Facebook raised more issues beyond being a low-effort photo source. There were some interesting differences in the audience and context of use that emerged and that affected the way that 4Photos and Facebook was used. ACKNOWLEDGEMENTS

We would like to thank our colleagues from the CML group and in particular Abigail Sellen and Richard Banks. REFERENCES

1. Apted, T., Kay, J., and Quigley, A. Tabletop Sharing of Digital Photographs for the Elderly. In Proc. CHI 2006, ACM Press (2006), 781. 2. Chalfen, R. Snapshot Versions of Life. Bowling Green State University Press, Bowling Green, 1987. 3. Crabtree, A., Hemmings, T., and Rodden, T. The Social Construction of Displays: Coordinate Displays and Ecologically Distributed Networks. In Public and

61

Full Papers

Proceedings: NordiCHI 2010, October 16–20, 2010

Two-Handed Input in a Standard Configuration of Notebook with External Mouse Florian Block and Hans Gellersen Lancaster University InfoLab21, South Drive, Lancaster UK {block, hwg}@comp.lancs.ac.uk ABSTRACT

notebook computers, motivated by the observation that many users use these with an external mouse instead of the built-in pointing device. The result is a configuration of notebook and mouse (“N+M configuration” hereafter) in which two spatial input devices are available, in principle, although in current practice the built-in device is left unused once an external mouse is attached.

This paper discusses two-handed input for interaction with notebooks, motivated by the observation that notebooks are often used with an external mouse. We present results of a survey of 905 notebook users, of which 63.8% reported occasional, and 47.0% regular use of a mouse instead of the built-in pointing device (a touchpad in 95.8% of the reported configurations). Based on this finding, we propose use of the built-in touchpad with the non-dominant hand when an external mouse is used as primary pointing device. We provide a systematic analysis of the input space of such a configuration, and contribute a set of techniques that specifically exploit touchpad properties for input with the non-dominant hand. These techniques include flick, scale and rotate gestures; absolute positioning with tokens; and touchpad use as key modifier. The techniques are demonstrated in a variety of GUI applications in a standard environment of notebook with external mouse.

The observed use of an external mouse with notebook computers indicates a preference of many users for a mouse over the built-in device, for primary spatial input performed with their dominant hand. We propose that the built-in device can then be used beneficially with the non-dominant hand. A first benefit is that the built-in device and the mouse can be used simultaneously for two-handed input techniques, with asymmetric roles that naturally follow user preference of mouse over built-in device. A second benefit is that the built-in device can be used for customised input techniques, complementary to “general-purpose pointing” covered by the mouse in an N+M configuration. Based on these considerations, we make three main contributions in this paper: empirical work surveying notebook use to provide evidence that N+M configurations are common; theoretical work analysing the input space of such configurations to identify interface design opportunities; and design work contributing and demonstrating novel touchpad interaction techniques for notebooks used with an external mouse.

Author Keywords

Input Device, Touchpad, Laptop, External Mouse, TwoHanded Interaction, GUI. ACM Classification Keywords

H5.2. User interfaces: Input devices and strategies; Graphical user interfaces. INTRODUCTION

Two-handed input for spatial interaction with graphical user interfaces has been widely studied, with experimental evidence of performance advantages and cognitive benefits [5,15]. In practice, however, most interfaces for desktop interaction continue to be operated with a single spatial input device, most commonly a mouse, alongside a keyboard. In this paper, we consider two-handed input on

We have conducted a survey of notebook use with over 900 participants. The survey confirms our observation that external mouse use is common and widespread, with about two-thirds of notebooks occasionally used with an additional mouse, and half of notebooks regularly used in an N+M configuration. The built-in pointing device is most commonly a touchpad, available in 95.8% of the reported N+M configurations, and consequently the focus of our consideration of complementary input with the nondominant hand. The survey also provides qualitative insight into user preference of mouse over touchpad or other builtin pointing device. Understanding why a touchpad is less preferred as primary pointing device is relevant for our consideration of their use in an alternative role, complementary to the mouse.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. NordiCHI 2010, October 16–20, 2010, Reykjavik, Iceland. Copyright 2010 ACM ISBN: 978-1-60558-934-3...$5.00.

62

Full Papers

Proceedings: NordiCHI 2010, October 16–20, 2010

Based on the survey, we contribute an analysis of typical N+M configurations in which the input space is defined by the combination of mouse and touchpad, in addition to the keyboard. A pairing of mouse and touchpad has a precedent in work of Hinckley et al., where a mouse and an external touchpad were combined for two-handed tracking and dragging tasks [12]. However, in this work we look at the input design space of N+M configurations more generally, and contribute a detailed characterisation of N+M interfaces in terms of Card et. al.’s taxonomy of input devices [6].

inherently asymmetric, first in that the two devices have different and distinctive properties [4,12], and secondly in that users have an a priori preference for the mouse as primary pointing device. Guiard’s Kinematic Chain model is therefore of key relevance to our work as it provides theoretical insight into principles governing asymmetric two-handed interaction [8]. The model describes a distinct role for the non-dominant hand, in setting the frame of reference in which the dominant hand operates (“Right-toleft Spatial Reference in Manual Motion”). It further proposes that the non-dominant hand performs less finegrained movement (“Left-Right Contrast in the Spatial Temporal Scale of Motion”). In particular the latter point is significant for our purposes. Notebook users who prefer a mouse over a touchpad tend to do this because of (perceived) finer-grain control over movement (as evident from our survey, see below). Arguably, a (perceived) limitation in accuracy and speed will be less evident when two-handed techniques are designed such that the touchpadoperating hand performs less, and less fine-grained, movement than the mouse-operating hand. This corresponds with experimentally observed left-right-hand input performance, finding that the non-dominant hand is well suited for tasks that do not require precise action, such as scrolling [13].

Our theoretical analysis of the N+M input space gives rise to a number of novel interaction techniques that we contribute in the final part of this paper. The techniques we present are based on customisation of the touchpad, which becomes possible when its original function, cursor control in a graphical interface, is taken over by an external mouse. The techniques include flick, scale and rotate gestures, absolute input with a token on the touchpad, and touchpad use as an input modifier. These techniques are known in principle from other forms of touch interface (e.g., tablets, touchscreen, interactive tabletops) but contribute distinct new interaction styles when they are provided on built-in notebook touchpads. The utility of the techniques is demonstrated in a variety of applications that we have implemented in a standard GUI environment.

Two-handed techniques reported in the literature in which the dominant and non-dominant hand perform complementary roles can be broadly classified as: • Setting context: various studies of two-handed input directly apply Guiards’s reference principle by using the non-dominant hand for context-setting of graphical workspaces while the dominant hand performs finegrained actions (e.g., scrolling/selecting text [5], panning/rotating viewports for painting [15], and navigation in 3D design [22,24]). • Toolglasses: another use for the non-dominant hand in GUI environments is to move toolbars, palettes or menus to a point of interest, to bring them closer to where the mouse operates, or to resolve occlusions [14,15,17]. • Gestures: two-handed techniques have been demonstrated in which the non-dominant hands performs gesture input to select commands or modifiers in conjunction with spatial positioning [1].

RELATED WORK

According to sales trends, notebooks are on the brink of being more widely adopted for desktop interaction than traditional PCs, but the input configurations in which they are used have received only limited research attention. Prior studies include surveys of different user groups with respect to ergonomics of notebook use [10,21], comparative evaluation of built-in pointing devices [7,20] and ethnographies of notebook usage in the home [23], but we are first to present a survey that provides quantitative and qualitative data on adoption of an external mouse for notebook interaction. In spite of the evidently widespread use of an additional mouse for notebook interaction there has been no prior work considering use of the built-in pointing device for complementary or alternative input alongside the mouse. It is evident from other work on capacitive touch sensing surfaces over the last decades that touchpads can in principle support other forms of input, distinct from mousestyle pointing [4] and including gestures [1,18]. However, this potential has not previously been explored in the concrete context of notebook-integrated touchpads. An example though of providing a new technique based on a notebook touchpad is ThumbSense, where touchpadsensing is used as an implicit key modifier for emulation of mouse buttons on the keyboard [19].

In all of these cases, the dominant hand is used for finegrained positioning, which naturally maps to mouse operation in an N+M scenario, while the non-dominant hand performs actions that can be realised on notebook touchpads. The combination of touchpad and mouse has been previously investigated, however with a touchpad as external device [12], rather than integrated to be close to hand from the keyboard. It is important to note conceptual differences between mouse and touchpad in such a configuration, and that the devices operate in different kinaesthetic reference frames. However general techniques such as context setting and moving toolglasses can be adapted to work with relative input from both devices, or

The proposed “activation” of notebook touchpads complementary to an external mouse facilitates interaction with two continuous input devices, leveraging a large body of research on two-handed input. The configuration is

63

Full Papers

Proceedings: NordiCHI 2010, October 16–20, 2010

alternatively with the touchpad as absolute and the mouse as relative input device. Research also suggests that separation of kinaesthetic reference frames in two-handed interaction can be compensated with visual feedback [2].

N

%

1. What built-in pointing devices does you notebook have? Touchpad 867 95.8 Pointing Stick 51 5.6 Track Ball 16 1.8 Touch-screen 14 1.6 2. Do you ever use an additional external mouse with your notebook?

Two-handed desktop interaction in practice involves the use of keyboard shortcuts or hotkeys. Single hotkeys or a combination of modifiers and hotkeys (such as CTRL+C for copy) can execute application commands as shortcut for graphical widgets, and are known to generally save time compared to clicking GUI widgets [16,17], particularly when invoking functions from a menu [9]. Such interactions differ from the above examples of two-handed input as the keyboard is not a continuous input device, but they inspire consideration of how touchpads in an N+M configuration can support hotkey techniques.

Yes

577

63.8

No

328

36.2

3. How often do you use your external mouse with your notebook in the office / at home / while traveling? *

A SURVEY OF NOTEBOOK USE

Regularly

We have conducted a survey of notebook use with the primary aim to corroborate our observation that many users regularly use an external mouse in extension of their notebook configuration. In addition to quantitative data we sought to gain qualitative insight into preference for an external mouse over the build-in device, to inform the development of novel techniques that leverage N+M configurations.

Occasionally Never

338 / 393 / 83

58.6 / 68.1 / 14.4

110 / 165 / 134

19.1 / 28.6 / 23.2

115 / 46 / 359

19.9 / 8.0 / 62.2

* percentages based on number of users of an external mouse Table 1: Survey results on built-in poiting devices and external mouse usage (contains only selected items of the actual questionnaire).

The survey was conducted with an anonymous online questionnaire, to obtain data from a larger population. The questionnaire was designed to be short to maximize response rates. It contained 11 questions in 3 sections. The first section asked for basic demographic data. The second section assessed the user’s technical equipment (notebook, docking-station, built-in input) and their qualitative rating of built-in devices. The third section concerned the external mouse (where applicable), asked for a qualitative rating in comparison with the built-in input, assessed in which applications/situations the mouse was used, and also collected data on use of mouse features (mouse wheel, additional buttons etc.). The questionnaire contained multiple-choice questions and scaled questions, but users also had the option to enter comments.

questionnaire. Pressing submit would add the anonymous questionnaire data to our database. With a separate database we made sure, that users of shared IP addresses could not complete the questionnaire several times. We received a total of 989 submissions. Fifteen submissions were identified as invalid (either empty or corrupted). Another 68 participants were filtered out since they did not use a notebook computer (control question answered with ‘no’). There were no duplicate entries. 14 submissions were missing age, and 7 missed information on gender. However, all these surveys were otherwise complete, and thus included in the analysis. After filtering, we had 905 valid responses (10.1 % response rate). The total number of responses came from 964 different IP numbers. After an inspection of all datasets with a counter greater 1 we came to the conclusion that the datasets were unique, which is realistic given the fact that we expected shared computers / IP-addresses. Consequently we included these datasets into our analysis.

Sampling

Our sampling frame were students of a local university. Convenience sampling was then used to determine the final sample. A call for voluntary participation for the online questionnaire was advertised on the university mailing list, reaching a total of 8975 students, of which 5900 were undergraduate students, 1900 postgraduate master students and 1175 research students (MPhil/PhD). Ethical approval was granted by the university ethics committee.

Results

Demographics. Of the 905 participants 333 were male and 565 were female (7 participants did not provide their gender), with an average age of 23.0 years (s.d. 7.3y, youngest 17y, oldest 65y).

Data Collection

A campus-wide mailing was sent out to call for participation. The email was explicitly recruiting notebook users. Volunteers followed an anonymous link to the web questionnaire. The questionnaire was fitted on a single html page, with minor scrolling necessary to complete the

Assessing Built-In Input Devices. The by far most frequently found built-in input device is the touchpad (95.8%) followed by pointing stick (5.6%), track ball (1.8%) and touch-screen (1.6%, cf. Table 1). 2.2% of all

64

Full Papers

Proceedings: NordiCHI 2010, October 16–20, 2010

participants used a docking station and 3.0% use a notebook stand either in the office or at home. This is an important issue to consider, since both docking stations and notebook stands could render the built-in input impossible or difficult to use. Participants were also asked to rate their built-in pointing device in terms of efficiency, accuracy and comfort on a 5-point Likert-scale. In all three categories, the built-in pointing device received close to neutral ratings: efficiency scored 3.4 (s.d. 1.1), accuracy 3.6 (s.d. 1.0) and comfort 3.6 (s.d. 1.1).

This observation was confirmed by the explicit feedback users gave. Users were asked why they preferred the external mouse over their built-in input. They could choose either one of the preset choices (‘mouse is faster’, ‘more accurate’, ‘more comfortable’) and had the option to enter comments. 66.72% of all mouse users selected ‘faster’, 75.2% selected ‘more accurate’ and 76.1% selected ‘more comfortable’. The comments will be discussed in the next paragraph. Another item in the questionnaire assessed applicationspecific preference (“For which applications do you prefer to use an external mouse”). Participants had multiple choices between three basic categories (‘office’, ‘communication & information retrieval’, ‘entertainment’) and the option to enter specific applications in a text box. Of all external mouse users, 80.9 users selected ‘Office’, 61.5% ‘Communications/Information Retrieval’ and 77.0% ‘entertainment’. Users entered a variety of additional applications, mostly graphics software (such as Photoshop or CAD) but also video editing and sound composing.

External Mouse Usage. Of the 905 notebook users who participated in the survey, 577 (63.8 %) use an external mouse at least occasionally. An overview of mouse usage can be found in Table 1, items 2 and 3. Users were asked whether they made regular use of mouse in any three situations (in the office, at home, or while travelling). Of our total of notebook users, 47.0% are using an external mouse regularly in at least one of these settings. Of our subset of “N+M users”, 58.6% reported that they use the mouse regularly at the office, and 68.1% at home. Unsurprisingly, regular use is low for the mobile setting with only 9.2% (when travelling). Occasional use can be observed across all three settings (office: 19.1%, home: 28.6%, travelling 23.2%), including the mobile setting. This confirms that external mouse usage in a mobile context is less common, but it also shows that a considerable proportion of users carry a mouse with them when they travel.

User Comments. The comments provided by the participants of our survey gave more detailed insight into qualitative aspects of mouse and built-in input. There were 63 comments in response to ‘Why do you use an external mouse, instead of the built-in pointing device?’ Within these comments we found 24 occurrences of ergonomic aspects. Strain to the wrist was mentioned, as well as general fatigue over longer work-sessions. 4 users gave a more detailed response, stating the advantage of the external position of the mouse over the fixed position of the built-in input device. A conceptual disadvantage of the touchpad was mentioned, namely that ‘clicks’ could be accidentally triggered whilst typing. 20 participants mentioned the general ‘ease of use’ of the external mouse in comparison with the built-in input. Many of these comments included implied references to ergonomic factors (‘easier for the hand’, ‘mouse is bigger’) but also to accuracy (‘generally easier and more accurate’). 4 participants explicitly mentioned an improved efficiency with the mouse (‘quicker use’, ‘faster in certain situations’).

Mouse Features. External mouse users were asked which additional features they used. The multiple choices were mouse-wheel, additional buttons (such as the ones facing the thumb) and thumb-wheel (some mice have an additional jog-dial where the thumb rests). The most common feature used was the mouse-wheel (90.1% of all mouse users), followed by additional buttons (38.1%) and thumbwheel (16.6%). Qualitative Assessment of Input Devices. First, we analysed the user rating of the built-in pointing device. We grouped users into those who use an external mouse and those who do not. Our hypothesis was that users who use an external mouse would rate their built-in input device significantly worse than those users who are not using an external mouse. This would give us implicit evidence for the users’ motivation to use a mouse, which could later be matched with the explicit feedback we gathered. A Kruskal-Wallis Test was used to analyse differences of ratings for efficiency, accuracy and comfort. It confirmed our hypothesis, showing that there is a significant difference between our two groups, with p < .01 for all items, efficiency (χ2(1) = 68.2), accuracy (χ2(1) = 45.6) and comfort (χ2(1) = 80.2). Ranking in all three categories was lower (worse) for participants who use an external mouse. This indicates that by choosing an external mouse, users seek to improve all three aspects for pointer input.

A second set of comments (64 in total) concerned application-specific aspects. 24 participants stated that they use their external mouse across all applications. 20 participants highlighted the benefits of the mouse specifically for the graphics software, such as Photoshop, CAD or other design tools. Some users gave a more generalized explanation, related to the better ‘moveability’ or ‘manoeuvrability’. One participant stated: ‘[I use the mouse] for […] tasks which require more control of the cursor than the touchpad allows’. This was also supported by several statements that certain tasks are very difficult to perform with the built-in input, such as ‘creating diagrams’, ‘drawing pictures’, ‘creating graphics’ or ‘dragging and dropping’.

65

Full Papers

Proceedings: NordiCHI 2010, October 16–20, 2010

Discussion

The survey confirms that the proportion of notebook users who use an external mouse is indeed large, with 63.8% of surveyed notebook users using a mouse at least occasionally, and 47.0% using a mouse regularly. Dockingstations or notebook-stands, which could compromise or impede use of built-in input devices for novel techniques, are used only by a small fraction of users (2.2% and 3.0%). The survey method (convenience sampling) and population (university students) imply that we need to be careful in drawing conclusion with respect to notebook use in general. However the sample size is clearly significant and the results are strongly indicative of N+M configurations being commonplace as a distinct interface for desktop interaction.

Figure 1: Input space of notebook configurations with touchpad and external mouse (using Card et al.’s taxonomy [6])

The results also show that the touchpad is by far the most common built-in input device in notebooks (95.8%). Consequently, many of our considerations in this paper are specifically focussed on interaction with touchpad and mouse.

absolute position of one finger on the surface (the number “2” within the squares applies only to multi-touch sensing touchpads). Additionally, touchpads can determine proximity to a certain degree and thus give an estimate about the distance between finger and touchpad (while this input is unreliable for higher distance, it is essential for determining the point of contact). Most touchpads also have two co-located buttons, similar to a mouse. Using connect compositions (double arrows) one can derive also relative movement from the absolute positions in x and y.

A consistent motive for using an external mouse was a higher quality of input, in addition to ergonomics. This has many factors, most importantly accuracy, speed and expressiveness. These factors are specifically relevant for tasks performed with the dominant hand, where tasks performed by the non-dominant hand, such as context setting and toolglass placement, require less accuracy and speed.

Newer generation notebooks (e.g., the Macbook Air) have touchpads that can sense at least two fingers at a time. Even though this is not standard yet, we include these capabilities as they are likely to become commonplace. With two points of contact some of the mentioned primitives double (symbolised by “2” within the squares). Additionally, several new primitives can be derived (pentagons). The angle between two fingers can be mapped to rotation (rZ) and the distance between two fingers can be mapped to a new linear value. Since the described taxonomy does not accommodate for distances or scalars we interpreted the distance as a new linear value (the absolute relation between two positions). Its place along the X column was chosen arbitrarily. Both absolute distance and angle themselves can be connect-combined to their appropriate delta-values (e.g., taking the delta between distances could be mapped to scaling an object).

THE INPUT SPACE OF NOTEBOOK+MOUSE

Our survey has established that N+M configurations are common, and that in most of these cases the ‘neglected’ built-in device is a touchpad. Our input space is thus characterised as follows: 1.

The mouse is the preferred input device for the dominant hand, in the role of pointer input

2.

The keyboard is used for text entry and hotkey access

3.

The built-in input device is unused and can be treated as third independent input device

Figure 1 shows the input space of N+M configuration using the taxonomy of Card et al. [6]. Each input device is represented by a number of input primitives that are connected with different composition operators (merge, connect and layout). The mouse consists out of three input primitives (symbolised by circles): relative movement across X and Y, and 3 buttons (the number in the primitive reflects multiple primitives of the same kind). Based on our survey results, we also include the mouse wheel (triangle), which delivers a delta angle around the X axis.

Note that in the original design space, connect compositions were described as very basic mappings from one domain to another. The conversion from two points of touch to distance / angle is more complex than originally described, since it combines multiple primitives of different kinds and maps them to a single primitive in another domain (aggregation). However, we feel this is still within the spirit of the taxonomy, as it helps capture the complete set of input primitives that a device, in this case the touchpad, can support.

The keyboard provides a set of buttons (~90) and is mounted on the same panel as the touchpad (described by a layout compositor, symbolised by a dotted line).

The description of the N+M input space provides a baseline for understanding and exploring interaction opportunities. For our specific purposes, it allows us to systematically

The touchpad (squares) provides a range of primitives. On the most basic level, a conventional touchpad delivers

66

Full Papers

Proceedings: NordiCHI 2010, October 16–20, 2010

analyse potential use of the touchpad, in itself, and in conjunction with mouse or keyboard. TOUCHPAD TECHNIQUES FOR N+M CONFIGURATIONS

In the following we introduce a number of interaction techniques designed to leverage the touchpad in N+M configurations. The techniques exploit that general purpose pointing is performed with the mouse, allowing use of the touchpad to be customised.

Figure 4: A coin can act as physical token for absolute positioning, for instance of a palette or toolglass [3].

operate the touchpad while the fingers are in the home row for typing, eliminating the necessity to acquire and use the mouse for scrolling (Figure 3). This can be useful for example for looking up information in another part of the document before continuing to write (e.g., when crossreferencing). It can also be used to simply move the current page so that the cursor is in the centre of the screen (context setting for convenience was observed in [15]). With a common mapping (Y movement to scrolling) navigating across large text documents causes significant strain for the thumb. However with flicking, we found that we could comfortably use the thumbs to navigate within large documents. This example shows that the fixed location of the touchpad is not necessarily a constraint, but can in fact be advantageous.

Flick Gestures for Context Setting

The touchpad is well suited for context setting, both for technical and conceptual reasons. Technically, it provides highly responsive data on absolute finger position and proximity as well as relative movement in two dimensions, allowing expressive panning across virtual workspaces. Its input capabilities are supported by the touchpad’s flat character, affording unique interaction styles, such as flick gestures that have become popular with the iPhone. Figure 2 illustrates how flick-gestures can be applied to context setting in map navigation. In this example, the touchpad acts as direct handle for Figure 2: Flick gestures used for an underlying map: once the finger touches the navigating maps. touchpad, any movement is mapped directly to the panning of the map. When the finger is flicked off the touchpad, momentum is induced into the virtual map causing it to start spinning in the direction the finger has flicked into. The motion stops either by applying virtual friction over time or by pinning the finger down on the touchpad, once the map is at the desired location.

Absolute Input with Touchpads

Various examples of two-handed interaction such as context switching or for moving Toolglasses use absolute input. While the mouse is not able to sense absolute position, the touchpad can provide appropriate input primitives by sensing the finger’s absolute location relative to its boundaries. When applied to a practical scenario, such as moving Toolglasses, several observations can be made. On the one hand, absolut input gives direct and fast control over the Toolglass since it eliminates the continuous ‘pushing and re-orienting’ of the finger as is typical for relative interaction with the touchpad when used as pointer. On the other hand, using the touchpad’s absolute input felt counter-intuitive for several reasons. First, the touchpad’s established role for pointer input suggests mainly relative movement across the flat surface, not pointing to absolute locations. Secondly, it is also conceptually different from other absolute input devices, such as the puck used in [15], in that the touchpad lacks a physical token that would reflect the current absolute position.

This example demonstrates how the touchpad – once we can specialise its mapping – can make use of inherent affordances when it is mapped to a dedicated task (“spinning the globe”). In our example, the user might even prefer the touchpad for panning the map over the dominant hand, since it provides a more specialised interface for the given task than the mouse. Flicking is also efficient for interaction in that it can reduce the amount of finger activity necessary to overcome long panning distances. Within text-editors, flicking can be applied to horizontal scrolling, enabling new interaction styles. The touchpad’s position close to the keyboard generally allows the thumb to

Figure 4 shows, how this problem can be overcome in a simple manner, by adding a coin on the touchpad. A coin amplifies the capacitive fingerprint and a threshold can be set such that the controlled object (e.g., a toolglass, palette, or menu) would only be moved when moving the coin. The use of a coin (or other token) extends the touchpad with an important affordance when used as absolute input device, as the token is intuitive to use and gives a permanent physical indication of where the object is in respect to the screen. An added benefits is passive tactile feedback when the coin hits the borders of the touchpad. Moreover, it is possible to

Figure 3: The touchpad’s location allows the thumbs to flick through documents while leaving the fingers on the home row.

67

Full Papers

Proceedings: NordiCHI 2010, October 16–20, 2010

Figure 5: Scale and rotate gestures on standard touchpads using the button as second point of contact.

distinguish tokens of different size by their capacitive footprint. Scale and Rotate Gestures on Single-Point Touchpads

New generations of touchpads allow the sensing of multiple points of contact and thus support gestures for rotating and scaling (cf. Input Space). This can be benefitial for tasks such context setting (zooming in and out, rotating the view) and object manipulation. However, most existing touchpads do not support this, and can only track a single point of touch. To address this limitation, we contribute a technique for scale and rotate gestures with a single point of touch combined with a touchpad button.

Figure7: Simultaneous context setting with the non-dominant hand and free-form selection with the dominant hand.

(e.g., CTRL, ALT) is more centrally placed and larger, and thus easier to acquire. Standard hotkeys are binary triggers, however in combination with the touchpad new possibilities arise. For example, when a hotkey has been selected, movement on the touchpad can be used to parametrise the selected command. Figure 6 illustrates this with selection of a Brush tool by pressing touchpad and “B”, followed by thumb movement on the pad to manipulate brush size. As this example shows, touchpads can support entirely new types of hotkeys, that can be invoked and parametrised by one coherent action.

Figure 5 illustrates these techniques, in the context of map navigation. By taking a button as position of the thumb when clicking, the finger can move around the thumb either to perform scale or rotate gestures. The modes can be separated, clicking either left or right button for exclusive scaling/rotating, or holding both buttons together to achieve a combined effect. While this is a workaround for the missing second point of touch on the pad, it also has distinct advantages as it provides more control about what transformation the user intents to perform (e.g., achieving only rotation without scaling is difficult to perform with the dual touchpad, because the distance between two fingers would have to be kept constant).

APPLICATIONS

We have implemented several applications and applets that demonstrate the interaction techniques we designed for N+M configurations. The first one, a Virtual Desk application, is custom-built and directly taps into raw pointer input from the operating system, with global mouse hook to suppress all mouse messages and then implement our own GUI that can separately process touchpad and mouse input. The other applications extend existing software (such as Microsoft Word and Adobe Photoshop), with a background process that uses an API (Synaptics) to receive touchpad input separated from the primary pointer. Touchpad events are processed partly via injecting simulated mouse and keyboard events (e.g., simulating the mouse wheel for scrolling in Word), and partly via application specific API (e.g., the Photoshop COM API).

Hotkey Invocation and Parametrisation

Hotkeys are binary actions triggers and can be used as shortcuts for invoking applications via graphical interface elements such as icons or menus. Hotkeys often consist of a modifier and key, such as CTRL+C, allowing overloading of character keys with added functionality. The touchpad can add interesting aspects to hotkey interaction, as shown in Figure 6. When it is conceptualised as a single button (which is either touched or not touched), then it can be used as a modifier, that in contrast to conventional modifiers

Virtual Desk

This application was developed to provide a simple interface for distributing and organising large amounts of digital documents on a virtually infinite desk. Documents, such as PDF files can be simply dragged onto the virtual desk and are displayed by a stack of paper depicting a snapshot of its cover page. The large size of the virtual desk allows a user to spatially arrange and organise documents across large areas of virtual screen real-estate without introducing occlusions. Since as a result the virtual desktop

Figure 6: The touchpad can be used as a key modifier for hotkey selection combined with parametrisation in one action.

68

Full Papers

Proceedings: NordiCHI 2010, October 16–20, 2010

significantly exceeds the actual screen, frequent adjustment of the viewport is necessary, to navigate between distant documents, and when documents are selected and moved between distant locations on the virtual desk. For this purpose, the touchpad is mapped as a dedicated interface for controlling the viewport, while the mouse can simultaneously interact with objects, such as performing selection or dragging.

playback without leaving their typing position. As an alternative we have mapped audio control to the touchpad with a thumb tap for ‘pause’, a double tap for ‘play’ and slide with the thumb to ‘rewind’. The advantage of this technique is that it allows accurate playback-control, keeps fingers in typing position and does not require an additional device to be attached to the system. PhotoshopMagicLense

The touchpad interaction enables flick gestures for rapid panning across the table (cf. Fig. 2) as well as shrink / stretch gesture to set the zoom level of the viewport (cf. Fig. 5). This gives distinct advantages over the alternative of mouse and keyboard input. Figure 7 shows a typical problem that arises within virtual viewports: the user starts a free-form selection realising that not all documents that should be included in the selection are shown in the current viewport. Normally, once the mouse reaches the boundaries of the window, automatic scrolling is applied. However, since the mouse is in an active dragging state, scrolling automatically causes the mouse to leave a trace, which in our scenario could lead to the inclusion of unwanted elements that unexpectedly come into view. With the touchpad, the viewport can be independently adjusted with the non-dominant hand while the mouse can select revealed items in a controlled way. This is also superior to view adjustment via the keyboard, since it offers continuous spatial input instead of only binary controls.

We have implemented a toolglass for photoshop that can be positioned with the non-dominant hand. Positioning was realised in absolute terms by mapping touchpad input directly to a corresponding screen position. However the natural inclination of users for movement of the toolglass was to apply relative movement as familiar from pointing. To address this, we introduced use of a physical token into this scenario as described above. The token is intuitively perceived by users as representing the toolglass, and accordingly positioned and moved on the touchpad. DISCUSSION

A number of specific design issues were identified during the implementation of our techniques. First of all, the mapping of sub-task to either touchpad or mouse is not always clear. For instance, in the VirtualDesk application we mapped panning and zooming to the touchpad. On the one hand this enables the user to adjust the zoom level of the viewport while dragging or selecting objects. On the other hand, it is then impossible to zoom and pan at the same time, which might be a preferable mapping, for instance when searching for a particular object. An alternative mapping could use the mouse-wheel for zooming, which would enable simultaneous zooming with the mouse and panning with the touchpad. In general, this motivates a more systematic analysis of how the presented input primitives can be mapped to different subtasks. It is also interesting to consider, switching of mappings during usage and how the user could actively chose bimanual mappings that are most suitable for their current task.

EarthPad

Earthpad is a plug-in for Google Earth that uses the touchpad as direct handle for exploring the globe with flick&spin and zoom&rotate gestures. This applet demonstrates how application-specific mappings can be implemented on a touchpad to provide intuitive interaction styles to existing applications. ThumbScroll

ThumbScroll was implemented as an applet that enables touchpad scrolling for text editors. Similar to Virtual Desk, this technique can make selection of text across several pages easier, by performing scrolling with the nondominant hand while simultaneously selecting text with the dominant hand. Thumb-scrolling can also be used to navigate the document without leaving the home position for text input. The applet demonstrates customised use of the touchpad in the context of text editing, taking advantage of its fixed location with respect to the keyboard.

The implementation of concrete applications also brought out issues that we had not seen covered in prior work on two-handed input. Specifically, viewport adjustment with the non-dominant hand during a freehand selection with the right hand (as shown in Figure 7) affects the selection directly. The effect can be likened to moving a piece of paper while drawing on it, and in our application we found that this sometimes led to the selection of unwanted objects, One solution would be to move the mouse pointer during panning, so that it would retain its absolute location on the virtual canvas but this would have the disadvantage that the mouse pointer can get lost out of view. This example demonstrates that there are potentially unresolved issues when applying two-handed interaction to concrete tasks in practise.

TranscribePad

For transcribing it is necessary to listen and control recorded audio while typing textual content into a word processor. These sub-tasks are closely entwined, since the user often has to interrupt typing and pause or rewind the audio-stream. Traditionally, the function keys are used to provide access to audio playback, but this requires the fingers to leave their typing position. External devices, e.g. foot pedals, are common to allow users to control audio

An interesting general insight from this work was that the touchpad, while found to be only second best for its original purpose (i.e., pointing/dragging), exhibits many beneficial

69

Full Papers

Proceedings: NordiCHI 2010, October 16–20, 2010

aspects when it is customised for tasks it has not originally been designed for (i.e., gestures, absolute input, mode control). The techniques we have demonstrated are effective in leveraging distinctive properties of touchpads. This includes the responsiveness of the transducer to gestures, the flat physical layout enabling certain styles such as flicking, and the comprehensive set of input primitives, such as absolute input.

we have shown how the touchpad, in such a configuration, can be ‘activated’ for alternative input, complementary to pointing. We have contributed techniques that facilitate two-handed touchpad+mouse techniques on a ready platform of notebooks routinely used with external mouse, and in addition introduced distinctly novel uses of the touchpad for thumb interactions, absolute input, and parametrised hotkeys. The utility of the contributed techniques has been demonstrated through concrete applications.

Our survey had brought out qualitative issues concerning the use of touchpad’s as primary pointing device. However concerns related to accuracy and efficiency become less pronounced in the context of tasks performed with the nondominant hand, such as context setting and toolglass positioning. Ergonomic issues would also be expected to be factor in less, as interactions with the non-dominant hand would be less demanding in asymmetric tasks. Furthermore, while the touchpad’s fixed position has been highlighted as negative aspect for pointer input, some of our applications demonstrate that this can be advantageous for other forms of interaction. In this context, we see particular potential in thumb interaction since it provides access to the touchpad while the hand is in the common posture of resting on the keyboard. The thumb has the touchpad in immediate reach and can realistically perform a wide array of tasks. This is interesting, considering that both thumbs commonly serve the sole purpose of pressing the space bar.

REFERENCES

1. Balakrishnan, R. and Patel, P. 1998. The PadMouse: facilitating selection and spatial positioning for the nondominant hand. Proc. CHI’98, 9-16. 2. Balakrishnan, R. and Hinckley, K. 1999. The role of kinesthetic reference frames in two-handed input performance. Proc. UIST '99, 171-178. 3. Bier, E. A., Stone, M. C., Pier, K., Buxton, W., & DeRose, T.D. (1993). Toolglass and magic lenses: The see-through interface. Computer Graphics Proceedings, Annual Conference Series, 73-80, New York: ACM SIGGRAPH. 4. Buxton, W., Hill, R. & Rowley, P. (1985). Issues and techniques in touch-sensitive tablet input, Computer Graphics, 19(3), Proc. SIGGRAPH’85, 215-223. 5. Buxton, W., Myers, B., A Study in Two-Handed Input, Proceedings of CHI’86: ACM Conference on Human Factors in Computing Systems, 1986, 321-326. 6. Card, S. K., Mackinlay, J. D., and Robertson, G. G. 1991. A morphological analysis of the design space of input devices. ACM Trans. Inf. Syst. 9, 2 (Apr. 1991), 99-122. 7. Fallot-Burghardt, W., Fjeld, M., Speirs, C., Ziegenspeck, S., Krueger, H., and Läubli, T. 2006. Touch&Type: a novel pointing device for notebook computers. In Proc. NordiCHI’06: 4th Nordic Conference on Human-Computer interaction (NordiCHI’06), 465-468. 8. Guiard, Y. (1987). Asymmetric division of labour in human skilled bimanual action: The kinematic chain as a model. Journal of Motor Behaviour, 19, 486-517. 9. Grossman, T., Dragicevic, P., Balakrishnan, R. Strategies for accelerating on-line learning of hotkeys. In Proc. CHI 2007. 10. Harris, C. and Straker, L. Survey of physical ergonomics issues associated with school childrens’ use of laptop computers. Industrial Ergonomics, Vol. 26, Issue 3, September 2000, 337-346 11. Hinckley, K., Pausch, R., Proffitt, D., Patten, J., & Kassell, N. (1997). Cooperative Bimanual Action. Proceedings of the CHI’97, 27-34, New York: ACM. 12. Hinckley, K., Czerwinski, M., Sinclair, M., Interaction and Modeling Techniques for Desktop Two-Handed

We have focused on interactions with the touchpad, since it was found to be the most common built-in input device in our sample of N+M users. However, other input devices, such as pointing sticks or trackballs can also accommodate new interaction styles. Trackballs, for instance, have unique affordances that could be applied for instance to rotating objects (such as the globe). Also, our survey found notebook configurations that comprise both a touchpad and a pointing stick. This can give rise to new interaction styles that combine both devices (e.g., thumb on touchpad and finger on the pointing stick for rotate and scale gestures); or that allow switching between navigation modes (absolute navigation with the touchpad, and relative navigation with the pointing stick). Also, emerging multi-touch pads have the potential to support further interaction styles. CONCLUSION

Two-handed input and touch-based interaction techniques have been the subject of HCI research for a long time. The significance of this work is that it considers two-handed input in the context of a concrete interface configuration of notebook with external mouse and built-in touchpad. As foundation for our work, we have contributed quantitative and qualitative data collected through an online survey of notebook use. Through this survey we have evidence for the widespread use of notebooks in the above configuration. The touchpad is a highly expressive input device in terms of the input primitives it provides. Intriguingly though, the touchpad is left completely unused in configurations where an external mouse is attached to a notebook. In this paper

70

Full Papers

Proceedings: NordiCHI 2010, October 16–20, 2010

Input, Proc. UIST’98 Symposium on User Interface Software & Technology, 49-58. 13. Kabbash, P., MacKenzie, I.S. & Buxton, W. (1993). Human performance using computer input devices in the preferred and non-preferred hands. Proc. InterCHI ‘93, 474-481. 14. Kabbash, P., Buxton, W.& Sellen, A. (1994). TwoHanded Input in a Compound Task. Proc. of CHI '94, 417-423. 15. Kurtenbach, G., Fitzmaurice, G., Baudel, T., and Buxton, W. (1997). The design of a GUI paradigm based on tablets, two-hands, and transparency. Proc. CHI’97 Conference on Human Factors in Computing Systems, 35-42. 16. McLoone, H., Hinckley, K. and Cutrell, E. Binamual Interaction on the Microsoft Office Keyboard. In Proc. Interact, 2003. 17. Odell, D.L., Davis, R.C., Smith, A. and Wright, P.K. Toolglasses, marking menus, and hotkeys: A comparison of one and two-handed command selection techniques. In Proc. Graphics Interface 2004, 17–24 18. Rekimoto, J. 2002. SmartSkin: an infrastructure for freehand manipulation on interactive surfaces. In Proc.

of CHI’02 Conference on Human Factors in Computing Systems, 113-120. 19. Rekimoto. ThumbSense: Automatic Input Mode Sensing for Touchpad-based Interactions. In Proc. CHI’03 Conference on Human Factors in Computing Systems, 852-853. 20. Sawin, D. A., Stewart, A.M. and Calcaterra, J. A. (2002): Pointing Stick versus Touch Pad: Working Together. Proc. HFES 2002 Annual Meeting, CSTG Bulletin, Vol. 29-1 21. Sommerich, C. A Survey of Desktop and Notebook Computer Use by Professionals. Proc. HFES 2007 Annual Meeting, Industrial Ergonomics, pp. 1124-1128 22. SpacePilot. 3Dconnexion. http://www.3dconnexion.com/3dmouse/spacepilot.php 23. Woodruff, A., Anderson, K., Mainwaring, S. and Aipperspach, R. Portable, But Not Mobile: A Study of Wireless Laptops in the Home, Proc. Pervasive 2007, Toronto, Ontario, May 2007, 216-233. 24. Zeleznik, R. C., Forsberg, A. S., & Strauss, P. S. (1997). Two pointer input for 3D interaction. Proceedings of 1997 Symposium on Interactive 3D Graphics, 115-120.

71

Full Papers

Proceedings: NordiCHI 2010, October 16–20, 2010

Tenori-on Stage: YouTube As Performance Space Mark Blythe School of Design Northumbria University, UK [email protected]

Paul Cairns Department of Computer Science University of York, UK [email protected]

ABSTRACT

created in different modes and users can play the device by flipping between blocks or performing other manipulations such as key transpositions.

This paper reports findings from four related studies of the “Tenori-on” as it appears on YouTube in order to consider Web 2.0 as a performance space. A quantitative analysis of returns for “Tenori-on” attempts to model how posts achieve and maintain popularity. This analysis suggests sustained posting and engagement amongst users rather than initial product launch enthusiasm. A content analysis of the videos returned demonstrates a very different response to the launch of other technologies like the iPhone 3G. A grounded theory explores comments to the most viewed video returned which was a post by the artist Little Boots. A range of comments indicate virtual applause and suggest that YouTube has been appropriated here as a space for performance. Finally perspectives from critical theory are drawn on to consider the meanings of the Tenori-on in this user generated context and the ways users creatively resist the most obvious affordances of the device.

Figure 1: Tenori-on. Photo by Gabriel “Gab” Pinto. Creative Commons

The designers were particularly concerned with the beauty of the light and sound, the ease of performance and the quality of the product as a whole [29]. There was an explicit aim to unite form and function in a digital instrument:

Author Keywords

Tenori-on. YouTube. User Criticism, Critical Theory

Experience,

Interaction

“In days gone by, a musical instrument had to have a beauty, of shape as well as of sound, and had to fit the player almost organically. Modern electronic instruments don't have this inevitable relationship between the shape, the sound, and the player. What I have done is to try to bring back these [...] elements and build them in to a true musical instrument for the digital age.” [29]

ACM Classification Keywords

H5.m. Information interfaces and presentation (e.g., HCI): Miscellaneous. INTRODUCTION

The Tenori-on is a musical instrument designed by Toshio Iwai and Yamaha. It is part sequencer and part synthesizer with an interface made up of a grid of LED buttons which the user activates to make loops of sound (see figure 1). A line of lights pulses across the grid indicating bars of musical time. When one of the buttons on the grid is pressed it lights up and a note plays as the pulse passes over it. The user can create patterns of recurring notes to make loops that form single blocks. Up to sixteen blocks can be

The Tenori-on was first made commercially available in Britain in 2008 to test the market before a wider global launch [36]. This study began as an attempt gauge responses to the instrument through the videos returned for a search on the term “Tenori-on” on YouTube. Although the returns offered numerous insights on the user experience of the device they also indicated how YouTube is being used as a performance space. When this study began the most viewed return was a video of Iwai himself demonstrating the device at a launch event. During the study it became a video posted by Little Boots. The returns for “Tenori-on” remain relatively dynamic as the next section will indicate.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. NordiCHI 2010, October 16–20, 2010, Reykjavik, Iceland. Copyright 2010 ACM ISBN: 978-1-60558-934-3...$5.00.

Modeling the Data

Results for the search term “Tenori-on” on 7th July, 2009 produced a typical outcome. The results page suggests that there are over 1,100 hits for this search term. However,

72

Full Papers

Proceedings: NordiCHI 2010, October 16–20, 2010

when the results pages are trawled for the view counts, it becomes clear also that the results are no longer sorted by view count with view counts in the 360th to 380th videos varying wildly from several tens of thousand to zero. Thus we only model hits retrieved up to the 360 th video.

or secondly, an existing video is viewed in proportion to how much it has already been viewed. This latter behavior leads to the power law demonstrating a “rich get richer” process. In many ways, this process makes a lot of sense in the YouTube context. People may search for videos across a wide range of criteria but generally they will view the more popular ones. Thus the rich videos do get richer. But also, popular topics are also likely to get more videos on that topic and hence there is a steady growth in the number of videos to be viewed.

When sorted by view count, it is clear that whilst the top few videos have view counts of several hundreds of thousands, this rapidly tails off but in such a way that there is still a “long tail” of viewed videos. There are several well understood long tail distributions that produce this shape, the two simplest and most common being the exponential distribution and the power law distribution.

Why then is there not a proper power law distribution for view counts? One possibility is that though topics do grow, there is also a “shelf-life” to the topics and interest in adding videos to a topic falls away. Whilst popular videos continue to be viewed, there are diminishing returns in putting up a new video as the likelihood of getting viewed under the Yule-Simon process becomes vanishingly small. There is some evidence that YouTube viewing behavior follows a Yule-Simon process where the richer get richer but the uploaders get bored.

The log-log plot of view count against the rank when ordered by view count is shown in Figure 5. Whilst the initial part of the plot is linear, it is clear that there is a tailing off from a straight line with a particular down turn just as the search results run out. This suggests that a power law distribution is a good starting point to consider view counts. Very similar results have been seen in searches over other terms such as “Shadow of the Colossus” [12] and “iPhone 3G” [8] where a rapid drop off or knee is more readily apparent.

Evidence against this comes from what might be termed the degree of churn in YouTube videos. With a Yule-Simon process, there is little churn – once a video hits the top, it stays at the top. However, in many other power law situations there is a lot of churn [5] where the top 10 or top 100 ranked items vary enormously and that is exactly what is seen in YouTube. In a rich gets richer process, how do the richest ever get supplanted? Although many topics such as iPhone 3G or Shadow of the Colossus could be expected to have a shelf life strictly related to the release date, the Tenori-on is an instrument which might have appeal beyond the initial novelty. Sorting returns by the date of the upload in the week beginning 14 th September, 2009, there were 15 new videos. Sorting returns by the previous month showed that there had been 52 new uploads. This is comparable to the overall monthly average of 1,200 Tenori-on hits in the two years since its launch in Britain. This indicates a community of users continuing to keep a topic active and dynamic long after an initial launch. What then is this community doing?

Figure 2. A log-log plot of view count against rank for the hits from search “Tenori-on” with best fit line for a power law.

This shape of distribution in YouTube returns has been noted before by Cheng, Dale and Liu [14] also with a much more distinctive knee. The videos considered there were not the results of a particular search but rather obtained from a substantial trawl of all YouTube content. They propose, as others have, that the tail better fits a Weibull or Gamma distribution, each of which have a distinctive knee shape. Whilst this may be the case, this does not provide much insight into what processes drive viewing behavior. However starting with the power law distribution it is possible to consider possible processes that might lead to the observed view count distributions.

CONTENT ANALYSIS OF THE FIVE HUNDRED MOST VIEWED RETURNS

A content analysis [28] was performed on the first five hundred returns for the search “Tenori-on” sorted by most views on the 11th June 2009. Content analysis involves categorizing each video against pre-existing codes and counts the number of videos in each category. Figure 2 shows percentage returns for each of the categories.

The power law distribution was first accounted for by Yule [41] and then simplified by Simon [33]. It corresponds to a steady state situation where the rate of increase in view counts is proportional to the number of view counts each video has already achieved. This can be generated by a step by step process where at each step two things can happen. First, there is a small probability that a new video is added

73

Full Papers

Proceedings: NordiCHI 2010, October 16–20, 2010

Coulter then attempts to play it and a long silence follows ended by audience laughter. In a post of the British singer Little Boots playing Glastonbury there is a similarly awkward silence as she picks up the device “hang on…” she says to laughter. After a technician arrives and fails to fix the connection she decides to play something else and turns to her synthesizer [23]. When Little Boots plays a keyboard the Tenori-on is sometimes placed on top of it and remains in shot as she sings. In band settings however it is raised up on a stand like a music stand. Its lights are lost in the stage lighting and it is not an integral part of the performance [26]. Similarly, a Bjork performance featuring the device begins with a close up of one of the backing musician holding it but it is mostly not visible in the wider stage shots [7].

Figure 3: Content analysis on first 500 returns for “Tenori-on”. Composition

Songs

The vast majority of returns featured people playing the device but there are distinctions to be made in terms of how they use it. Almost forty percent of the returns featured an original composition. Here users had composed pieces of music on the device and either set it to play a sequence of loops and blocks or directly manipulated the instrument to change blocks manually. Occasionally these Tenori-on compositions were accompanied by video effects or lightshows created with movie making software like iMovie. These were entirely instrumental pieces but often featured more than one instrument e.g. drum machines, synthesisers, compressors, effects units and other electronic instruments. Interestingly there was only one video featuring the Tenori-on and a non-electronic device; “Tenori-on meets sax” [27].

Although there are a great many videos featuring original compositions on the Tenori-on, they vary dramatically in quality as the comments below them readily illustrate. “Com4jai” like many others posted a video called “my first tenori-on song”. “Fewasdr” comments: “I could play that better by randomly slapping my penis on it” [18]. More kindly “utlewis” notes “you suck at this, no offence”. “Lionomega” asks “yes but where is the song?” (Ibid). The sub category of songs represents a wide variety of material and quality. Little Boots is a singer and song writer who came to prominence partly through the popularity of her videos on YouTube and MySpace. Unlike the majority of posts featuring instrumental compositions Little Boots is among the minority of Tenori-on users on YouTube to post songs. Just 4% of the returns featured anyone singing a song with the device. However these posts were amongst the most popular returns of all. Of these the most popular, and indeed the most popular “Tenori-on” returns of any kind at the time of this analysis were the posts by Little Boots. The Little Boots songs featured innovative cover versions as well as original compositions. The videos are usually static shots of the singer sitting in a bedroom and playing either the Tenori-on alone or a combination of Tenori-on with other instruments e.g. piano and stylophone. These will be returned to in later sections.

Performance

The performance category often also featured original compositions but these were performed in public rather than filmed in a bedroom or studio. Some of these performances featured well known performers like Bjork using the device, these were a small minority however. Most of the perfornances were by DJs or musicians in small venues. Many of these videos were shot at launch events or conferences and some feature Toshio Iwai himself playing the device before an audience of academics. It is interesting to note that the reception of such videos changed over time. The early comments on Iwai‟s performance are largely positive but later posts declare that he “sucks” and people should go and listen to “Little Boots”.

Words and Pictures

A small number of posts used the device primarily to create graphic displays of words like “Truth” or “Happy New Year”. Others used it to create simple pictures like an apple or animations of Eygptian hieroglyphic people dancing. A small percentage of the videos were demos of rival products such as iPhone apps with similar grid like interfaces and functionality (e.g. PakSound, Soundgrid, EasyBeats or Sequence). These offer similar grid-based interfaces but cost one or two dollars rather than sevaral hundred. Other minority posts were videos on a possible recall of the device because of an electrical fault, there were also two interviews with Iwai.

A small number of videos feature the device in a concert setting where it was not the primary focus. Musicians were playing a set and produced the instrument for one or two songs. Typically the performers apologized and joked about it. “I have to activate the doomsday device, hold on one second… You’re all going to feel a stinging sensation and then you’re going to be blind” [16]. At another concert the same artist tells the crowd “You may have noticed a flashing device, it’s a Tenori-on, it’s a …well, it’s a flashing device, I’m going to flash you with my device”.

74

Full Papers

Proceedings: NordiCHI 2010, October 16–20, 2010

Review and demos

pizza box. The only graphic on it is a very simple line drawing of the device on the front. It is certainly not going to win the sort of awards that Apple‟s packaging designers are accustomed to receiving. But perhaps the reason for the lack of unboxing videos is that the users were thinking of it primarily as an instrument rather than a new gadget.

Although the device was played in demonstrations the focus was in showcasing different modes rather than composing music. Some of these featured musicians who had been given the device to play something on it for the purpose of promotion. One series of videos made by Yamaha specifically demos each mode of the device in a step by step guide.

The overall shape of the data then is quite different to that of returns on searches like iPhone. There are very few reviews, there are almost no unboxing moments. The vast majority of posts are musical performances of one kind or another.

There were remarkably few reviews, indeed only one in the first five hundred returns. This is extraordinary considering that returns for searches on other new devices like the Apple iPhone return such large quantities of reviews [8]. Devices like the iPhone and even individual apps like twitter clients receive detailed reviews which often focus on aspects of usability (ibid). Only one such review was returned for the Tenori-on and it was posted by a specialist technology webzine publication. It is possible then that the YouTubers who are posting are reading the device more as an expressive instrument rather than a piece of functional technology. Don Norman famously notes that nobody would expect a violin to be usable [30] perhaps this is also the case for electronic instruments like the Tenori-on.

The next section will consider in more detail the most popular return at the time of writing: the Little Boots video “ready for the fun” [24]. ANALYSING YOUTUBE’S NUMBERS

YouTube now provides audience demographics based on the information gathered from users with YouTube accounts who have logged on. According to the site statistics at the time of writing the “Ready for the fun” video was most popular with males from 35 to 44, followed by males aged between 45-54 and males aged between 2534. It was viewed most often from Britain, where Little Boots is from and where the Tenori-on was first made commercially available. That the video should be most popular in Britain is perhaps no surprise, that the video should be so much more popular with men than women is perhaps less predictable. Other Little Boots videos, such as some of her performances at Glastonbury are most popular with 13-17 year old girls.

Satire

There were also remarkably few humorous or satirical posts. Of the few that were returned some might be considered fairly unsophisticated forms of humour, one simply features a Tenori-on spelling out in lights the word “cock”. But others were more subtle, one, for instance, was a clip from the IT crowd where IT support worker Roy is shown wearing headphones and enthusiastically nodding along to the Tenori-on he is playing. When another character walks in he puts it away without comment and the audience laughs at this clear demonstration of Roy‟s essential geekiness. An even more direct satirical attack on the device comes in a demonstration of an “unplugged” Tenori-on. Kentaro Fukuchi‟s video begins with a sign noting that the Tenori-on is very cool but also rather expensive [21]. It goes on to show him making an “unplugged” version consisting of four vinyl pipes and a piece of bubble wrap. Fukuchi then plays the instrument for our listening pleasure by popping the bubbles on the bubble wrap attacking not only the price of the device but the sound quality.

The timeline in Figure 4 also supplied in “statistics and data” section goes some way to explaining what is happening:

Unboxing

Of the first five hundred videos only one featured a Tenorion unboxing. Unboxing videos accounted for over twenty percent of the first hundred most viewed videos for a search on the iPhone [8]. Unboxing videos are a very interesting YouTube phenomenon, it has been claimed that they provide an empirical justification for theoretical constructs of user experience which emphasise the importance of anticipation (Ibid). Why then should there be so few unboxing videos for the Tenori-on? Were the users not excited about opening it? The Tenori-on packaging consists of a fairly rough cardboard box something like a take away

Figure 4: YouTube “Statistics and Data” for little boots READY FOR THE FUN!!!. The letters in Figure 4 show the dates at which the video was embedded in sites like MySpace or referred from Google searches. It is likely that after being embedded in

75

Full Papers

Proceedings: NordiCHI 2010, October 16–20, 2010

the MySpace site other demographics than the middle aged males searching for “Tenori-on” would have viewed the video on YouTube although they may not have logged in as users.

clearly been removed although traces of flames wars about them still remained. Figure 5 shows the percentages of comments assigned to each code.

YouTube claims that there are more or less equal numbers of males and females on their site [40]. There are several Tenori-on groups on Facebook and the most popular one has four hundred and ninety one members. Of these only around fifty are female. New technologies are often marketed specifically at males so it is possible that this explains the overwhelming gender bias in the audience for the most popular “Tenori-on” return. GROUNDED THEORY

On 28th July 2009, the most viewed return for the search “Tenori-on” was “little boots READY FOR THE FUN hotchip Tenori-on cover” with 399,627 views, 1,520 ratings with an average of five stars and 808 text comments [24]. The video is a single take performance of a song shot from a fixed camera. An attractive young woman in a black vest is sitting in a darkened room next to a keyboard with some cluttered shelves just visible behind her. She is lit by a strong light from screen left creating a chiaroscuro effect and also providing a strong visual platform for the white lights of the Tenori-on. In real time we see the artist setting a bass line in one block. She then creates another block with a synth pulse and another with a minimal drum pattern. Further blocks are added with synth pad sounds making up a chord sequence. Finally she adds a single voice triplet of lead notes. It is a virtuosic and clearly carefully rehearsed performance. There are almost no mistakes or mis-steps. Rather than begin the video with the Tenori-on part already created she performs its creation from scratch.

Figure 5: Comments on Little Boots video The praise was most often a textual form of applause “Bravooo” “A-MAZE-ING!!” “wow”. These comments often contained onomatapeic attempts to represent the sound of cheering “yay!”, “Whoo!”. These brief sound based responses indicate the ways in which the YouTube audience respond to the post as a performance. Just as they would clap at a live event they use text to applaud. These types of praise occasionally became direct address “whoop whoop herd u on radio 4 womans hour !” some followed praise with a question for the artist “What do you call the sound effect at 4:31 ? Bare foot slapping on bedroom oak effect wood floors?”. Although there are no responses from Little Boots there is a sense of direct communication with the artist in these posts. Some included requests for her to play their home town or recommendations of music or other technologies she might like. Many of theses posts were genral forms of approabation “you rock!” but others focussed on specifics such as her skill using the device, her voice, or her hair. Her hair was mentioned specifically in 12 posts, many more commented on her general beauty.

When the Tenori-on part is complete she begins singing and uses the transpose function to shift the entire sequence to create a different progression. By transposing the blocks and moving between them she creates a very complex and dynamic backing for her vocal which is showcased at one point by dropping the Tenori-on out altogether while she sings acapella. She ends the track by clearing each of the blocks until only one two note pattern remains. Finally she swirls her hand over the Tenori-on making a pattern and a random fall of notes.

Beauty

Many of the comments regarding her beauty simply noted that she was a “hot babe” . Others commented on the combination of looks and talent “what a babe. what a electronicmusician...”. Two of the comments were explictly sexual “oh shit i snapped my boner!!”. Many more were more emotional responses. “Simplely....I in love” Some of these declarations of love were followed by optimistic requests like “be my girlfriend”. Or “can you be my internet girlfriend”. Other emotional responses included more direct accounts of how the video made them feel, for example “sweet. made me feel good” or “goosebumps all over”. Often comments related to “sweetness” and cuteness “aww”.

A grounded theory analysis was performed on the comments for this video. Grounded theory begins with data, rather than pre-existing categories. Open codes are developed to summarise the data, these are then grouped together and linked in axial coding, the final stage of selective coding involves the selection of typical quotes to illustrate the “theory”. Theory here may refer merely to a broad description or set of categories rather than a fully worked predictive schema [13].

Technology

Praise

The comments that were not some form of praise for the artist‟s looks or talent were often praise of the Tenori-on.

The vast majority of comments on the video were in praise of Little Boots, indeed, some negative comments had

76

Full Papers

Proceedings: NordiCHI 2010, October 16–20, 2010

“intimate” peformance then funcioned very well as a marketing device for the more fully produced commodity in production.

Seventeen percent of the comments specifically referenced the technology. Some of these were questions about what the device was often followed by explanations from other viewers. Some were comments about the innovative way in which Little Boots was using the Tenori-on by people apparently familiar with the device “It must be in the choice of Tenori-On functions; key changes most important, the beepy-boopy Bounce mode probably least important.” This comment is interesting because it picks up on the innovative use of key change in the performance. Of the five hundred videos sampled very few make use of the key change button in this way and Little Boots is the only performer to use it to structure verse / chorus changes.

Intimacy on YouTube: the limits of ratings

The seeming intimacy of the close up performance allows the users to feel engaged in an emotional dialogue with the artist. The intimacy of the setting was occasionally contrasted to other media “Amazing. These solo versions on YouTube actually sound better than the mega-overproduced tracks on your EP. Your minimalist instrumentation lets us hear your great tunes and voice better, I think. Rock on girl” The video appears somehow more authentic, it is described in another comment as “acoustic”, another writes “Good on you for releasing wee home recorded vids like this, theres something very intimate about it, and that is never a bad thing”. The setting and the performance are taken as modest and somehow genuine.

A number of comments specifically reference the other Tenori-on performances on the site “You're the only one I have found so far who knows how to use the thing a write music with it. Everybody else seems like random noise.” One viewer noted that this was the best piece of advertising so far for the Tenori-on. Two viewers, however, posted jokes about it “cool...pong is back!!!” similarly “I like how she sings and can play space invaders at the same time :)” The slightly dismissive tone of these posts indicate a pleasure in “geeky” technology being used in this way “it's nice to see a girl so into synth-pop geekery doing so well.” Part of the orgininality being enjoyed here is the innovative use of the instrument which is in marked contrast to the way it is usually used, even by its inventor.

The star rating system offers a strictly numerical form of expression. It is interesting that the text boxes are used to emulate sound, noise and more intimate forms of expression such as the fan letter. The responses indicate that automated features such as rating are important but perhaps limited as forms of expression. FINDINGS SO FAR

Even if models could offer a compelling account of patterns of popularity (rich get richer plus decay over time) this would not tell us why one video rather than another is more popular. If it were simply the case that the rich got richer this still does not explain how they get rich in the first place. Clearly “Ready for the Fun” is returned for other searches, e.g. “Little Boots” and picks up more views and achieves a most viewed ranking, But this does not entirely explain how an artist and a technology can capture the public imagination in the way that Little Boots did. The next sections draw on critical theory to consider the meanings of the Tenori-on in the Little Boots video.

Criticism

Although abusive comments had been removed their traces remained in the flames sent by fans defending Little Boots against her detractors. “I think we all feel sorry for you” [the person who had been criticising Little Boots] because you are obviously unhappy with life. Don't you know what pop music is about? Its about trends and styles and the latest thing so do yourself a favour an chill.” It seems that the removed comment had slated Little Boots as trivial or trendy. Some direct criticism remained however. Some of this criticism, like much of the comments on YouTube in general is homophobic and mysoginistic “she is actually like all other chicks, she went on x factor and didn't get through.” Other posts dismissed the video as a “gimic” or expressed resistance to coverage in other media announcing her as “the next big thing.” Three of the posts expressed fears that the authenticity on display in the video would be swallowed by the music industry “I will still like your art after you've been over exploited by the music Industry!” Others contrasted this “amateur” performance with the artificiality of music industry products: “I really like it. Is it now the trend to go against what the industry says.” One viewer on the other hand felt that she could be great with some help “from the pros.”

CRITICAL THEORY

Critical theory is a field which encompasses many perspectives. It draws on literary studies, psychoanalysis, linguistics, cultural and media studies as well as philosophy [20]. As computing technology penetrates still deeper into everyday culture, perspectives from critical theory are increasingly relevant to studies of HCI [e.g. e.g. 1, 6, 19, 31, 32, 3, 8]. Critical theory is primarily concerned with interpretation and meaning. But critical “readings” are always open. There could never be a final reading of a poem, play, or any other cultural artifact. Nobody could plausibly claim to have finally discovered what Hamlet means. The meanings of cultural artefacts are never singular, they are always multiple. For this reason critical theory includes perspectives which are not only different from one another but sometimes completely opposed. Readings such as those by the cultural critic and philosopher Slavoj Zizek are “imagined hermeneutic

The references to other media were primarliy discussions of appearances by the artist either in concert or on television. A large number also referenced her debut CD “Hands” with promises that they would buy it. The “acoustic” and

77

Full Papers

Proceedings: NordiCHI 2010, October 16–20, 2010

understandings” [44] or provocative interpretations and make no empirical claim. For example Zizek claims that British toilets combine both inspection (water) and fast disposal (flush) indicating inclinations towards empiricism and pragmatism. This is not to claim that this is what their toilet designers were aiming at nor is to claim that this is what most Britons think while they are on the loo. The following sections then are likewise interpretations which make no empirical claim but rather, supplement the more empirical studies which precede it.

is skillfully composed. The sound quality is particularly good and the mix of sound levels between voice and instrument has clearly been given serious attention. Although the bedroom setting suggests an artless or naïve performance Little Boots is clearly very conscious of the way she is presenting herself.

Tenori-on as Commodity Fetish

Perhaps the most obvious interpretation of the Tenori-on on YouTube is that it exemplifies perfectly commodity fetishism. When Marx first formulated the notion of commodity fetishism he was concerned to indicate the ways in which commodities are figured as mysterious or magical objects “endowed with special powers” [44 p. 300]. This can be seen in the YouTube responses where delight and wonder are mixed equally with speculation about how much the device costs. This wonder explains perhaps the impulse to post videos of the first composition: look at this magical and mysterious object which I own.

Figure 6: Still from Little Boots MEDDLE bedroom version

That our technologies are now as much about fashion as our clothes is such a commonplace that it is increasingly challenged, for example Charlie Brooker, “anyone who thinks their phone is an expression of their personality hasn‟t got one” [11]. Indeed the adoption of a fetish is not about simple identification. Stonewashed jeans are supposedly associated with a working class lifestyle but they have always also been worn by the upper class. For Zizek it is in the gap between what something is supposed to represent and how it is actually appropriated where meaning is made; for this reason it is appropriate then that today‟s most popular clothing stores are called “the gap” [42]. In order for us to adopt a technology there must also be a gap. As previously noted many of the videos featuring the Tenori-on in performance show the performers joking about the device and their own geekiness: “I know very well that this is ridiculous, nevertheless…”

It is possible that she usually sits around in her bedroom wearing hot pants like the ones in Figure 6 but it is more likely that the performance is very thoughtfully staged. It is also clear that these performances are far from spontaneous. They are not the improvised patterns submitted by the majority of Tenori-on users posting to YouTube. Rather they are carefully planned and rehearsed performances. In other words the Tenori-on is far from the only reason this video is popular. In Lacanian terms the Tenori-on might be thought of as “object small a” [42]. This is a difficult Lacanian concept that Zizek illustrates with reference to the films of Alfred Hitchcock. Every Hitchcock film features a “MacGuffin” some plot element that motivates the characters. What the MacGuffin is does not matter, it serves merely to drive the characters forward, it might be a stolen microfilm, a plot by enemy spies, a lost love, a murderer – it could also be ambiguous [38].

However there were almost no unboxing videos in marked contrast to most other technology releases. This indicates perhaps that the device is either something more or something less than a commodity fetish.

The Tenori-on in Little Boots videos also functions in this way. It literally drives the video forward signalling both beginning and end, it provides a motivation for making and watching the video. Here is Little Boots playing this strange new instrument. The MacGuffin is a surplus of meaning, an excuse for self presentation.

Tenori-on as MacGuffin

The comments on the Little Boots video indicate that the technology is not the only thing being admired and desired. Of equal if not much more interest is Little Boots herself. It is likely that the Little Boots videos are not as artless as they may appear. John Bowers has remarked that “unplugged” performances are in fact always plugged. If an acoustic performance is to be recorded and broadcast it must be miked up and amplified just as carefully as an electronic instrument would have to be (Bowers, personal communication). In each of the Little Boots posts the shot

The Little Boots videos are far from typical of most of the Tenori-on returns. They are not the improvised patterns submitted by the majority of Tenori-on users posting to YouTube. They are quite unlike the compositions where users create spontaneous, solely instrumental patterns by exploiting the most obvious affordances of the device. What then can be said about the particular use of the device

78

Full Papers

Proceedings: NordiCHI 2010, October 16–20, 2010

in the music rather than the video? The next section turns to another perspective suggested by critical theory: technological determinism.

instrumental compositions and perhaps explains some of its power and popularity. TENORI-ON STAGE

As with previous studies drawing on YouTube, the medium is as important as the context of the posts [8] The majority of posts featured users making music of one kind or another with the device. In this sense, these users appropriate YouTube as a venue for performance. While Iwai may have partially achieved his goal of making an instrument for the digital age, challenges remain in terms of providing a space for performance. As previously noted, when the device is used in a concert setting (e.g. 7, 26) it is lost in the space and stage lights. It is at its most arresting and beautiful in a small setting like a bedroom, or indeed a TV studio, with a close focus on the device.

Tenori-on and Technological Determinism

The ease with which a complete beginner can create a pleasing sound with a Tenori-on was one of the features that were heavily promoted when the device was launched [36]. The affordances of the Tenori-on are towards patterns which even first time users create unwittingly as soon as they light up one of the buttons. The resulting patterns of music that might appear intrinsically modern or at least twentieth century. The music of Philip Glass is intensely patterned and repetitive. Minor variations in patterns lead to very complex pieces but for the uninitiated listener they can sound unbearably repetitive if not mechanically looped. Similarly dance music is a form which in the last twenty years has made an art out of manipulating loops of sound. Again, to those who are not familiar with the genre much of it can sound alike: “Call that a beat, they had proper bass in my day.” Likewise, the music which accompanies video games is often deeply patterned. Of course the current generation of video games can play any form of music but early games had to generate their themes “in house” and with very limited computational resources. Early video games like Sonic the Hedgehog featured soundtracks composed and played entirely on computers. They were as patterned and repetitive as any avante garde piece or dance track.

In some ways YouTube is a better performance space for the Tenori-on than a concert hall. In the bedroom videos both device and performers can be seen and appreciated. Beyond texting crude substitutes for applause the audience can, to a degree interact. There are several Facebook groups for the Tenori-on, and its members exchange tips about the device. It is not difficult to imagine a dedicated Tenori-on social networking site which would facilitate not only sharing recordings and videos but also perhaps collaborations. The ongoing uploads suggest a community of users who are interested in each others‟ work as well as their own. Such a Tenori-onTube might also provide more imaginative ways for viewers to applaud than the basic YouTube textbox, e.g. sound or graphic presets. As jazz clubs provided a space to develop new forms of music for new instruments like the saxophone perhaps a dedicated site might help the development of a Charlie Parker for the Tenori-on.

Wikipedia offers a range of definitions of technological determinism, here is one; „The idea that technological development determines social change' [39]. Although it is an extreme position it is hard to deny in every respect. The act of turning to Wikipedia for a definition rather than a book is evidence of the almost immediate impact of new technologies. But in a sense the Tenori-on provides a counter-example to technological determinism. The patterned music it produces existed long before it did. The patterns of music in Bach which could theoretically repeat ad infinitum were the dominant form signaling religious concerns with eternity. It was not until the Enlightenment that composers like Beethoven began to write linear music with identifiable beginnings and endings. Patterned music then is nothing new. In this sense the message preceded the medium.

Such a site might rank uploaded performances by views as with YouTube but it might also be interesting to allow users to categorise the submissions themselves. User-based categorization might be constructed as a content analysis where users could assign pre-existing categories (e.g. genre). More radically an open coding such as that practiced in grounded theory might allow for more creative and surprising patterns of user categorization to emerge. DISCUSSION: ALL THE WEB’S A STAGE

Many musicians are beginning to use YouTube as an intimate performance space with a range of instruments. In Reggie Watts‟ video “I just want to” the artist uses only a microphone with a sampler and mixer to produce an entire song on the fly. He begins by recording samples of “beat box” vocal percussion and sung bass lines which he then loops and mixes with instantaneous recordings of backing vocals and live lead vocals. Watts creates every part of the song in a single take and performs the whole piece in real time [37]. The virtuosity is astonishing and would be lost outside this intimate YouTube “bedroom” staging. But YouTube is a stage for other kinds of digital performance.

And as Little Boots demonstrates, there is more than one way to adopt a new technology. Part of the pleasure of the “ready for the fun” video is the virtuosity of Little Boots‟ performance and her mastery of the instrument. Little Boots‟ use of the device to produce a recognizable song “ready for the fun” by the electro-pop band Hot Chip. Little Boots‟ careful, controlled and skilful use of features like the transposition function illustrates the ways in which the most obvious affordances of the device can be creatively resisted. It is this creative resistance which, amongst other things, sets the video apart from the more general slew of patterned

79

Full Papers

Proceedings: NordiCHI 2010, October 16–20, 2010

Players of games like Shadow of the Colossus archive particular performances of game play [12]. And there is increasing attention to amateur multi-media such as machinima and mash ups [2] where YouTube and other host domains are performance spaces.

demonstrated models of popularity based on the rich getting richer within a limited shelf life did not hold. Further regular uploads demonstrated a community of users that continue to be engaged with the device. The qualitative content analysis demonstrated that responses to the Tenorion were quite different to those to gadgets like the iPhone indicating perhaps that the device is being taken seriously as a musical instrument. The grounded theory of the most popular return also indicated that the community found the device almost as compelling and beautiful as Iwai hoped. Finally perspectives from critical theory helped articulate the multiple meanings of the device as well as the notion of YouTube as performance space. YouTube offers unprecedented glimpses into the lives of users and the ways in which they adopt and adapt technology. Observation and description of these data are not enough, there must also be interpretation. Of course, in HCI there will always be base level questions of functionality, usability and acceptability but increasingly there is the further problem of meaning.

The British theatre director Peter Hall argued that all that is necessary for an act of theatre to occur is for someone to walk across an empty stage while someone else is watching [10]. It is clear that many, if not all, of the postings to YouTube are performances but the interface is seldom considered as a stage. The posts are also films yet film theory is seldom referenced in HCI. Zizek is perhaps the best known cultural critic and theorist of film alive today and yet his appearance in an article on YouTube may seem bewildering. As HCI begins to concern itself with issues like aesthetics it is essential that the field engage with the arts and humanities [15]. As Jeffrey Bardzell has argued it is simply not the case that there are no definitions of what we mean by “aesthetics” as is sometimes claimed, there are standard textbooks on it. As Web 2.0 technologies are appropriated for creativity and performance, traditions of thought more readily associated with literary and cultural studies become increasingly relevant. There is then a growing recognition that researchers who ignore relevant work in the humanities and social science risk repeating previous work or revisiting previous philosophical dead ends [15, 9]

REFERENCES

1. Agre P. Toward a Critical Technical Practice: Lessons Learned in Trying to Reform AI. In Bowker G., Star S., Turner W., and Gasser L., eds, Social Science, Technical Systems and Cooperative Work, Erlbaum, 1997. 2. Bardzell J. (2007) Creativity in Amateur Multi-Media: Popular Culture, Critical Theory and HCI. Human Technology Volume 3 (1), February 2007, 12–33

This paper combines very different forms of analysis from social science and the humanities. Of the various forms of analysis directed at the Tenori-on on YouTube in this paper the critical theory will appear most alien and disreputable to a traditional HCI audience. George Sokal famously hoaxed a journal of cultural studies into publishing errant pretentious nonsense [34]. Since then it has been easy to sneer at the humanities. But scientists should not forget how often their own journals are duped by charlatans [22]. There is a lot of bad critical theory which is easy to laugh at but then there is also a fair amount of bad science which can be equally amusing [22].

3. Bardzell, J. Interaction criticism and aesthetics. In Proc. CHI 2009. ACM Press (2009), 2463-2472 4. Bardzell, S. 2010. Feminist HCI: taking stock and outlining an agenda for design. CHI '10. ACM, New York, NY, 1301-131 5. Batty, M. Rank clocks. Nature, 444 (2006), 592-596 6. Bertelsen, O., Pold, S. Criticism as an Approach to Interface Aesthetics. In Proc. NordiCHI 04, ACM Press (2004), 23-32 7. Bjork. Bjork ~ Who Is It? (Live in Paris ~ from Voltaic) (feat. Tenori-On). http://www.YouTube.com/watch?v=dFsHuu0ah_g

Critical theory is an umbrella term for a collection of sometimes mutually exclusive approaches to interpretation. The perspectives available within it are not methods but styles of thinking. The interpretations of the Tenori-on on YouTube here are examples of one perspective – psychoanalysis. They not suggested as definitive readings, there could be no such thing. There are a range of other perspectives (feminism, dramatism, performance theory) that would also be relevant. As interactive technology penetrates further into every aspect of our lives and culture the value of these different styles of thinking is becoming clear e.g. [4].

8. Blythe, M. and Cairns, P. Critical methods and user generated content: the iPhone on YouTube. In Proc. CHI '09. ACM, New York, NY, 1467-1476 9. Blythe, M., McCarthy, J., Light, A., Bardzell, S., Wright, P., Bardzell, J., and Blackwell, A. 2010. Critical dialogue: interaction, experience and cultural theory. In CHI EA '10. ACM, New York, NY, 4521-4524 10. Brook P. (1984) The Empty Space. Atheneum. New York 11. Brooker C., (2007) Dawn of the Dumb: Dispatches from the Idiotic Front Line. Faber and Faber. London

CONCLUSION

12. Cairns, P., Blythe, M. Research Methods 2.0: doing research using virtual communities. In Zaphiris, P., Ang,

This paper has reported findings from four related studies of the Tenori-on on YouTube. The quantitative analysis

80

Full Papers

Proceedings: NordiCHI 2010, October 16–20, 2010

J. (eds) Social Computing and Virtual Communities, Chapman & Hall, 2009.

28. Mayring P. Qualitative Content Analysis in Flickr. In Kardorff, U., Steinke, E. eds. A Companion to Qualitative Research. Sage, London, 2004.

13. Charmaz K. Constructing Grounded Theory. Sage, London, 2006.

29. Nishibori, Y. and Iwai, T. TENORI-ON. In Proc. SIGGRAPH 2005, ACM Press, 2005.

14. Cheng, X., Dale, C., Liu, J. Statistics and social network of YouTube videos. 16th Int. Workshop on Quality of Service, IEEE London (2008), 229-238.

30. Norman, D. A. Human-centered design considered harmful. interactions 12, 4 (2005), 14-19

15. Cockton, G., Bardzell, S., Blythe, M., and Bardzell, J. 2010. Can we all stand under our umbrella: the arts and design research in HCI. CHI EA '10. ACM, New York, NY, 3163-3166

31. Satchell C. Cultural Theory and Real World Design: Dystopian and Utopian Outcomes. In Proc. CHI 2008, ACM Press, 2009, 1593-1602 32. Sengers P., Boehner K., Shay D., Kaye J., Reflective Design (2005) Critical Computing Aarhus Denmark pp 49-58

16. Coulton, J. My Monkey w Tenori-on Pt 8. http://www.YouTube.com/watch?v=diu875wE8Ls 17. Coulton, J Jonathan Coulton plays Code Monkey live at Dingwalls London. http://www.YouTube.com/watch?v=hCZeb83i2DE

33. Simon, H. (1955) On a class of skew distribution functions. Biometrika, 42(3-4), 425-440

18. com4jai My First tenori-On song. http://www.YouTube.com/watch?v=eWV46e8Z4Wk

34. Sokal, G. (1996), Transgressing the Boundaries: Towards a Transformative Hermeneutics of Quantum Gravity. Social Text 46/47 217-252.

19. De Souza C. The Semiotic Engineering of HumanComputer Interaction. MIT Press, Cambridge, MA, 2005.

35. Strauss A. and Corbin J. (1998) Basics of Qualitative Research: techniques and procedures for developing grounded theory. Sage Publications, London.

20. Easthope, A., McGowan K., eds. A Critical and Cultural Theory Reader, OU Press, Milton Keynes, 1992.

36. Walker, T. (2008) This gadget rocks! The world's newest musical instrument. The Independent 5 March, 2008

21. Fukuchi, K. Tenori Off http://www.YouTube.com/watch?v=ZNrV8LjRcRU

37. Watts R. (2009) I just want to. YouTube http://www.youtube.com/watch?v=344OpaQCAQI

22. Goldacre, B. Bad Science, 4th Estate, London, 2008.

38. Wikipedia, MacGuffin http://en.wikipedia.org/wiki/MacGuffin

23. Little Boots Little Boots at Glastonbury Festival Pt 1. http://www.YouTube.com/watch?v=YCdQriYrXA8

39. Wikipedia Technological Determinism http://en.wikipedia.org/wiki/Technological_determinism

24. Little Boots little boots READY FOR THE FUN!!! hot chip tenorion cover. http://www.YouTube.com/watch?v=N6tLRCDqJ2c

40. YouTube YouTube Fact Sheet http://www.YouTube.com/t/fact_sheet

25. Little Boots little boots MEDDLE bedroom version acoustic on piano, tenorion and stylophone oo la la. http://www.YouTube.com/watch?v=Zcc8gE54Md8

41. Yule, G. U. A mathematical theory of evolution, based on the conclusions of Dr. JC Willis, FRS . Phil. Trans. of the Royal Society, Ser. B, 213 (1925), 21-87.

26. Little Boots Little Boots performs 'Remedy' at the John Peel Stage Glastonbury Festival 2009 http://www.YouTube.com/watch?v=drFeVGVJ3Zs

42. Zizek S. Looking Awry: An Introduction to Jacques Lacan through Popular Culture. MIT Press Cambridge MA, 1992.

27. LustigAkustik Tenori-On Meets Sax On A Friday Night http://www.YouTube.com/watch?v=kff9b4x921I

43. Zizek S. The Fragile Absolute – or Why is the Christian Legacy Worth Fighting For? Verso London, 2000. 44. Zizek S. In Defence of Lost Causes. Verso London, 2008.

81

Full Papers

Proceedings: NordiCHI 2010, October 16–20, 2010

Training Software Developers in Usability Engineering: A Literature Review Anders Bruun Aalborg University, Department of Computer Science Selma Lagerlöfs Vej 300, DK-9220, Aalborg Oest, Denmark [email protected] ABSTRACT

However, small companies do not even have the privilege of staffing usability specialists and these have to cope with issues such as the constraint of low budgets. In practice this means that small software companies do not have the funds to pay for comprehensive consultancy or staffing of usability specialists [13, 18, 28] as they are expensive to hire [23]. A survey conducted by Gulliksen et al. acknowledges this by showing that usability specialists are primarily employed by medium-sized or large companies [12]. In their study 70 % of the 194 respondents, are employed usability specialists in companies with at least 50 employees, which are categorized as medium-sized [9]. This is also supported by Rosenbaum et al. that shows that most usability specialists are employed in large companies consisting of 250 or more employees [25].

Software companies focusing on Usability Engineering face two major challenges, the first being the sheer lack of usability specialists leading to missing competences in the industry and the second, which regards small companies suffering from the constraint of low budgets, thus not being able to fund usability specialists or comprehensive consultancy. Training of non-usability personnel in critical usability engineering methods has the potential of easing these challenges. It is, however, unknown how much and what kind of research that has been committed to novice training in UE methods. This paper presents a comprehensive literature study of research conducted in this area, where 129 papers are analyzed in terms of research focus, empirical basis, types of training participants and training costs. Findings show a need for further empirical research regarding long term effects of training, training costs and training in user based evaluation methods.

The fact that small companies usually do not have staff with usability competences is expressed as one of the main barriers for incorporating UE in the development process [22]. This is also supported in the survey presented in [25] that shows that 17.3 % of 134 respondents mentioned missing competences as one of the main obstacles.

Author Keywords

Training, developers, usability engineering, literature review.

One way of solving the problems of incorporating UE in software companies is to increase knowledge of the subject across the set of stakeholders [12]. Thus, training nonusability personnel in critical usability engineering methods has the potential of easing problems regarding the lack of usability specialists in the industry, which is experienced in large companies. Training also provides an opportunity for small companies to apply UE methods as the developers themselves are driving the UE process, thus lessening the need to staff usability specialists, which cannot be funded.

ACM Classification Keywords

H5.2 Information interfaces and presentation (e.g.,HCI): User Interfaces -- User-centered design, Training, help and documentation. INTRODUCTION

During the last decade large software companies have increased their focus on introducing usability engineering (UE) methods into the development processes. One challenge for these companies is the sheer lack of usability specialists in the industry, which leads to missing competences across the board and hence, problems with incorporating UE in the development processes [16, 24].

Some studies provide promising insights regarding training of usability novices in UE methods. Metzker and Offergeld describe a software project in which developers participated in contextual task analysis, which motivated the participants to produce components with a high level of usability [22]. Another study presented in [17] describes how a nonusability specialist learned to apply a usability evaluation method. The participant stated that only little experience with cognitive psychology was needed in order to apply the method. Seffah et al. describes the opposite case in which usability specialists having sufficient technical knowhow eased integration of UE methods in the development process and gained wide acceptance from developers [29].

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. NordiCHI 2010, October 16–20, 2010, Reykjavik, Iceland. Copyright 2010 ACM ISBN: 978-1-60558-934-3...$5.00.

82

Full Papers

Proceedings: NordiCHI 2010, October 16–20, 2010

A reason for this could be that their perspectives are more closely aligned with that of the developers.

Papers were not selected if they had a purely pedagogical focus and did not describe training in particular UE methods or rendered too few details to uncover whether the training regarded analysis, design or evaluation methods. This was also the case if papers provided descriptions of particular methods aimed at novices or focused on skills needed by novices but no training was mentioned or given, for instance if novices were used as participants in experiments but details of their training was left unmentioned. Another constraint was if usability experts acted as participants or the methods used were dependent on expert knowledge, in which case the papers were excluded. The same goes for papers describing surveys of current use in software companies and papers focusing on description of courses aimed at UE lecturers. The final constraint regarded cases in which the same study by the same authors was described in different papers. In this case only one of these was chosen where journal versions were prioritized over conference papers, otherwise the latest version would be selected for inclusion.

Although seeming to be a promising approach for solving the above mentioned problems, it is still unknown how much and what kind of research that has been committed to novice training in UE methods. This paper presents a comprehensive literature study focusing on what research has been conducted in the area of training novices in UE methods. The following section describes the method used for identifying relevant research papers. Then an overview of the identified papers will be presented and analyzed in terms of research focus, empirical basis, types of training participants and cost. After this the findings are discussed according to future research needs and the final section presents conclusions. METHOD

This section describes the strategy undertaken to cover the body of knowledge regarding training of novices in UE methods. The overall strategy was based on the iterative process of identifying and reading papers, then identifying and reading potentially relevant papers from references or citations.

Identification process

In order to identify the first set of relevant papers a preliminary screening was performed. This was done using Google Scholar, as this search engine covers scientific papers from a broad set of publishers and proceedings. The search criteria were based on a full-text search in which all the words “training”, “developers” and “usability” were required and resulted in 33,800 records (search conducted December 1st 2009). As it would be too tedious a task to read abstracts from all these papers the first 200 abstracts were read of which 23 potentially relevant papers were selected and read in full. Eight papers were selected as relevant and defined the result of the screening process.

Selection of papers - criteria

Papers were selected as relevant if they described or focused on training of novices in UE methods. Here “novices” are defined according to Bonnardel et al.’s definition: “Lay-designers are people with little or no formal training in either web design specifically or its attendant skills (e.g., database design, graphic design, user interface design, etc.)” [5].

Papers referenced in the selected 8 papers from the screening were marked as potentially relevant. In addition the relevant 8 papers were looked up on Google Scholar which provides a utility to identify which papers are citing these. All citations were also marked as potentially relevant. Subsequently abstracts from all referenced and cited papers were read and papers fitting the selection criteria were read in full. This process continued until closure was reached after 8 iterations.

This definition does not, however, define what “little training” means exactly. For this the definition of Howarth et al. is used: “Additionally, all the evaluators selected for the study had less than one year of job experience related to usability engineering, thereby qualifying them as novices” [15]. Thus, this paper defines novices as persons with less than one year of job experience and no formal training in Usability Engineering methods.

Table 1 presents the statistics from the overall reading process and shows that 4155 abstracts were read and that 286 papers were read in full ending up with 129 actually relevant papers. Note that because of the length of the references these 129 papers are listed on a website [1] and are cited using parentheses and not square brackets, e.g. “(46)”.

Conference and journal papers were selected as relevant sources as these have gone through scientific peer reviews in order to be accepted at conference proceedings or journals. Ph.D. and master thesis’s were also included as they have been reviewed at higher exams. Books were not considered relevant in this study since it is uncertain whether or not these have gone through the same level of review. Table 2 shows how the selected papers are distributed according to publication type.

Table 2 presents an overview of the 129 papers distributed according to publication type. From the table it can be seen that the level of quality of the majority of papers adhere to conference proceedings and journal standards.

83

Full Papers

Proceedings: NordiCHI 2010, October 16–20, 2010

Potentially relevant

Read in full

Actually relevant

Screening

200

23

8

References / Citations

3955

263

121

Total

4155

286

129

Table 1: Statistics from the overall reading process.

Conference proceeding

73

Journal

42

Workshop

7

Thesis

7

Total

129

Table 2: Distribution of papers according to publication type.

Figure 1: Number of papers distributed according to research focus and empirical basis (n = 129).

OVERVIEW OF CURRENT RESEARCH

This section provides an overview of the body of research focusing on training of novices in UE methods.

Practice – Field studies

The remaining categories of research focus consist of 59 papers with the purpose of describing, developing or evaluating UE methods for use in industrial practice. These are labeled as “Practice” in Figure 1 and are divided in 2 categories. Thirteen papers (10 %) have a field focus in which institutionalization of UE methods is done through practitioner training while considering the organizational context. These papers consider company size, staffing and development process and 3 are empirical studies where papers (1) and (36) present studies of integrating UE methods into small-sized companies through developer training and (66) uses a large company as their case. The remaining 10 papers describe non-empirical research by presenting theoretical frameworks for integration of various UE methods into development processes and present suggestions for industry training programs (12, 26, 27, 28, 52, 54, 67, 81, 86, 104).

Research focus

By reading the research questions of the individual papers 3 categories of research focus were identified and are described in further detail below. Figure 1 provides an overview of the 129 relevant papers categorized according to their research focus and empirical basis. University Focus

Seventy papers (54 %) have a university focus in which the main purpose is description, development or evaluation of UE contents for university curricula or pedagogical approaches applied in teaching UE methods to university students. The identified pedagogical papers do not focus on the curriculum contents, but on the way contents are taught, which differentiates this set of papers from those considering contents. Twenty-four are empirical studies of which 14 describe and evaluate courses with respect to student performance and perceived usefulness (6, 8, 9, 14, 24, 25, 55, 65, 90, 91, 93, 94, 120, 123). Ten are pedagogical papers evaluating the comprehensibility or effectiveness of the approach (15, 31, 35, 57, 58, 60, 83, 92, 98, 100), of which 2 are comparative studies of training materials comparing the effect of varying training conditions (15, 31). The remaining 46 papers are nonempirical of which 39 describe development of new curricula contents and course outlines (10, 11, 13, 17, 18, 20, 23, 29, 32, 33, 39, 41, 72, 74, 76, 78, 79, 80, 82, 84, 87, 89, 95, 96, 97, 99, 101, 102, 103, 105, 108, 113, 117, 118, 119, 121, 122, 125, 126, 128). Seven are studies describing new or previously applied pedagogical methods and brief descriptions of lessons learned (7, 34, 37, 38, 56, 69, 71).

Practice – Laboratory studies

Forty-six papers (36 %) focus on description, development or evaluation of UE methods or tools isolated from the organizational context and is thus labeled as “laboratory”. Six of these papers conduct empirical studies of novice performance when applying UE methods of which 5 present comparative studies of differences between usability specialists and novices (3, 85, 109, 111, 129). The sixth paper (46) evaluates the comprehensibility of a novel UE method when used by novices without comparing novices and specialists. Nine of the 46 “laboratory” papers focus on development or descriptions of software tools to support novices in applying UE heuristics or methods. Four of these are empirical studies that describe development of tools and evaluation of these using novices (50, 70, 106, 115) and 5

84

Full Papers

Proceedings: NordiCHI 2010, October 16–20, 2010

Non-empirical

are non-empirical describing the development of specific software applications to support novices (4, 42, 59, 75, 114). Seventeen of the 46 “laboratory” papers have the main purpose of comparing two or more UE methods empirically using novices (2, 19, 21, 22, 30, 43, 44, 48, 49, 53, 61, 62, 63, 64, 68, 77, 107), while 14 papers are empirical evaluations of single methods applied by novices (5, 16, 40, 45, 47, 51, 73, 88, 110, 112, 113, 116, 124, 127).

The table shows that 12 papers of the 59 (20 %) describe non-empirical research targeted at practitioners. The majority of these discuss field issues of institutionalizing UE into software companies and the remaining describes development of tools to support novices. Students

Thirty-two papers (54 %) are empirically based using students for observations on training in methods targeted for use by practitioners in isolation from the organizational context, i.e. laboratory experiments. Most of these papers concern evaluation of a single or multiple UE methods and a few regard studies of pedagogical approaches and tool support.

Summary – Research focus

The above observations show that the majority of papers focus on development, description and evaluation of university curricula content or pedagogical approaches (54 %) and UE methods in isolation from organizational context (36 %). Few of the identified papers (10 %) focus on institutionalization issues in the field by considering the organizational context and that only 3 of these (2 %) are empirical studies on the subject. As this paper seeks to identify the body of research regarding training of novices in UE methods targeted for use in industrial practice, the remaining sections focus on the papers doing so, i.e. the 59 papers in the “Practice” categories shown in Figure 1.

Practitioners

Thirteen papers (22 %) describe empirical studies based on observations of novice practitioners trained in UE methods, which are also aimed for use by practitioners. The papers described in this set have a field focus on institutionalization via training of practitioners and others evaluate UE methods and supporting tools in laboratory conditions isolated from organizational contexts.

Participant types

Table 3 presents an overview of the 59 papers distributed according to their empirical basis and training participants. The matrix is divided in columns representing the empirical basis of the relevant papers, i.e. non-empirical and empirical, where empirical studies may base findings on experiments using either students, practitioners or both as subjects. Note that it was not possible to determine whether the subjects in the empirical studies of papers (15, 73) were students or practitioners, thus an additional category labeled “unclear” was necessary.

Both

A single paper (2 %) uses students and practitioners as the empirical basis. That paper describes a comparative laboratory study where the purpose is to evaluate a UE method using 4 different experimental conditions, of which 1 includes practitioners and the others include students. Summary – Participant types

Summarizing on the above it is shown that most of the current empirical research focusing on training in UE methods, for use by practitioners, is actually done using students as the empirical basis. Additionally, few of the identified empirical studies report of training novice practitioners in UE methods targeted for use in industry practice. Another observation is the fact that only 1 paper includes students and practitioners in the same study.

Empirical basis Nonempirical

Empirical Students

Practitioners

4, 12, 26, 27, 28, 42, 52, 59, 67, 81, 104, 114

2, 5, 16, 19, 21, 30, 31, 35, 40, 43, 44, 45, 46, 49, 50, 51, 61, 62, 63, 64, 68, 77, 85, 106, 107, 109, 110, 111, 112, 113, 127, 129

1, 3, 22, 36, 47, 48, 53, 54, 66, 70, 86, 115, 124

n = 12

n = 32

n = 13

Both 88

Unclear 15, 73

Training contents

n=1

Figure 2 provides an overview of the UE methods which are described and taught in the 59 papers targeting practitioners. For simplicity the methods are categorized according to the usability engineering activities in which they are to be applied [26]. The first activity category is “analysis”, which covers survey or task analysis methods used for needs analysis or user profiling. The second category is “design”, which regards design principles, patterns or prototyping methods applied during creation of the user interface. The third activity category is “evaluation”, which covers usability evaluation methods such as heuristic inspection or usability testing with users.

n=1

Table 3: Papers distributed according to empirical basis and participant type (n = 59).

85

Full Papers

Proceedings: NordiCHI 2010, October 16–20, 2010

Figure 2: Number of papers targeted at practitioners distributed according to training contents (n = 59).

Figure 2 shows that 4 papers report on training in analysis methods exclusively, where 2 are empirical studies (1, 51) and 2 are non-empirical (52, 104 ). It can also be seen that 16 papers focus on training in design methods only, of which 4 are non-empirical (4, 42, 59, 114) and 13 are empirical (3, 16, 19, 31, 35, 61, 62, 66, 70, 86, 106, 115). The majority of papers focus on training novices in evaluation methods solely where a single paper is nonempirical (67) and 29 are based on empirical observations (2, 5, 21, 22, 30, 40, 43, 44, 45, 47, 48, 49, 50, 53, 63, 64, 68, 77, 85, 88, 107, 109, 110, 111, 112, 113, 124, 127, 129).

Figure 3: Number of papers focusing on training cost distributed according to research focus (n = 21). Training costs and laboratory studies

Fifteen of the papers considering training costs have description, development or evaluation of UE methods or supporting tools as a focal point. None of these consider organizational contexts, thus being categorized as laboratory studies. One of the 15 papers is a non-empirical study and describes the development of a tool to support novice practitioners in design tasks (42). The last 14 are empirically based of which 5 use practitioners as participants and 1 uses students and practitioners. The papers evaluate novice use of UE methods and supporting tools, where training contents regard either evaluation or design methods exclusively (47, 48, 53, 70, 88, 115). The remaining 8 papers use students as the empirical basis of which 1 describe development and evaluation of a tool to support practitioners in design activities (106) and the final 7 compare evaluation methods (21, 45, 68, 77, 85, 109, 127).

The remaining 8 papers describe training in several categories of methods where 3 are empirical (46, 54, 36) and 5 are non-empirical (26, 27, 28, 81, 12). Summarizing Figure 2 it can be seen that 11 papers describe training in analysis methods, 24 in design methods and 35 in evaluation methods targeted for use by practitioners. Training costs

This section provides an overview of the reported training costs in the 59 papers targeting practitioners of which 21 papers (36 %) mention training costs as a focal point. Training costs and field studies

Differences in training hours

Figure 3 provides an overview showing that 6 of these 21 papers consider the organizational context, thus having a field focus on institutionalization. The 2 empirical studies are using practitioners as the empirical basis. One of these 2 papers train participants in analysis methods (1) while the other describes training in analysis, design and evaluation methods (36). The remaining 4 papers having this research focus are non-empirical and present theoretical frameworks for integration of various UE methods into development processes. Two of these recommend training in several methods (27, 28) while the final 2 suggest training in design or evaluation methods exclusively (67, 86).

Fifteen of the 59 papers mention training costs in number of hours as the measuring unit of which 7 focus on cost (21, 36, 45, 47, 48, 53, 109) and 8 do not (2, 5, 30, 40, 49, 62, 107, 111). Figure 4 shows a box plot of these 15 papers distributed according to their focus on cost and training duration in hours.

86

Full Papers

Proceedings: NordiCHI 2010, October 16–20, 2010

cost focus. Another, less apparent, indicator is the medians, which are 7.5 for papers with a non-cost focus and 5 for those considering cost. However, these indications are inconclusive due to the few papers describing training costs in hours and due to the overlaps in training hours between papers focusing on cost and those that do not, especially in the first and second quartiles. The fact that there are considerable variations in the number of hours, even if training contents are similar, is also adding to the inconclusiveness. Thus, the needed amount of training in, e.g. evaluation methods is still an open question. Summary – Training costs

In sum only 2 empirical studies focus on institutionalization and organizational contexts in conjunction with training costs. Figure 4 also indicate disagreements on what constitutes a necessary amount of training to educate novices in UE methods, even in the first 3 quartiles, in which all papers describe similar training contents (evaluation methods exclusively). Variations in the fourth quartiles may be explained through differences in training contents, i.e. one paper covers analysis, design and evaluation where others consider evaluation or design methods only.

Figure 4: Box plot of empirical based papers targeted at practitioners (n = 15 papers mention training cost in hours).

The first quartiles (0 – 25 %) show a lower bound of 1 – 3.5 hours for papers not focusing on training cost where the 2 papers contained herein focus on training students in usability evaluation methods exclusively (2, 40). In case of cost focus the lower bound is between 1 – 2.5 hours and includes 2 papers describing training novice students (45) and practitioners in evaluation methods solely (53).

Evaluation Methods

This section focuses on papers reporting of training in usability evaluation methods. Methods related to analysis and design activities are of course also important training areas, however, results from usability evaluation methods have proven to be effective in creating the wake-up calls necessary for companies to start focusing on UE or to increase the awareness of developers [14, 27], which is why this section is dedicated to evaluation methods. From Figure 2 it can be extracted that 30 of the 59 papers (51 %) describe empirical studies of evaluation methods.

The second quartile (25 – 50 %) shows that the number of training hours is between 3.5 – 7.5 and 2.5 – 5 hours for non-cost and cost focus respectively. Papers (5, 107) do not focus on training costs and report on training students in evaluation methods exclusively, which is also the case for paper (21) focusing on cost. The third quartiles show larger differences spanning from 7.5 – 20.9 hours for papers not having training costs as a focal point and 5 – 7.5 for papers that do. These 2 sets contain 4 papers which describe training of novice practitioners focusing on cost (47, 48) and training of students with no cost focus (30, 49). The 4 papers all describe training in evaluation methods solely. The fourth quartiles span from 20.9 – 40 hours in the case of papers without a training cost focus and 7.5 – 40 for the papers mentioning low training costs. The 2 papers not mentioning costs describe educating students in design or evaluation methods solely (62, 111). One of the 2 papers considering low training costs educate novice students in evaluation methods exclusively (109) and the other train novice practitioners in all aspects of UE, i.e. analysis, design and evaluation methods (36).

Figure 5: Number of empirical studies targeted for use in practice distributed according to participant type (n = 30).

Thus, from Figure 4 it can be seen that papers focusing on cost overall spend less hours training participants in UE methods. This is in part indicated by the upper bounds of quartiles 1, 2 and 3, which are lower for papers having a

87

Full Papers

Proceedings: NordiCHI 2010, October 16–20, 2010

Figure 5 provides an overview of the specific evaluation methods taught in the 30 papers distributed according to the participant types. Paper (88) is represented as “both” as the empirical base consists of students and practitioners.

considering the context, in which the UE methods are to be used. This, however, is a crucial area of research, as methods isolated from the real world context potentially are of little practical relevance [31]. In software companies there exist several constraints such as team buy-in, resources, change management, political environments, technology and personnel [19, 29, 31], all of which pose challenges not experienced in studies using university students as participants or evaluation of methods in isolation. Three of the papers focusing on institutionalization (2 %) present empirical studies of integrating UE methods into companies through developer training (1, 36, 66). Results from all 3 studies acknowledge the potential of training novice practitioners in UE methods. Post-project interviews conducted in (1) revealed that participants perceived the training as being satisfying and important to future usability work, an observation similar to the findings described in (36), where the 13 participants graded the training course as 4.25 on a 1 – 5 scale (5 = very good). (66) report of lessons learned statements which indicates that 80 % liked the training course and 20 % did not and (1) and (36) indicate similar reactions where the practitioners wanted more training in UE. (36) reveals more detailed results by showing that the novice practitioners, on the other hand, experienced problems in categorizing the severity of identified usability problems when asked to conduct a heuristic inspection. A design exercise conducted before the training is also compared to one conducted after and shows some minor improvements in proposed designs. Thus, these studies have shown promising short term results, however, there is still a need for empirical research of long term effects on institutionalizing UE via training of novice practitioners. This is important because we need to include and cater for various organizational constraints.

Usability testing

Ten of the 30 papers report of training participants in user based methods and are mentioned as either think-aloud, usability test or user testing in the various studies. Taken together these are labeled as usability testing. Figure 5 shows that a single paper report on training in usability testing using novice practitioners as the empirical base (36) and 9 papers report on student training in usability testing (2, 30, 45, 50, 64, 109, 110, 111, 127). Non-user based evaluation methods

A total of 26 papers describe training in non-user based methods. Ten of these consider training in Cognitive Walkthrough (CW) where 4 are based on observations of novice practitioners (22, 47, 48, 124) and 6 on students (30, 40, 43, 49, 64, 107). The majority of papers report using the Heuristic Inspection (HI) method of which 4 are based on training of practitioners (22, 36, 53, 88) and 10 on students (21, 30, 44, 63, 68, 77, 85, 88, 112, 113). Seven papers describe various non-user based methods such as Metaphors of Human Thinking, Barrier Walkthrough, Abstract Tasks or Programmable User Model, which are categorized collectively as “other non-user based” evaluation methods in Figure 5. This figure shows a single paper describing training of practitioners in a method from the “other non-user based” category method (66) while the 6 papers are student based (2, 5, 21, 43, 44, 129). Summary – Evaluation methods

As mentioned in the introduction of this paper resource demands is one of the main obstacles companies face when wanting to incorporate UE into the development process. For this reason it is not only important to focus on training costs but also the costs of using the particular methods. Considering the 30 papers describing training of usability evaluation methods, 26 (87 %) report of training in the Cognitive Walkthrough, Heuristic Inspection and other non-user based methods, all of which are considered lowcost as no laboratory or end-users are required. There also exist varieties of user based usability testing methods focusing on low-cost, i.e. Instant Data Analysis or Rapid Iterative Testing and Evaluation [20, 21]. However, none of the above 30 papers focus on training participants in a resource saving user based evaluation method.

As mentioned above, it is important to focus on practitioners in industry. Research in UE methods targeted for use by practitioners is reported in 59 of the identified papers (46 %), of which 12 are non-empirical studies and 32 and 13 papers apply novice students and practitioners respectively, while a single paper uses both. Thus, current empirical research mainly focus on using students as participants, which in turn indicates a future need for empirical studies using practitioners employed in the software industry. In total 10 % of the 129 identified papers consider this type of participants. One of the reasons for the focus on using students as the empirical basis is that this approach is less demanding in terms of planning and external involvement. Nevertheless it is still important to get more focus on using practitioners as the empirical basis in order to include constraints such as team buy-in, political environments etc.

DISCUSSION

The above observations show that the current body of literature primarily focuses on development, description or evaluation of curricula contents or pedagogical approaches aimed at university students (54 %). Another large research area is description, development or evaluation of UE methods isolated from organizational contexts (36 %). Less focus (10 %) is devoted to institutionalization issues

As argued in the introduction high resource demands is one of the largest obstacles of introducing UE into software companies, which especially is the case for small software companies that have insufficient funds to pay for comprehensive consultancy or staffing of usability

88

Full Papers

Proceedings: NordiCHI 2010, October 16–20, 2010

specialists [4, 13, 18, 25, 28]. According to Seffah et al. practitioners want to produce software with a high level of usability, but as they work under time and budget restrictions they do not want UE to consume too much of their time [29]. For this reason a considerable amount of research has been committed to developing resource saving methods, mainly in the area of usability testing [20, 21, 23]. Resource saving methods are especially relevant for small companies that do not have the funding to incorporate UE. It also implies that not only should the taught UE methods be resource saving, but this must also apply to the training itself as there is no point in teaching resource saving methods if small companies cannot afford the training. This is also supported in [16] and [30], where the latter shows that most organizations tend to view training as a cost rather than an investment. Although cheap training is important, this review shows that 21 (36 %) of the 59 papers describing methods for practitioner use, focus on training costs. Additionally, 2 of these 21 papers are empirical studies of institutionalization issues in which the organizational context and training of practitioners are considered. Results show that although papers focusing on training costs report a lower number of training hours compared to papers without cost as a focal point, there are still disagreements on what constitutes a necessary amount of training to educate novices in UE methods. In this regard there is a need for obtaining knowledge of how much training is actually needed in order to apply UE methods satisfactorily. There is also a need for further empirical research considering cost and institutionalization issues.

ease the institutionalization of UE, a notion supported in [7]. This study shows that 10 papers describe training in user based evaluation methods, of which 1 uses practitioners as the empirical base. Thus, there is a need for empirical studies of training novice practitioners in user based evaluation methods. Considering the importance of user based methods and previous discussions on training costs, focus on organizational contexts and practitioners, this paper shows that a single empirical based paper is focusing on these crucial issues (36). That paper shows promising first results in integrating UE into a small software company through training in user based evaluations. The training provided positive attitudes towards UE from the developers who afterwards were further motivated in learning about usability aspects in software development. The user based method taught was, however, not reported as being resource saving, thus in part conflicting with the needed aim of introducing resource saving methods via cheap training to small companies. This also constitutes a future research need. CONCLUSIONS AND FUTURE RESEARCH NEEDS

This paper has presented a comprehensive literature study of the research conducted in the area of training novices in Usability Engineering (UE) methods. The 129 identified papers have been analyzed in terms of research focus, empirical basis, types of training participants and training costs. Five key areas for future research needs are identified and listed below.

Usability evaluation methods have proven to be valuable in generating the wake-up calls needed to make companies focus on UE and in increasing the awareness of developers [14, 27]. Thus, evaluation methods could be the potential best way to start a series of UE training sessions. Fonseca et al. [10] and Edwards et al. [8] also describe university curricula in which they begin the courses with letting students evaluate user interfaces, as this increases their awareness of interface problems. The survey described in [25] also states that usability testing in and without a lab are the most preferred methods by software companies. Furthermore several studies indicate that user based testing is superior to inspection or walkthrough methods in this regard. User based tests provide valuable first hand observations of the problems experienced by real users, which in turn increases motivation for making adjustments to the user interface [14]. The reason for this may be located in the fact the user testing provides empirical evidence of the problems at hand compared to theoretical inspections [6]. Additionally, the study presented in [11] indicates that a majority of evaluators prefer user based methods over the inspection methods Metaphors of HumanThinking and Cognitive Walkthrough. A similar study described in [2] shows that user based methods was rated above inspection methods regarding pleasantness of use. Thus, several studies indicate that user based testing potentially increases practitioner buy-in, which in turn may

Firstly 13 papers (10 %) focus on institutionalization issues by considering the organizational contexts, of which 3 are empirical based studies. The 3 empirical studies in this area have focused on measurements of short term effects of introducing UE into the organizations, thus indicating a need for further empirical research of long term effects. Secondly there is also a future need for focus on empirical studies using practitioners as few of the identified papers (10 %) consider this type of training participants. Third, due to disagreements in the current body of literature, this review also shows a need for further empirical research focusing on training costs and the amount of training necessary to obtain a satisfactory level of method usage. Fourth, in addition to training costs, few (2) of the identified papers are empirical studies in which institutionalization and cost issues are considered at the same time, hereby indicating another need. Fifth, as discussed previously, user based evaluation methods seem to provide the best wake-up call for software companies, and this review shows that a single empirical based paper is focusing on training novice practitioners in such a method. This leaves room for future research in this area.

89

Full Papers

Proceedings: NordiCHI 2010, October 16–20, 2010

REFERENCES

Human-Computer Interaction 21, 2 (2006), 173-196. Taylor & Francis (2006).

1. List of selected papers analyzed in this literature review. http://www.cs.aau.dk/~bruun/references.html.

15. Howarth, J., Andre, T. S. and Hartson, R. A Structured Process for Transforming Usability Data into Usability Information. Journal of Usability Studies 3, 1, 7-23 (2007).

2. Ardito, C., Costabile, MF, De Angeli, A. and Lanzilotti, R. Systematic evaluation of e-learning systems: an experimental validation. In Proc. NordiCHI 2006, ACM Press (2006).

16. Ji, Y. G., Yun, M. H. Enhancing the minority discipline in the IT industry: A survey of usability and UserCentered design practice. Int. Journal of HumanComputer Interaction 20, 2 (2006), 117-134. Taylor & Francis (2006).

3. Bailey, G. S. Iterative methodology and designer training in human-computer interface design. In Proc. INTERACT 1993, ACM Press (1993). 4. Bak, J. O., Nguyen, K., Risgaard, P. and Stage, J. Obstacles to usability evaluation in practice: a survey of software development organizations. In Proc. NordiCHI 2008, ACM Press (2008).

17. John, B.E. and Mashyna, M.M. Evaluating a Multimedia Authoring Tool with Cognitive Walkthrough and ThinkAloud User Studies. Journal of the American Society of Information Science 48, 9 (1995), 1004–1022.

5. Bonnardel, N., Lanzone, L. and Sumner, T. Designing web sites: The cognitive processes of lay-designers. Cognitive Science Quarterly 3, 1 (2003), 25-56.

18. Juristo, N., Moreno, AM and Sanchez-Segura, M. I. Guidelines for eliciting usability functionalities. IEEE Transactions on Software Engineering 33, 11 (2007), 744-758. IEEE Computer Society Press (2007).

6. Brown, C., Pastel, R. Combining distinct graduate and undergraduate HCI courses: an experiential and interactive approach. In Proc. SIGCSE 2009, ACM Press (2009).

19. Karat, C. M., Campbell, R. and Fiegel, T. Comparison of empirical testing and walkthrough methods in user interface evaluation. In Proc. CHI 1992, ACM Press (1992).

7. Dayton, T. Skills needed by user-centered design practitioners in real software development environments: report on the CHI'92 workshop. ACM SIGCHI Bulletin 25, 3 (1993), 16-31. ACM Press (1993).

20. Kjeldskov, J., Skov, M. B. and Stage, J. Instant data analysis: conducting usability evaluations in a day. In Proc. NordiCHI 2004, ACM Press (2004).

8. Edwards, A., Wright, P. and Petrie, H. HCI education: We are failing–why? In Proc. HCIEd 2006, Springer (2006). Springer (2009). 9. European Commissions definition Medium-sized Enterprises:

of

Small

21. Medlock, M. C., Wixon, D., Terrano, M., Romero, R. and Fulton, B. Using the RITE method to improve products: A definition and a case study. In Usability Professional's Association Annual Conference 2002.

and

22. Metzker, E., Offergeld, M. An Interdisciplinary Approach for Successfully Integrating Human-Centered Design Methods Into Development Processes Practiced by Industrial Software Development Organizations. In Proc. IFIP International Conference on Engineering for Human-Computer Interaction 2001, Springer-Verlag (2001).

http://ec.europa.eu/enterprise/policies/sme/ facts-figures-analysis/sme-definition/index_en.htm 10. Fonseca, M., Jorge, J., Gomes, M., Gonçalves, D. and Vala, M. Conceptual design and prototyping to explore creativity. IFIP International Federation for Information Processing 289 (2009), 203-217. Springer (2009).

23. Nielsen, J. Usability inspection methods. In Proc. CHI 1994, ACM Press (1994).

11. Frøkjær, E., Hornbæk, K. Metaphors of human thinking for usability inspection and design. TOCHI 14, 4 (2008). ACM Press (2008).

24. Nielsen, J. Finding usability problems through heuristic evaluation. In Proc. CHI 1992, ACM Press (1992). 25. Rosenbaum, S., Rohn, J. A. and Humburg, J. A toolkit for strategic usability: results from workshops, panels, and surveys. In Proc. CHI 2000, ACM Press (2000).

12. Gulliksen, J., Boivie, I., Persson, J., Hektor, A. and Herulf, L. Making a difference: a survey of the usability profession in Sweden. In Proc. NordiCHI 2004, ACM Press (2004).

26. Rubin, J. Handbook of usability testing. John Wiley & Sons Inc., New York, NY, USA, 1994.

13. Häkli, A. Introducing user-centered design in a smallsize software development organization. Helsinki University of Technology (2005).

27. Schaffer, E. Institutionalization of Usability: A Step-ByStep Guide. Addison Wesley, Redwood City, CA, USA, 2007.

14. Høegh, R. T., Nielsen, C. M., Overgaard, M., Pedersen, M. B. and Stage, J. The Impact of Usability Reports and User Test Observations on Developers’ Understanding of Usability Data: An Exploratory Study. Int. Journal of

28. Scholtz, J., Laskowski, S. and Downey, L. Developing usability tools and techniques for designing and testing web sites. In Proc. Conference on Human-Factors & the Web 1998.

90

Full Papers

Proceedings: NordiCHI 2010, October 16–20, 2010

29. Seffah, A., Gulliksen, J. and Desmarais, M. An Introduction to Human-Centered Software Engineering. In Seffah, A., Gulliksen, J., Desmarais, M.C. (eds.) vol. 1, 59-70. Springer, Netherlands (2005).

31. Wixon, D. Evaluating usability methods: why the current literature fails the practitioner. Interactions 10, 4 (2003), 28-34. ACM Press (2003 ).

30. Taylor, M. J., England, D. and Gresty, D. Knowledge for Web site development. Internet Research 11, 5 (2001), 451-461. MCB UP Ltd. (2001).

91

Full Papers

Proceedings: NordiCHI 2010, October 16–20, 2010

Design and Evaluation of Player Experience of a Location-Based Mobile Game Tara Carrigy, Katsiaryna Naliuka, Natasa Paterson, Mads Haahr Trinity College Dublin College Green, Dublin 2, Ireland

{carrigyt, naliukak, patersn, mads.haahr}@tcd.ie ABSTRACT

Augmented Reality (AR) Games and Location-Based Mobile Games (LBMGs) continue to evolve, game designers creating immersive gaming experiences will face unique and unprecedented challenges.

This paper reports on the design and evaluation of player experience of a Location-Based Mobile Game set in Dublin, Ireland in which players act as paranormal investigators hunting for ghosts and gathering evidence of paranormal activity. The paper focuses on players’ experience of engagement and immersion, which was evaluated through a qualitative user study undertaken over a three-day period with the participation of 19 subjects. We first discuss the concept of immersion in gaming and then review related work before presenting the design and implementation of our prototype and the results of our user study. The results show that the experience succeeds in creating a high level of immersion at several stages in the game and that this immersion can be influenced by several factors including usability, control, modes of interaction, aesthetics, flow and, perhaps most significantly, choice of location.

Engagement, Immersion, Location-Based Mobile Gaming

In this paper, we report on a study of player experience of a working prototype of a LBMG. Our primary goal was to test our assumptions about factors that contribute to the creation of engaging, compelling and immersive gameplay. Another motivation was to expand our understanding of players’ engagement and to develop strategies for future development of this game. We begin by exploring established notions of engagement and immersion in gaming. Next we survey related work, focusing on research projects that evaluate player engagement, most specifically in the context of LBMG and more broadly in the context of Pervasive and AR Gaming. We then describe the design and implementation of the prototype and present our set-up and methodology for the qualitative evaluation study. Finally, we present the results of our study and discuss our findings with some strategies for further development.

ACM Classification Keywords

ENGAGEMENT AND IMMERSION

Author Keywords

In the general context of gaming, gamers, developers and researchers describe the players’ experience of engagement or involvement in the game as immersion [3]. The term immersion is often used to describe the experience of losing track of the outside world and the boundaries of the magic circle, the imaginative space in which the game is played [1]. As we have already alluded to, initial understanding of the term immersion in gaming referred to the context of video and computer gaming and is therefore inevitably associated with VR technologies where the player enters or plunges into the virtual game world [5,19]. By stating that immersion is ‘the sensation of being surrounded by a completely other reality’ and ‘the experience of being transported to an elaborate simulated place’, Janet Murray’s definition of immersion reflects this association [19]. In the VR and Virtual Environments (VE) research community immersion is commonly accepted as the qualities of a media that create sensory impact by surrounding the user [9, 29] and for decades this community has defined a set of ‘immersive factors’ almost exclusively in relation to the technology. For example, Slater proposes that ‘immersion can be objectively assessed as the characteristics of a

H5.m. Information interfaces and presentation: User Interfaces—User-centred design; Evaluation INTRODUCTION

Evaluation of players’ experience in any form of digitally mediated gaming almost always refers to engagement, and specifically immersion, as desirable aspects of gameplay. In the general context of gaming, the term immersion is used to loosely describe the degree to which the player becomes involved or engaged in the game. Until recently understanding of immersion in computer gaming has mainly been developed in relation to PC and Console Gaming. As the new contexts of Pervasive Gaming, Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. NordiCHI 2010, October 16–20, 2010, Reykjavik, Iceland. Copyright 2010 ACM ISBN: 978-1-60558-934-3...$5.00.

92

Full Papers

Proceedings: NordiCHI 2010, October 16–20, 2010

technology, and has dimensions such as the extent to which a display system can deliver an inclusive, extensive, surrounding, and vivid illusion of virtual environment to a participant’ [29]. In contrast, however, Witmer suggests that immersion is a psychological state that can be influenced by the immersive tendencies of the individual participant [34]. In VE research, immersion is often measured by presence, ‘the subjective experience of being in one place or environment, even when one is physically situated in another’ [34]. In this context it is common to assume that presence can only be experienced through immersion in the VE [29,34] and that greater presence leads to greater engagement.

Players expect to be rewarded for the time, effort and attention spent learning, what Lindley refers to as, the ‘gameplay gestalt’ [17], the pattern of player interaction with the game system. Cairns and Brown’s grounded theory of immersion suggests that as players pass through the barriers associated with the preliminary stages of psychological engagement, they are rewarded by a more imaginative experience that Cairns and Brown describe as total immersion [3,28]. Engagement and immersion therefore are part of a continuum of player experience, where initial engagement is associated with learning the game mechanics and is a prerequisite for deeper total immersion, which is the performance of the game mechanics in order to experience the gameplay. In this sense immersion is similar to flow, ‘the state in which individuals are so involved in an activity that nothing else seems to matter’ [6] and where attention becomes so focused that ‘sense of time is altered and sense of self is lost’ [3]. Flow is the ‘state at the boundary between engagement and immersion, of being totally absorbed in meeting the constantly unfolding challenge’, which Lindley also refers to as immersion in performance [17]. For the purposes of this paper, we take the essential difference between experiences at either end of this continuum as follows: engagement is more active than immersion, and total immersion is closer to flow, the immersive state achieved when this activity is balanced by the players’ skills and becomes so transparent that the player can experience deep but effortless involvement in the game [3,25].

Game theorists Salen and Zimmermann offer another perspective on immersion and presence when they observe that equating the experience of immersion with being ‘sensually’ transported to another separate, simulated reality so convincing that ‘the player truly believes that he or she is part of an imaginary world’ [27] is a fallacy that assumes the technologically mediated experience can become so sophisticated that it will be capable of ‘fully illusionistic experiences indistinguishable from the real world’ [27]. They propose that, in the context of gaming, the immersive fallacy of engagement through simulation is a one-dimensional approach to player engagement that ‘misrepresents how play functions’. In contrast Salen and Zimmermann suggest that engagement ‘occurs through play itself ’ in a process of double consciousness [27], whereby the players become engrossed in the game while simultaneously aware of the medium and the ‘artificiality of the play situation’ and thus perceiving play as something separate from, but nevertheless connected to the real world. This understanding of immersion does not disregard the power of simulation and the potential for engagement as a result of presence but it does present player engagement as a more complex and active process and this approach is especially relevant in the context of LBMG where the player is constantly shifting focus between the media and the physical environment and where gameplay moves fluidly between mediated and directly felt experience.

Immersion in gaming is considered to be more than the objective characteristics of the technology but also the subjective experience of the player as s/he plays the game. Recognising the different levels and stages of engagement and immersion in gaming, many researchers have emphasised that the experience of immersion is a transient and often fleeting state [3,10,23]. Part of the focus of this paper is to establish the factors that influence the players’ experience of engagement and immersion as they move through the game, and in particular highlight those specific to the context of LBMG.

Player engagement and immersion in games is essentially performative and participatory and occurs as a direct result of active involvement, attention and interaction. In games, players are actively rather than passively engaged in the gameplay experience and therefore the quality of the players’ interaction with the game system, through the game mechanics, is a key factor influencing immersion. Players enter the game through the game controls [3] and mastery of the game mechanics (the interface between the player and the game world) is the ‘price of admission’ [25] to the gameplay; as players learn how to use the game controls to play the game, their focus switches from conscious attention to internalised or tacit knowledge. This state of deep involvement shortens the distance between player and environment and results in immersion [4].

RELATED WORK

Researchers evaluating player experience in LBMGs have explored several factors that impact on player engagement and immersion, the most common focus being on the significance of location. Some LBMG blur the boundary between fact and fiction and leverage a combination of local legend and history to engage players, heighten their awareness and change their perception of the everyday places, where the games are located [2,22,34]. By evaluating REXplorer, a LBMG set in the Medieval city of Regensburg, researchers found that interweaving real world landmarks into the game’s narrative, which they based on a mixture of local folklore and medieval history, added an authenticity to the gameplay that increased players’ immersion [2]. Similarly, Visby Under and Frequency

93

Full Papers

Proceedings: NordiCHI 2010, October 16–20, 2010

1550, are both LBMGs that use the immersive qualities of location to bring history to life and enhance player engagement [30]. Likewise Nisi et al, in their evaluation of Media Portrait of the Liberties, also report achieving deeper story immersion and an enhanced sense of place by overlapping and connecting narrative with location [20]. Riot! 1831, a locative interactive play, used audio to recreate the riots of 1831 and GPS technology to situate this media at the exact location in Bristol where the riots occurred [24]. Participants reported empathy with the people involved in the riots and a sense of ‘walking in their footsteps’. Research revealed a high level of immersion as direct result of mapping historic narrative onto relevant locations but also exposed that immersion is a transient state that can be affected by environmental factors associated with that location. Evaluation of LBMGs created using the Mediascapes toolkit highlights how making use of aspects in the physical environment can make the games more engaging. In recognising the difficulty but nevertheless high potential for immersion associated with a tight coupling of game content and environment, the Mediascapes researchers propose a strategy for ‘designing for coincidence’ whereby environment features or the behaviour of other people become aligned with the content of the game and are perceived by the player as highly engaging ‘magic moments’ [24]. Finally in relation to choice of location, researchers evaluating LOCUNET found that locating LBMGs in public spaces could have a negative impact on immersion if players felt inhibited or threatened by the presence of passers-by [7].

may increase immersion in Locative Games [5]. In Songs of the North, researchers used an audio interface to create an engaging location-aware mobile game because they felt that audio was a more appropriate medium for creating an immersive hybrid gameworld [15]. Related work has also shown that the mode of interaction in LBMG can affect player engagement. For example, REXplorer [2] uses gestural based interaction to physically engage players, while the iPerg [32] researchers recommend that the use of authentic or real world interaction models, such as walking around to discover elements of the game, can increase player immersion. In addition to evaluations of LBMGs that focus on immersion and engagement, other related work includes research that has focused on methodology and models for evaluating player experience in the context of Pervasive Gaming. In the Pervasive GameFlow (PGF) model [13,14], Jegers develops a model of player enjoyment of Pervasive Gaming based on the GameFlow model originally developed by Sweetser and Wyeth [31] and following Csikszentmihalyi’s concept of flow as described in the previous section. With regard to immersion the PGF model recommends that pervasive games should enable players to shift focus from virtual and physical parts of the game and seamlessly transition between the gameworld and everyday contexts [13]. Ermi and Mäyrä’s SCI model of immersion [10] is also useful for evaluation in the context of LBMG because it opens up the possibility of measuring player immersion from the perspective of the player. The SCI model of immersion considers player immersion from three perspectives: Sensory, Challenge-Based and Imaginative and in doing so encompasses key elements of gaming and is closely aligned to game models proposed by the gaming and game theory community, for example Lindley’s Simulation, Gameplay and Narrative taxonomy [16,17].

As well as transforming the players’ perception of space with story, some researchers in the field of LBMG have explored techniques for augmenting space with visuals and sounds, which players will perceive as part of the hybrid gameworld, and have evaluated how this impacts on player engagement and immersion. For example, researchers compared player engagement in AR Façade to the previous PC version of the interactive narrative game Façade, and found that while the players of the AR version felt an increased sense of presence this did not necessarily result in increased player engagement [9]. Early user evaluation of TimeWarp, an outdoor augmented reality game played in the city of Cologne using a HMD, revealed that GPS tracking issues combined with the graphical realism of the AR objects, limited players’ sense of presence because the objects floated around too much or did not appear to be real [28]. Evaluation of Interference, a LBMG using AR technology to superimpose 3D models on the mobile’s live video stream, revealed that the aesthetics, which helped to blend the physical world into the game world, were the most significant factor contributing to player immersion in the game [33]. In relation to acoustic augmentation, research in VE has shown that accurately synthesized spatialised sound can increase presence [11] and researchers have shown how these techniques can be used to create realistic localised sound and a 3D audio soundscape that can be easily navigated by players and thus

DESIGN AND IMPLEMENTATION

Viking Ghost Hunt (VGH) is a location-aware adventure game, based on a Gothic ghost story set in Viking Dublin (800-1169). In this game the player assumes the role of a paranormal investigator and moves around the city hunting for ghosts, collecting evidence and solving the mysteries of haunted Viking Dublin. The game is designed as a singleplayer, immersive gameplay experience, in which the player is an active character in an unfolding drama. In order to progress the game, the player must unlock a sequence of location-specific narrative fragments, by completing a series of challenges and missions. Throughout the VGH game design, we focused on two design goals: to give meaning to play by maintaining the aesthetics of role-play and to exploit the characteristics of the locations by integrating them into the game to create an engaging gameworld.

94

Full Papers

Proceedings: NordiCHI 2010, October 16–20, 2010

Role-Play

the gameworld using a combination of sound, sight and touch. Collecting evidence by photographing ghosts using the camera function is also an intuitive and familiar mode of interaction.

In order to preserve the illusion or magic circle the players’ assumed role as a paranormal investigator must persist at all times. We set this up by presenting all of the aspects of gameplay, including the aesthetics, challenges and modes of interaction, in the context of paranormal investigation. Equipped with a paranormal investigation device and an audio headset, the players’ goal is to conduct investigations by hunting down ghosts and gathering evidence, by making audio recordings or taking photographs of ghostly apparitions. Players use Google maps to search for the location of an alleged haunting in Viking Dublin. Players begin their investigation by using the Radar (see Figure 1) to locate paranormal activity in the region. This Radar also includes an Electromagnetic Frequency (EMF) meter, which informs players of the levels of paranormal activity and the strength of the manifestations. A dowsing meter (based on the metaphor of dowsing rods used to divine water or paranormal activity) helps to guide the player in the direction of the haunting. Once the player has located the paranormal activity, s/he can begin to collect evidence that will substantiate the alleged haunting. Players use the Frequency Scanner (see Figure 1) to tune in and decode the static, in order to reveal the ghostly voices in the white noise, in a process reminiscent of Electronic Voice Phenomenon. Ghost View can be used to photograph visual manifestations. Players’ can review their progress in Casebook, a database that stores players’ evidence and contains extra narrative about the ghosts.

Figure 1. Radar (left) and Frequency Scanner (right). Use of Location

We set the game around St. Audeon’s, a Medieval church and public parkland situated within the Old Viking City Walls of Dublin because of its historical and thematic relevance and potential for a varied player experience. With the intention of blurring the distinction between fact and fiction, we based the game’s narrative on a mixture of local ghost stories and urban myths as well as factual history. We included contrasting locations in the mission design in order to test the impact of location on gameplay. The main location was ideal for the theme and atmosphere of the game and we were able to integrate features such as the Medieval style church, windy ramparts, high stone walls, dark archways and damp and eerie laneways; all classic motifs of the Gothic ghost story genre. Narrative was used to draw player’s attention to physical features in the environment and integrate them as part of the gameworld. Dialogue, in combination with sound effects, helped players imagine a scene from the past. For example, in the Viking era, the River Liffey was much broader than it is today and hence the grassy area outside the old city walls was once a beach; we recreated this scene using sound effects and descriptive narrative. Similarly, the narrow laneway and medieval staircase at the side of the church was transformed into the ‘Lane of Hell’ by a dense interplay of atmospheric dialogue enhanced by realistic sound effects. Exploiting the characteristics of the locations is depends on presenting relevant content to the player in relation to their position in the gameworld. We set this up by specifying regions where the assets would be triggered and then used GPS to detect when players had entered these regions. Since GPS technology is prone to lagging and errors, efforts were made to minimize inaccuracies by creating large regions

User Interface Design

The User Interface (UI) design was guided by the aesthetics of role-play, as described above, and the intention of facilitating embodied interaction [8] with the game as is appropriate for situated and location-based gaming. To these ends, we designed the UI to look and feel like a tool for paranormal investigation and to evoke a similar cognitive process as someone investigating a radioactive area with a Geiger counter. By employing a visual style of presentation reminiscent of mechanical devices of the past, we established a set of affordances that suggested the appropriate mode of interaction to the user while simultaneously reinforcing the aesthetics of role-play and atmosphere of the game. By way of engaging the player in the processes of searching for and collecting evidence, we applied the principles of direct manipulation by enabling phenomenological interaction via touch screen gestures and multimodal feedback in response to players’ actions and movements. As much as possible, the intention was to reference real world actions as interaction metaphors so that players’ could easily master the gameplay gestalt by leveraging their existing tacit knowledge [4,17]. For example, navigating the gameworld is achieved through the activity of searching for ghosts by walking around the location and using the Radar to detect paranormal activity. The players’ proximity to the ghost is indicated using a combination of visual representation and a fluctuation in audio and tactile feedback, hence the players can navigate

95

Full Papers

Proceedings: NordiCHI 2010, October 16–20, 2010

and positioning them at locations with good GPS reception (e.g. open areas not shadowed by trees and buildings). In situations where we needed to place content close to buildings, for example in the ‘Lane of Hell’ scenario, we used the physical properties of the space to compensate for the lack of GPS updates in this region. In this example, the assets were triggered by regions at either end of the lane thus limiting the experience of this content to the confines of the lane. Despite this approach, the resolution remained fairly low in precision and therefore the assets could only be loosely mapped to specific physical features.

enough apart to give the device time to re-load each of the individual assets as required. EVALUATION

We evaluated the VGH game by running a series of field trials over a three-day period. The 19 participants who took part had varying degrees of prior experience of gaming and ranged in age from 18-48. We evaluated the participants’ response to the game using a post-game qualitative questionnaire loosely based on a combination of established models of player engagement and immersion [10,14,31,34]. The questionnaire included both open ended and bounded questions designed to obtain feedback on engagement, control, usability, presence and impact of location. While an in-game evaluation technique such as ‘think-aloud’ may have captured a more immediate and visceral response to the game, this was not practical due to resource limitations. We also captured data during the game using the system logs. While this data did support the information gathered by the retrospective questionnaires it did not offer anymore insight into the player experience and therefore we will not present this data as part of our evaluation.

Figure 2. Using Radar’s touch screen zoom (left) and Ghost View (right). Augmented Reality

RESULTS AND OBSERVATIONS

Various Augmented Reality (AR) techniques were used to transform and augment the players’ visual and auditory perceptions of the location. With the aim of creating the illusion that the ghost is actually present in the player's surroundings, the players’ view of the scene is augmented by animations displayed on top of the live video stream in the viewfinder of the camera. The effect is perceived as more realistic when the ghost is anchored to a point in the scene and therefore appears to remain stationary in the physical surroundings even when the player moves the camera. Due to the impracticalities of placing optical markers into a public space and the expensive overhead of performing real-time image analysis, it was not possible to employ computer vision techniques to anchor the ghost overlay within the scene. Instead we tracked the direction of the viewfinder using the device’s orientation sensors and although (due to the unreliable nature of the sensor data) the ghost representation was not completely fixed within the scene, we speculated that the visual floating effect would be credible in the context of ghostly apparitions. Acoustic augmentation of the location was achieved by using spatialisation and reverberation techniques [19] to realistically match the game’s sound effects to those naturally occurring in the environment and as a way of deliberately creating ambiguity between the game and the real world. In addition a localised soundscape was created by varying the volume of the sound effects in response to the players’ movements around the hybrid gameworld. Audio and visual augmentations were constrained by the resource limitations of the Android development platform which requires the assets to be loaded into memory on demand. The implication of this constraint is that the file size needed to be very small thus hampering the creation of realistic effects and also the assets had to be positioned far

General Overview

Evaluation of the user study results reveals that overall participants were engaged in the game. Feedback from three of the bounded questions illustrate that the majority of participants (79%), agreed or strongly agreed, that it was a fun experience, time passed quickly and they felt engaged by the game. All participants reported varying levels of engagement at different stages of the game. In general, the findings support previous research that indicates immersion is a variable and transient state, which progressively deepens as players became more skillfully engaged in the gameplay [3,4,17,23]. Many players describe becoming more immersed as the game progressed and as they became more comfortable using the UI or accomplishing the challenges and as the story began to unfold. “It was more immersive once I got used to the UI and the story started to link one sighting to the next” “I didn’t feel immersed until near the end, the beginning isn’t so engaging, as I was trying to get used to how to control the game and find the evidence at the same time” Interaction and immersion in gameplay

The user study findings show that by far the most engaging aspects of gameplay were connected to the mastery of the game mechanics. For the most part, players found the search mechanic satisfying and enjoyed the process of physically moving around the environment hunting for evidence of paranormal activity and the sense of achievement when they accomplished the challenge. “The gameplay was lovely, the different modes and the physical moving around to track things down.”

96

Full Papers

Proceedings: NordiCHI 2010, October 16–20, 2010

Overall players used the multimodal features of the Radar interface as a guide when searching for ghosts; 79% of participants mainly used the graphical interface but also used audio and tactile feedback to support the visual feedback. Since information about the position of the ghost was presented in different ways, players could choose their preferred method or tools to help them find the ghosts. For example, testers might focus on visual, tactile or audio feedback or a combination all features.

“The most difficult part of the game was the device reliability: sometimes the GPS ‘ghost’ would point me in the wrong direction.” Usability issues occurred due to latency and unresponsiveness of the touch screen and sensors as a result of resource limitations while new assets were being loaded. For example sometimes the zoom function of the Radar became slow to react and therefore participants found it awkward to use and often “required moving slowly and stopping regularly for re-calibration”

“I used the proximity meter (i.e. number readout) to determine my distance.”

Impact of location

“The sound effects and the pulsing of the handset helped point me in the right direction.”

A significant finding of the user study was the impact that the location had on player engagement and immersion. Almost all the participants agree (84%) that the location contributed to the overall game experience because the historical significance of the main site gave context to the narrative, while the atmosphere of the place enhanced the players’ perception of the supernatural theme of the game.

Many players described collecting evidence as the most immersive part of the game because they were simultaneously engaged in the activity of recording evidence while also receiving a reward for progressing in the game in the form of a story fragment.

“The historical significance of the place influenced my perception of the game” Most participants had not previously visited this site and commented that this added a sense of intrigue to the location. “The church had walls, nooks and crannies; also, the alleyway by the side of the church, the high walls, the archway all added to a kind of ‘mysterious’ atmosphere.” Participants confirmed that their experience of the game varied in relation to the ambience of the locations; the aesthetics of the main site were much more conducive to the atmosphere of the game than the commercial streets and housing estates at the periphery of the historic site. The relative remoteness of the church grounds also enhanced player immersion because the location was quiet and less populated than the urban streets and therefore afforded uninterrupted engagement.

Figure 3. Player interacting with ghosts in St.Audeons.

“It felt satisfying. The ability to find audio and images was a good mix. I never got bored searching. The most immersive part was after I’d located a ghost or artifact, and am trying to record it”

“The isolated lane around the side of the church was more atmospheric and hence I felt more immersed”

“Felt most immersed with dialogue and with use of camera, felt engaged when I caught the floating apparitions. Dialogue gave more direction and hence more interactive.” Impact of control

The study shows that GPS and usability issues negatively impacted on the player experience with the result that only 63% of players said they felt in control of the game. In the main, efforts to reduce the negative impact of GPS errors were successful. In particular the strategy implemented in the ‘Lane of Hell’ region, as described in the design section, resulted in the delivery of relevant content at appropriate locations despite lack of GPS coverage. Nevertheless some players did experience bad GPS service and this caused a high degree of distraction and frustration.

Figure 4. Players at St. Audeon’s Arch and the ‘Lane of Hell’.

Players’ noted that the use of headphones helped to engage them in the game by reducing distractions without completely disconnecting them from the ambient sounds of

97

Full Papers

Proceedings: NordiCHI 2010, October 16–20, 2010

the location, which were still hear audible with the headphones on.

this helped to engage the player in the game because it communicated the players’ progress.

“The only thing that distracted me was a passing ambulance as I caught a sound ghost”.

“Felt kind of like a personal experience – like it was tailormade for me and the ghost was egging me on to the next clue”

Players found the locations on the busy urban streets outside of the church afforded a less immersion experience due to distractions such as noise levels, passersby and traffic safety issues. It was difficult to hear the audio at the points located near to the roadside due to the traffic noise and some felt that safety was an issue in this scenario. While a few players felt self-conscious playing the game in public, especially if they were in the way of other pedestrians when they stopped in the middle of the pathway to listen to the narrative, for the most part players took less notice of other people in areas such as the park as they felt these areas were already designated for play and one participant commented: “I was fully immersed and didn’t take notice of tourists and workmen around me”.

79% participants felt the game audio sounded natural in the environment and some testers refer to incidences when the game sound blended with the ambient sounds of the environment in a manner that created ambiguity as to which was virtual and which was real. “The natural sounds of the city were kind of a nice extra layer of atmosphere rather than a distraction, there were children playing and a church bell ringing and a little bit of wind.” Players also felt rewarded and satisfied when they encountered and photographed visual assets and their remarks reveal the cumulative success of this game mechanic.

Role-play and aesthetics

“Increasing use of visual (rather than auditory) clues heightened the experience as the game unfolded.”

In general, participants agreed (84%) that the visual appearance and sounds of the UI made the device feel more like a tool for paranormal investigation than a phone. In comparison less participants (68%) reported feeling like a paranormal investigator when they were playing the game and therefore we can conclude that while the audio and visual aesthetics supported the impression that the device was real, the game mechanics did not support role-play to the same extent. Nevertheless, participants described several features of the UI that they associated with paranormal investigation and felt that the headset helped make the role-play feel genuine.

Many participants reported feeling ‘engaged’, ‘entertained’, ‘excited’, ‘surprised’ or ‘satisfied’ when they encountered a visual manifestation of a ghost, others described the visual ghosts as ‘scary’, ‘realistic’, ‘cool’ and ‘good fun’ and one player remarked ‘the effect of it floating there was good, especially the first time when I wasn’t expecting it’. In general, the visual assets, were not as convincing as the audio ghosts. It seems that the limited animation sequence negatively impacted on the user experience and some participants’ remarks reveal that the animations were barely perceptible.

“I did feel like a paranormal investigator, hunting things down was great and twiddling with the knobs to tune in”

“Didn’t seem real. Could have made the ghost appear like it was more than just a picture and part of the scenery.”

“I had the device to find the object and I felt like the ghost hunters from the film”

Narrative Immersion

In addition, and reflecting the cumulative nature of immersion, many participants reported feeling more engaged in the role-play as the game progressed.

The most diverse feedback we received was in relation to narrative; while some players remained highly engaged with the game mechanics and less involved by the content of the story, others found that performing these mechanics gave players access to immersion in the narrative and “when clues began, the game was more exciting”

“After a few discoveries it was easy to get into the game and to feel more like an investigator.” Overall the findings show that participants perceived the game assets, and in particular the audio assets, as realistic and this contributed to player engagement. Participants reacted emotively to the dialogue spoken by the audio ghosts and many described the experience of listening to these ghosts as ‘scary’, ‘spooky’, ‘haunting’, ‘atmospheric’, ‘realistic’, ‘believable’ and as a result ‘engaging’, ‘immersive’ and ‘entertaining’.

“I wanted to figure out how to get the evidence… but I was not really interested in what the evidence was” “Story interests me most. I just like the ‘adventure’ idea of tracking down the information.”

“I found it [the ghost speaking] one of the most immersive features of the game – really enjoyed it.”

Some players noted that gameplay was a more essential aspect to the game but had they had more time they would have explored the narrative elements in more depth, while others stated that they were not that interested in using the casebook because they were more interested in gameplay.

Many players commented that when the ghosts spoke to them, the experience felt very personal and rewarding and

“The interaction with the game was more about achievement ….I would like to play the game again and

98

Full Papers

Proceedings: NordiCHI 2010, October 16–20, 2010

focus more on the story and not be distracted figuring out the equipment.”

focus on the feedback which they found most intuitive. Comparing user response to our two modes of collecting evidence (recording audio and taking photographs) also supports the argument that more familiar, ‘everyday’ interactions are easier to perform in this context because they are more immediately intuitive and serve as more expedient modes of interaction because they rely on players’ existing tacit knowledge [4,17]. These findings compliment the evaluation of REXplorer where players faced difficulties performing gestural interactions due to their novelty and associated learning curve [2]. As with previous research [7] our study reveals that players were less likely to experience immersion when they felt conspicuous playing the game in a public space however we also found that players were less inhibited when performing more ‘everyday’ interactions in public spaces designated for leisure activities. This confirms Jegers recommendation that LBMG should support a ‘seamless transition between the gameworld and everyday contexts’ in a manner that does not break the social norms associated with these contexts [13].

Most players felt they did not have time to use the casebook to find out more about ghosts but some players remarked that if they were playing under normal circumstances they would have used casebook more. Many players admitted to skimming the game and only did what they had to do to get to the end. “I skipped this bit [reading casebook] as I was lagging behind and felt I needed to move on to the next find.” The players who did use casebook reported that the backstories were interesting and increased their empathy with the characters. As with the other game features, immersion in the story increased as the game progressed. “As the game progressed, I certainly did get more immersed. As you find the different clues and items and realise you are following a particular ghost character it becomes more engaging” DISCUSSION

Control

Challenge-based Immersion

Since players enter the game through the game controls, usability issues can have a negative effect on immersion. At times GPS errors disrupted players’ enjoyment of the game and this well documented issue remains for anyone designing a LBMG. However in our user study, despite the fact that many participants did not always feel fully in control of the game many reported high levels of engagement and immersion and this seems to confirm Norman’s theory [3,17] that if the overall experience is interesting enough then users will be prepared to put up with minor usability issues. While some participants experienced difficulties at the beginning of the game, the findings of the study show that as players progressed, for the most part control ceased to be an issue. Having said that our recommendation for UI design for a LBMG is that the interface be made as simple and easy to control as possible because players have less tolerance for learning new interactions in this scenario and the inevitable GPS issues already create a difficulty level that need not be further extended by a complicated interface. Our study confirms Jegers’ recommendation that in relation to control, players in the context of LBMG need to be able to easily pick up the game and begin playing [13].

The results of the study show that engagement in play itself and immersion in performance were experienced by the players as the most immersive aspects of the VGH game. Furthermore the study shows that engagement in the game mechanics is cumulative, not only because as players’ skills increase they become absorbed in a state of flow, but also because by engaging in these mechanics, players become exposed to other gameplay elements such as reward and narrative. We found that players entered the game through the game mechanics and in this sense our study of immersion supports the research we reviewed at the beginning of this paper [3,10,25,27]. Reflecting Ermi and Mäyrä’s evaluation of PC games, we found that when we applied their SCI model of immersion [27] in the context of LBMGs, challenge-based immersion was also the type of immersion most frequently experienced by the players. For many players, recording audio or photographing ghosts was the most immersive part of the game because as well as creating atmosphere and communicated elements of the narrative, collecting evidence helped the players advance in the game. The study also indicates that this form of immersion is more easily achieved in the context of LBMGs when challenges are presented to the player in the form of easily controlled real world interactions.

Augmented Reality

The results of the user study show that choice of location can impact dramatically on sensory immersion in the context of LBMGs and by enhancing and transforming the players’ perception of their surroundings, atmosphere can become a unifying and highly engaging aspect of the game. The findings reveal that the aesthetics of the assets most effectively contribute to the players’ sensory immersion when they become subtly blended with the real world; for example, when the ambient sounds of the game merged with the environmental sounds of the location. The results

Real World Interaction

The results of the study confirm previous research indicating that in the context of LBMGs the most successful game mechanics are those that combine real world activities as part of the core interaction [2, 32]. For example, the search mechanic involves players walking around hunting for ghosts and the game supports a phenomenological approach to this challenge by providing multimodal feedback to players’ movements thus enabling players to

99

Full Papers

Proceedings: NordiCHI 2010, October 16–20, 2010

also reveal that if this juxtaposition is not smooth enough, as was the case with the AR visuals, then the players will perceive it as jarring and unconvincing and hence a disruptive rather than engaging factor. This supports the findings of the evaluation of TimeWarp [28] as reviewed in the related work section. While to some degree the floating effect of the AR visuals was perceived as acceptable within the ghostly theme and context of the VGH game, it is clear that the visuals need to be more integrated into the surroundings in order to achieve a credible visual effect. Stabilising the visual overlays within the scene would also enable designers to employ the cinematic technique of ‘mise-en-scene’ whereby explicit visual attention could be drawn to specific landmarks as the player views them through the lens of the device. This could also address the fact that some players felt the game did not facilitate enough engagement in the surrounding environment.

participants to achieve imaginative immersion to the extent that they almost had to close their eyes in order to block out their physical surroundings [23]. Our study reveals that while many aspects of the gameplay contributed to imaginative immersion, for example roleplay and the atmosphere of the location, the story itself was not a hugely engaging aspect of the game. While players enjoyed the activities associated with discovering the evidence and piecing together the clues in order to proceed to the next find, very few participants spent time exploring the stories behind the evidence, which together made up the over-arching narrative of the game. There are a number of reasons for this outcome, the first being that, as described above, players needed to master the game controls in order to gain access to the unfolding narrative. However the main reason was that this mode of interaction, reading text on a small screen, is inconvenient in an outdoor location-based context. In future developments of the VGH game we intend to focus on conveying all of the narrative through short audio and visual assets and in doing so take advantage of the more accessible descriptive potential of these mediums.

“Paradoxically the game encourages looking at the screen more than the surroundings.” These findings highlight one of the unique challenges of creating presence in the context of LBMGs using the processing power of a mobile phone. Given the limited technical resources of the device and the significant potential of leveraging the location, it seems that at the moment the greatest potential for creating presence and sensory immersion in this context lies in the power to subtly transform the existing characteristics of the location into a sensually engaging gameworld by enabling players to simultaneously focus on virtual and physical parts of the game.

CONCLUSION

In this paper we have observed that player experiences are cumulative. As players learned to gain control of the game mechanics they achieved deeper immersion in the narrative and imaginative aspects of gameplay. Supporting engagement and immersion in LBMGs is also dependent on utilising appropriate real world interactions and carefully selecting locations in relation to thematic relevance, atmosphere, aesthetics, safety issues, lack of potential distractions and social context. By leveraging the existing characteristics of the location, Augmented Reality techniques can be used to transform the surroundings in order to create a hybrid gameworld where the player can become immersed in the flow of play while simultaneously remaining connected to the real world.

Imaginative Immersion

Overall we found that players needed to achieve a degree of effortless engagement before they could fully experience imaginative immersion and therefore this type of immersion is not dissimilar to immersion in the general gaming context [3,10]. The choice of location can influence the level of imaginative immersion not only by supporting the atmosphere and narrative but by also ensuring safety and lack of distraction. In some instances the narrative and audio completely transformed the players perception of the landscape, for example, one participant referred to the ‘wistful description of the River Liffey’ and the soundscape that described the original course of the river.

ACKNOWLEDGMENTS

We wish to thank the rest of the VGH team (Roisin Cotton, Daniel Crowley, Alan Duggan, Tina Hedayet, Soren Kristian Jensen and Sean O’Reilly), the NDRC and all those who participated in the user study. REFERENCES

“The church and surroundings were smashing, really wonderful moment standing outside it and being told about the shores of the Liffey once being underfoot.”

1. Adams, E. and A. Rollings. Fundamentals of Game Design. Prentice Hall, Upper Saddle River, NJ, USA, 2006.

We speculate that part of the reason it was easy for players to imagine this alternate scene was the fact that the location where the game was situated was not that dissimilar to the one being described and was conducive to imaginative immersion due to the aesthetics of the location and lack of distractions. These findings contrasts with the evaluation of Riot! 1831 where researchers found that lack of visible structures that related to the experience made it difficult for

2. Ballagas, R., Kuntze, A. and Steffen P. W. Gaming Tourism: Lessons from Evaluating REXplorer, a Pervasive Game for Tourists. In Pervasive Computing 2008, LNCS vol 5013, Springer (2008), 244-261 3. Brown, E., & Cairns, P. A grounded investigation of game immersion. In Proc. of CHI 2004, ACM Press (2004), 1279-1300.

100

Full Papers

Proceedings: NordiCHI 2010, October 16–20, 2010

4. Calleja, G. Revising Immersion: A Conceptual Model for the Analysis of Digital Game Involvement In Situated Play. In Proc. of DiGRA Conference (2007).

19. Murray, J. H. Hamlet on the Holodeck: The Future of Narrative in Cyberspace. Free Press, New York, 1997. 20. Nisi, V., Oakley, I. and Haahr, M. Location-Aware Multimedia Stories: Bringing Together Real and Virtual Spaces. In Proc. of ArTech, ACM Press (2008).

5. Cater, K., Hull, R., Melamed, T., and Hutchings, R. An investigation into the use of spatialised sound in locative games. In Ext. Abstracts CHI 2007, ACM Press (2007), 2315 – 2320.

21. Norman, D. A. The design of everyday things. New York: Doubleday, 1988.

6. Csikszentmihalyi, M. Flow: The Psychology of Optimal Experience. New York: Harper and Row, 1990.

22. Paterson, N., Naliuka, K., Jensen, S.K., Carrigy, T., Haahr, M., Conway, F. Design, Implementation and Evaluation of Audio for a Location Based Augmented Reality Game. In Proc. of AMC Fun and Games, ACM Press (2010).

7. Diamantaki, K., Rizopoulos, C., Charitos, D & Tsianos, D. “LOCUNET: Location-based games as media communicating social meaning”. In Souza e Silva, Α. & Sutko, D. (eds.), Digital Cityscapes: Merging Digital and Urban Playspaces, Peter Lang Pub, 2009.

23. Reid, J., Geelhoed, E., Hull, R., Cater, K., Clayton, B. Parallel Worlds: Immersion in location-based experiences. In Proc. of CHI 2005, ACM Press (2005).

8. Dourish, P. Where the Action Is: The Foundations of Embodied Interaction. Cambridge: MIT Press, 2001.

24. Reid, J. Design for coincidence: incorporating realworld artifacts in location-based games. In Proc. of ACM DIMEA, ACM Press (2008).

9. Dow, S., Mehta, M., Harmon, E., Mateas, M., Presence and Engagement in an Interactive Drama. In Proc. of CHI 2007, ACM Press (2007).

25. Rigby, S., Ryan, R., The Player Experience of Need Satisfaction ftp://ftp.immersyve.com/PENS_Sept07.pdf

10. Ermi, L. and Mäyrä, F. Fundamental Components of the Gameplay Experience: Analysing Immersion. In Proc. of the DiGRA, (2005) 15-27.

26. Rouse, R. Game Design: Theory and Practice. Wordware Publishing, Texas, 2005.

11. Hendrix, C., & Barfield, W. The sense of presence within auditory virtual environments. Presence: Teleoperators and Virtual Environments, 5(3) (1996), 290-301.

27. Salen, K. and Zimmerman, E. Rules of play: game design fundamentals. MIT Press, Cambridge, Massachusetts, 2004. 28. Seah, M. and Cairns, P. From Immersion to addiction in videogames. In Proc. of HCI 2008, vol 1 BCS, 55-63.

12. Herbst, I., Braun, A. K., Mccall, R., Broll, W., Timewarp: interactive time travel with a mobile mixed reality game. In Proc. MobileHCI '08, ACM Press (2008), 235-244.

29. Slater, M. and Wilbur, S. A framework for immersive virtual environments (FIVE): speculations on the role of presence in virtual environments. Presence: Teleoperators & Virtual Environments, Vol. 6 Issue 6, (1997) 603-617

13. Jegers, K. Pervasive Game Flow: Understanding Player Enjoyment in Pervasive Gaming. Computers in Entertainment. ACM Press (2007).

30. Souza e Silva, A., and Sutko, D. M. Digital cityscapes: merging digital and urban playspaces. Peter Lang Publishing, 2009.

14. Jegers, K. Pervasive GameFlow: Identifying and Exploring the Mechanisms of Player Enjoyment in Pervasive Games. Ph. D. Thesis, Dept of Informatics, Umeå University (2009).

31. Sweetser, P. & Wyeth, P. GameFlow: a model for evaluating player enjoyment in games. In ACM Computers in Entertainment, 3(3) ACM Press (2005), 124.

15. Lankoski, P.; Heliö, S.; Nummela, J.; Lahti, J.; Mäyrä, F.; Ermi, L. A Case Study in Pervasive Game Design: The Songs of North. In Proc. of NordiCHI, ACM Press (2004), 413–416.

32. Waern, A., Montola, M., Stenros, J. "The three-sixty illusion: designing for immersion in pervasive games". In Proc. of CHI 2009, ACM (2009)

16. Lindley C. Game Taxonomies: A High Level Framework for Game Analysis and Design. Gamasutra (2003) http://www.gamasutra.com/view/feature/2796/game_tax onomies_a_high_level_.php

33. Wetzel R., McCall R., Braun A-K., Broll W. Guidelines for Designing Augmented Reality Games. In Proc. of the 2008 Conference on Future Play, ACM Press (2008).

17. Lindley, C. A. 2004. Ludic Engagement and Immersion as a Generic Paradigm for Human-Computer Interaction Design. In Proc. of ICEC2004, Springer (2004), 3- 13.

34. Witmer, B. G., & Singer, M. J. (1998). Measuring presence in virtual environments: A presence questionnaire. Presence-Teleoperators and Virtual Environments, 7(3), 225-240.

18. McMahon, A. “Immersion, Engagement and Presence: A Method for Analysing 3-D Video Games.” In Wolf, M.J.P and Perron, B. (eds.), The Video Game Theory Reader. London: Routledge, 67-86, 2003.

101

Full Papers

Proceedings: NordiCHI 2010, October 16–20, 2010

Understanding the Everyday Use of Images on the Web Boon Chew*, Jennifer A. Rode+, Abigail Sellen^ *University College London, +Drexel University, Microsoft Research Cambridge UCL Interaction Centre, 8th floor Malet Place Engineering Building, London WC1E 7JE [email protected] ABSTRACT

aspect of our personal lives.

This paper presents a qualitative study of domestic Webbased image use, and specifically asks why users access images online. This work is not limited to image search per se, but instead aims to understand holistically the circumstances in which images are accessed through Webbased tools. As such, we move beyond the existing information seeking literature, and instead provide contextual examples of image use as well as an analysis of both how and why images are used. The paper concludes with design recommendations that take into account this wider range of activities.

For some time now, researchers in Information Retrieval (IR) have argued that we need to look beyond simple models of search to support what people really do on the Web. They have argued that a model which assumes that users formulate goals, specify queries, and then look for results to satisfy those goals oversimplifies what people do with the Web [9, 15]. This is sometimes called “directed search” or “look-up”. A more recent emphasis on “exploratory search” stresses the ways in which users are often opportunistic when they use the Web, and that their strategies can be heavily reliant on browsing and serendipity [9, 15]. Others have gone one step further pointing out that “searching” or even the broader term “information seeking” assumes that Web use is purposeful, even though research shows that Web users may sometimes have no explicit goals at all [23].

Author Keywords

Image search, domestic, photowork, Web use, field study, diary study ACM Classification Keywords

H5.m. Information interfaces and presentation (e.g., HCI): Miscellaneous.

Despite this, the simple look-up model continues to dominate in terms of driving the underlying design not only in text search, but also in specialized image search tools: users enter a set of keywords, and a rank ordered array of images is returned. Of course there are many specialized sites which focus on images (such as Flickr and Facebook) and also innovations in the way images are searched for (including searching by color [26, 10], or other visual features [10, 19]), but the fact remains that mainstream image search, at least in terms of the user experience, is not significantly different from any other kind of keyword search on the Web.

INTRODUCTION

Despite widespread use of images on the Internet [1, 7, 20], little is known about people’s behavior or motivations when accessing images via the Web and in the course of everyday life. There are studies on image search, but these tend to focus on innovations in the technology itself [e.g. 8, 19] or the performance of different techniques as assessed by various kinds of objective measures such as the relevance and ranking of returned images given a set of keywords [e.g. 2, 20]. Further, in characterizing the behavior in question as image “search”, this, in itself, makes implicit assumptions about what users are doing. In this paper, we want to show how a deeper exploration of what people do with images, and why they access them from the Web, goes beyond simple notions of search. We want, instead, to put on-line image use in the broader context of users’ activities and intentions. We choose a domestic context here to reflect the fact that Web use, including image use, is now firmly entwined in every

This study asks whether, like general Web search, this view of what users do with images on the Web is, in fact, too narrow, and thus whether we are missing opportunities to deliver more compelling kinds of Web-based tools. It does this by looking at the range of behaviors and motivations for engaging with images online from a user’s perspective. We limit the scope of this study in the following ways: First, we focus on the everyday ways in which people search for images on the Web from home, as opposed to purely professional use. We also constrain our enquiry to images which are not within people’s own personal collections, such as home photo and video collections. Largely this is because this topic has been covered extensively elsewhere [e.g. 12, 22] and the issues raised are likely to be quite different from searching for images for which one has little familiarity. Finally, there is no doubt that one of the main uses of images in the domestic sphere

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. NordiCHI 2010, October 16–20, 2010, Reykjavik, Iceland. Copyright 2010 ACM ISBN: 978-1-60558-934-3...$5.00.

102

Full Papers

Proceedings: NordiCHI 2010, October 16–20, 2010

is pornographic [3, 24]. Because of the highly personal and sensitive nature of this aspect of image use, we chose not to include this in our work.

in-depth interviews with users to delve into the strategies, motivations, and contexts where individuals engage with images on the Web. Our approach was to conduct in-depth interviews, but also to ground these using a careful record of participants’ activities over an extended period of time— a method used previously to investigate more general Web use behavior [23]. This way, one can capture a realistic “slice” of image search activities in the context of everyday life, along with the rich descriptions and stories that accompany them. This approach generally means using a relatively small sample of participants, but gathering a detailed set of data about each person in question.

Our contribution is to provide deeper insight into the ways in which people engage with images on-line. We will refer to this behavior as image “access” or “use”, to reflect the fact that, as we will show, these behaviors go beyond searching and browsing. In addition to providing this deeper understanding through contextual examples, a second contribution is to use this to show how the design of Web-based tools might better support the ways in which people want to use images from the Web. Accordingly, we develop a framework for classifying the various motivations for image use, and reflect on the implications of the range and diversity of these activities.

METHOD Participants

With this aim we undertook a study of nine individuals’ domestic image use practices, who were recruited through snowball sampling. Our participants included five men and four women ranging in age from 24 to 59: one participant was under 25, four were aged between 26-35, three between 36 and 45, and one over 46. With the exception of one person (a graphic designer), we recruited participants with professions that were not image-intensive. This included: a bookshop assistant, an accountant, a civil servant, an eco-blogger, a recruitment agent, a student, a homemaker, and a retiree who actively volunteered for charities. Our main criterion for selection was also that they reported using images from the Web at least 5 times a week. All participants used the internet for at least 30 minutes a day and were familiar with general Internet activities such as searching for content on the Web, browsing image galleries, and manipulating images on the Web (e.g. downloading, enlarging, and viewing).

RELATED WORK

Much of the literature on image search is technical in nature, focusing on new ways in which to increase the relevance of retrieved images through, for example, better matching of queries to images [5], better tagging of images with meta-data [27], or ways to do content-based image retrieval [13, 29]. New features and interfaces too, form a considerable body of work, exploring concepts such as new ways to cluster and present results, [see 1 for a review]. New and specialized image search tools are also appearing [19] but most are specialized to particular communities. In terms of understanding users’ intentions or behaviors when using images on the Web, there is much less pertinent research, and most of this concerns image search behavior in organizations and professional contexts [1, 2, 6, 16, 30]. Here, though, there are some interesting findings, pointing to the fact that, in contrast to general Web use, image professionals [16, 30] are more heavily reliant on browsing. More general analyses of image search logs [1, 10] also reveal that, in comparison to general Web search, image search relies on rapid browsing of many more pages of search results per query, involves the exploration of many more paths per query, and much more iterative modification of queries per session. One might reasonably conclude from this that image search is either more reliant on flexible navigation and exploration than mainstream Web searching, or that the tools are less successful for helping users find what they need.

Procedure

The study employed a diary study method [21] similar to the one used previously in Sellen et al. [23]. A diary study was chosen over an ethnographic approach given the likelihood of the experimenter’s presence influencing the environmental validity of findings combined with the impracticality gaining access for 24/7 ethnography. On a simple diary form, participants were asked to log a short description of each Internet activity involving images. Participants were told to log all Web-based activities in which images (such as illustrations, photos, and graphics) were deemed to be important in the accomplishment of their tasks. This could include using images in order to accomplish a goal, finding an image as an end goal in itself, or engaging with images in other ways that might not involve explicit goals. When in doubt, participants were encouraged to include rather than exclude an activity. They were also asked to exclude activities which involved images from their own personal photo collections. In addition to a short description of each activity, they were also asked to log the activity start time. They were told that these entries only needed to be sufficient to trigger a longer discussion of each activity in the ensuing interview. When necessary, participants’ browser histories were used to fill in gaps in the data and to validate the diary entries.

In terms of more in-depth studies of users’ behaviors and intentions, most studies of the everyday use of images outside the workplace focus on people’s use of their own personal photo collections. For example, Kirk et al. [12] provided an overview of “photowork”, or activities people perform with their digital photos after capture but prior to end use such as sharing, focusing on practices in the home. Other studies have addressed other aspects of personal photo use in the home, such as browsing [11], archiving [22] and sharing [17], particularly in the use of photo sharing Websites such as Flickr [14, 18]. Taken together, whether in professional contexts or otherwise, there are very few studies which have conducted

103

Full Papers

Proceedings: NordiCHI 2010, October 16–20, 2010

Participants recorded an average of approximately 3 activities per day (see Table 1).

understanding of the value of images and the activities surrounding it.

Initial interviews introduced participants to the purpose of the study and the diary study method, and were used to gather demographic information about the participants and their households. After initial interviews, participants were contacted via phone or in-person for follow-up interviews approximately twice a week, each interview lasting between 30 to 60 minutes. The frequency of interviews depended on how many activities had been logged in a day. If fewer than 4, then the interview would be delayed until more activities were recorded. However, the interval between interviews was never less than 3 days, or more than 9 days.

Participant Pseudonym Elle Sue Florence Gracia Colin Elizabeth Andy Tom William

Overall Total 18 17 18 22 20 21 17 16 30

No. of days per person 3 9 12 8 10 8 9 9 8

Average No. of Imagebased Activities per day 6 2 1.5 3 2 2.5 2 2 4

Table 1. Overview of number of image-based activities logged per participant. (Participants’ names have been changed to protect their identities.)

During these interviews, participants were asked questions to explore both the “why” and the “how” of each activity logged, including why they were looking for specific images, how often the activities occurred, the process of the activity, internal or external influences on that activity (e.g. an email reminder or a high-level goal), and other contextual aspects (e.g. the environment or situation as the activity was taking place). Interviews were audio-recorded. All participants were given a shopping voucher for their participation on completion of the study. .

Table 1 provides a summary of the number of activities collected for each participant over the course of the study. The total number of image use activities in our corpus was 179, ranging from 16 to 30 activities per person over the course of the study. The average number of image use activities across different participants ranged from 1.5 to 6 per day. Strategies and Tools for Image Access

The data show that participants used one of four main ways to access and engage with images on the Web: they either used image-specific search tools, used social networking sites, visited domain-focused Websites, used Websites featuring maps, or used some combination of the four. The most frequent method was the use of domain-specific Websites that contained images (e.g. news, blogs, ecommerce Websites). Image-specific search tools (in this case exclusively Google Image Search) were used less than 10% of the time, despite its straightforward interface and relative popularity.

FINDINGS

We begin by providing an overview of the activities we examined, including the operational definition of image use we adopted for the purpose of the analysis. Next we look at the “how” of the activities we explored, highlighting the technologies and strategies used by participants in accessing images, and summarizing key observations regarding commonly used Internet sites. Third, and most importantly, we examine the “why” of these image use activities, and present an analytical framework (a taxonomy) for understanding users’ motivations for domestic image use.

Image-Specific Search Tools

Google Images is a general image search tool that retrieves images based on keyword queries, and the only such tool we saw used in the course of the study. Only five participants reported use of Google Images. They used it infrequently (less than 10% of the time) and mostly to support quick searches for specific images that were easily identifiable via a keyword (e.g. “fitflops”, “Van Gogh”). One participant, Gracia, used Google Images more frequently compared to other participants. She spoke English as her second language, and sometimes used Google Images to learn new English words. In one example, she used several keyword combinations to physically describe an outdoor cooking stove in order to find recognizable images and subsequently “guess” the correct term. Another participant, William, attempted several keyword combinations on Google Images to discover humorous pig photos. While this is a small sample, this suggests that tools such as Google Images are appropriate for goal-directed searches where users have ideas for a specific keyword and expected search result, occasionally supporting exploratory searching involving

Overview of Image Use Activities

In our instructions to the participants, and in our analysis, we sought to gather information on any activities in which the Web was used to obtain image-based content of significant importance or value to the participants during the activity. For example, if an image was present on a Website but was ignored, we did not count this as an activity of interest. Obviously, these judgments are subjective. However, we relied on participants to make those judgments, taking at face value the explanations for each activity that they offered up. This means that the analysis that follows includes activities where finding an image may not necessarily be the end goal. It also means that many of the activities considered here may not necessarily be “searching” or “browsing” tasks--images may be accessed for other purposes. This broader definition of image use rather than search, per se, aims to embrace a larger spectrum of image-related activities in order to get a more grounded and realistic

104

Full Papers

Proceedings: NordiCHI 2010, October 16–20, 2010

keyword variations. However, the fact that this is done infrequently either means that there are other effective ways of answering such queries, or that the need to do keyword search on images happens relatively infrequently.

engaged in more focused search. For example, William routinely used eBay’s advanced search feature to routinely locate new listings of items to bid on, followed by scanning thumbnails that matched his specific requirements.

Social Networking Sites

The examples above show how domain-specific Websites can be more effective than general image search tools (e.g. Google Images) due to features that are designed to support search strategies that are relevant to users of that site and the tasks that they engage in (e.g. searching for new items to bid on eBay).

Facebook, while primarily designed for social networking, provides a comprehensive photo gallery feature that members can use to upload images. Some participants made use of Facebook photo galleries to see what their friends were up to and posted comments on certain photos as a means of informing them that they had seen the photos or to amuse them. There were elements of serendipitous discovery as participants often visited photo galleries by clicking on a thumbnail of that gallery located on their Facebook home page. The activities involving Facebook showed how behavior is motivated by participants’ goals to connect with friends [28], and how the thumbnails and photo galleries catered to this need.

Maps

Maps are a unique type of image use, as they are visual in nature and afford various visual tasks such as exploration and recognition. The study observed the use of interactive maps (e.g. Google Maps) as well as illustrated (static) maps. They are distinguished from domain-specific Websites because they were used to provide confirmation and assistance for geographical and spatial navigation. For example, maps were used to confirm the location of a particular landmark (e.g. the location of a real estate property) or to obtaining the general direction to walk from a specific venue.

Only two participants used Flickr during their image related activities. One participant visited Flickr to view photos from her friends and sister about once a week to find out news about them, while another participant browsed Flickr mainly for amusement or inspiration.

Summary

Domain-specific Websites Containing Images

In summary, we can see that there were many different ways in which the participants in the study obtained images they were interested in, or otherwise used them in the context of larger activities. More than half of these were interwoven with routine Web use, often leading to serendipitous browsing and exploration. Others involved specific kinds of search strategies tailored to particular tasks, and thus making use of specific Websites designed for those tasks. Interestingly, fewer than 10% of the activities we examined actually involved the use of specially designed tools for image search. In order to understand this more deeply, we turn now to examining users’ motivations behind these activities.

The majority of image use activities involved domainspecific Websites such as news sites, company Websites and e-commerce sites (on average 42% of the time). Most of these were routine activities, such as reading the news or checking the weather, sometimes leading to other activities through serendipitous discovery of images. Some activities were occasional, but focused around a topic of interest such as property hunting, whereby participants were searching for specific types of images such as property floor plans. In all cases, images were important or even critical parts of the activities in question. It became clear that one reason that many of the image use examples were linked to domain-specific Websites was that they were interwoven into participants’ normal routines when using the Web. For instance one participant routinely visited the BBC Website for news and the weather. Another looked at Glastonbury event photos which led to deeper exploration of articles and more images relating to artists and bands.

Motivations behind Image use

The primary focus of this research was to understand the underlying motivations when people engage with images on the Internet. An analytical framework consisting of thirteen search motivation themes subsumed under four higher-level categories was constructed using grounded theory. These themes are concepts that were based on codes we assigned to each activity based on the various motivations behind how images were used. Some activities fall under multiple themes, and thus are not mutually exclusive. Table 2 summarises these categories and shows the overall number of examples in the corpus falling into each high level category. We now describe each of these categories, along with their sub-categories, in more detail.

Other times, participants were observed making routine visits to certain Websites in the course of more directed search. Elizabeth’s use of a property agent’s Website to examine properties for purchase, which was characterized by her search for floorplans and photos of the property, is an example of this. In this case, she commented on the importance of images: “The photos I usually look for are to get a better feel for the house, and the floor plans are very important as well–especially for knowing how to decorate the place”.

Learning and Research

The most frequent motivation for image use can be summarized as for the purpose of “learning and research”. Here we found a whole range of activities involving images that were undertaken in order to gain new insight and ideas on particular themes, topics or concepts. However, within

Some participants also performed routine tasks involving ecommerce sites (e.g. Amazon, eBay) and, once there,

105

Full Papers

High-level Categories

Proceedings: NordiCHI 2010, October 16–20, 2010

Frequency of Corpus Overall

Learning and Research

123

Image Access as Secondary goals

52

Recreating or Connecting to Remote Experiences

77

Images as the Objects of Communication

20

The key to these activities was that they were specific and specialized to the participants’ particular interests, and could be gleaned from sites which provided additional contextual details along with the images. This tended to happen as a matter of routine, where sites that were considered both trusted and familiar were visited regularly to check for new content.

Subcategories Images to support ongoing interests or research Images to satisfy curiosity Images for visual discovery Images as ideas Images as alternative answers Images as aids for geographical orientation Images as indexes Images for connecting to remote places Images for re-living past experiences Images for connecting to people and their lives Images to connect to personalities Images to do the work of communication Images for social interaction

Images to Satisfy Curiosity

Participants occasionally engaged in more focused and directed search for images to satisfy their curiosity about a particular issue or question, which can be considered another way in which images were part of learning. Fieldnotes for Tom: Tom was reading that people were claiming that TV presenter Christine Bleakley was older than her reported age. He did a search for her photos out of curiosity on various Websites through Google text search, but he couldn't find anything conclusive. He said in response that he often engages in random browsing on a daily basis–“it’s quite a nice thing to do before you go to sleep”. Fieldnotes for Andy: There was a picture of Nick Grinthorpe on the rotating banner at the top of the BBC Website, and Victoria Hesketh was next to him (of whom he was a fan). He clicked on the image which led to an article, and there were about 7 to 8 photos of her playing for a show, and he looked at them all.

Table 2. Framework for motivation for image use including the total number of instances of the corpus falling into each high level category. (Note that any one activity can fall into more than one category). this larger category, we found four main reasons why this occurred, as we shall describe. Images to Support Ongoing Interests or Research

Both Andy and Tom explained that their curiosity led them to look for images, confirming previous work which found that image search queries were often motivated by “curiosity or pursuit of entertainment” [7].

Similar to Marchionini’s concept of “searching to learn” [15] or Sellen et al’s “information gathering” [23], many image-based activities were in support of deepening or updating one’s knowledge with respect to existing interests, hobbies or research topics. These activities were characterized by regular, repeated activity around specific topics or domains which represented a significant part of the overall corpus (40% overall). Whether it be about parenting, modeling, sports, family history or politics, the use of images to support ongoing interests formed anywhere from half to three quarters of the examples of image use for learning and research.

Unlike the previous category, where images were entwined with regular visits to familiar sites, this kind of image use tended to be more focused around particular queries. This was reflected in the way this was accomplished, namely, mostly through visiting various Websites found through regular text search, or by exploring further on the site where the object of curiosity was found. Images for Visual Discovery

Moving on from supporting particular themes or topics, sometimes learning derived from the viewing of large collections of images where similarities and differences in visual features played an important role. This kind of learning was characterized by novelty for participants, where they specifically commented that they learned something new.

For example, some participants who were looking to purchase property consistently examined thumbnails for details such as the condition of an apartment and the external area of the location from apartment specialty sites. Elle, who was a part-time model, regularly followed the work of models and designers, and Gracia often searched illustrators’ portfolio Websites.

Fieldnotes for Sue: Sue was searching on Amazon for "meat mincers" but was unsure of what meat mincer she needed to get. As she was browsing the search results, she realised that there were two types of meat mincing devices – a clamp-type and a table-top type. In the scenario above, it was obvious from the images that there were two types of meat mincers from the product photos returned, as they were visually distinguishable.

Fieldnotes for Elle: Elle received an email from [model A], so she went to modelmayhem.com to see [model A’s] photos - she wanted to see the new work she had been doing since the last time she chatted with her. Later, on Facebook, she viewed photos of another model, [model B], after posting a comment on her wall. She looked at her photos because "she's gorgeous and a very nice person" and that she looked like a celebrity. Fieldnotes for Gracia: Gracia explained how she was looking for new work from [photographer A] on his Website. She found some new art there, then started searching for more of his work on Google Images, which she said was easier as it catches her attention.

In another example, as mentioned earlier, William used Google Images to search for humorous images of pigs by attempting different keyword combinations. Here, he used the phrase “weird pig”, and discovered an image of a pig with tusks:

106

Full Papers

Proceedings: NordiCHI 2010, October 16–20, 2010

the customer, who was in front of him. The whole process took about 5 minutes.

Fieldnotes for William: He went to Google image search, tried to think of a variation of pigs that he hadn't used before, and entered "weird pig" and got a lot of results… He selected these photos because he'd never seen anything like them before - one was of a pig with tusks – a very striking image, "a very exciting pig".

These activities show how images can sometimes be more efficient than text in delivering answers or offering up information. Having the choice of text or image in both of the cases described allowed for greater flexibility and efficiency.

The element of discovery in both scenarios depended on the visual similarity or dissimilarity of the images (meat mincers and pigs) within the search results. Keyword queries also allowed participants to experiment with different search outcomes in order to explore these features along various dimensions such as shape, type, size, and so on.

Images as Aids for Geographical Orientation

Images can act as navigational aids, especially in combination with maps, to confirm the location of a particular landmark or for orientating oneself within a map. Although participants were already familiar with the location, they used images to obtain visual cues (e.g. landmarks, directions) to aid the physical navigation of a place.

Images as Ideas

A final way in which images were used for learning and research was in a more inspirational capacity. Rather than looking for information updates, or accruing additional knowledge about familiar topics, on the contrary, a few participants browsed images to be inspired and get new ideas for creative projects:

Fieldnotes for Florence: Florence thought she would drop by the Japan Centre to buy things on the way home. On her Web browser, she searched for "google maps japan centre london", which led her to the Japan Centre Website. On the “location” page of the Japan Centre site, there was an embedded Google Maps widget, which she used to find out where the Piccadilly Tube station was to get her bearings right. She also checked the address of Japan Centre, and noted the photo of the front entrance of Japan Centre at the top of the page because “it [was] easier to recognize the place”. As she closed the browser window, she made a mental note of the address, the photo of the entrance, and that it was on Piccadilly Street.

Fieldnotes for Gracia: Gracia likes to look at other illustrators’ work to learn from them or to see how far she's grown as a graphic designer. She called this constant ogling at other people’s work a "fixation". Fieldnotes for Sue: There were a lot of beautiful images on this blog — she would look at the images before reading the content, as the images were very important to her. She visits the blog regularly because she wants to update herself about DIY, recycling and design.

The above illustrates Florence’s use of the shop’s entrance as a way to identify the correct place. She also visually used the map to identify direction, match the address with the geographical space, and presumably the distance of travel as well. It is also interesting to note despite amount of information, she felt confident not to write it all down for future reference, as though she was depending solely on her mental model of the journey, made up of visual parts such as the shop’s entrance and pre-existing knowledge of the area. This shows how images can be useful placeholders for things that are useful for judging geographical placement, orientation, distance and meaning.

Here, the important aspect of these activities was finding large collections of images which would surprise and be stimulating. Key here was the ability to browse quickly through many different high quality images around very high level topics, rather than particular kinds of objects. Image Access as Secondary Goals

Images are sometimes used to facilitate other activities. This is characterized by an activity where a participant has a primary goal, but performs sub-tasks where images are used. There were three distinct types of scenarios where this pattern was observed, as we shall describe.

Images as Indexes

Because images are often easier to visually scan, participants made use of images as indexes to locate an item on lists or grids, such as in search results, product listings and catalogues. For example, Elizabeth’s search for a radio program on the BBC iPlayer Website returned search results with illustrations or artwork representing each program. Because she recognized the artwork belonging to the radio program she wanted, it was relatively easy for her to locate that program.

Images as Alternative Answers

In some activities, images were used as alternative ways of quickly finding an answer to a query. For example, Gracia’s recipe search for an authentic Mexican dish returned American versions of the recipe, which she didn’t want: Fieldnotes for Gracia: Gracia first looked for recipes for “arroz a la mexicana” via the main Google search but the results returned inauthentic versions of the dish. She then used Google Images and looked for images of the dish that looked authentic, and clicked on the image that led to the correct recipe.

A similar pattern was observed with Tom and Colin who identified specific DVDs they wanted on sites like play.com and find-dvd.co.uk just by looking at the cover art. This activity of visual browsing was essential as each cover art is unique, and subtle differences in the cover art can make a difference in actual product features such as whether it is a special edition DVD or a boxed set. Again, this represents an economical alternative to scanning

Another example was William’s search for a specific book for a customer waiting directly in front of him: Fieldnotes for William: He was searching for a specific bible for a customer, and the book supplier had a catalog online. He went to the Website and searched for "pocket bible". He clicked on a few options, and the picture was there, so he showed it to

107

Full Papers

Proceedings: NordiCHI 2010, October 16–20, 2010

headline and descriptive text for entries in a sequence, especially when it involves visually scanning for a known image or visual pattern.

These examples show how the images can provide a sense of nostalgia. It seemed that seeing details such as “bits of grass” were critical to the experience.

Recreating or Connecting to Remote Experiences

Images for Connecting to People and their Lives

Participants also used images to understand, connect to or imagine experiences that were otherwise remote, either spatially or temporally. Here, the rich visual nature of images allowed participants to experience Web-based information in ways that text-based content could not. Here, quality and fidelity of the images was important as we will discuss.

Images also represented a way for participants to get to know their friends better, or keep abreast of their whereabouts and activities. These activities often occurred regularly, repeating on a daily to weekly basis. An example of this was Florence’s regular visits to her sister’s photos on Flickr. Fieldnotes for Florence: Once a week, Florence checks her Flickr account to check on new photos from friends. She decided to visit Flickr because her sister was involved in a fashion show recently and so might have uploaded some photos… [However], she didn't see anything new… and left. She uses Flickr as a way of checking up with her sister.

Images for Connecting to Remote Places

Images were sometimes used by participants to “get a feel for a place”, referring to the sense of wanting to experience a specific physical location not previously visited. In one activity, a participant was looking at his friend’s photos of Wembley Stadium on Facebook:

The scenario above illustrates the habitual process participants often went through to get news. In this case, the images represented events or activities pertaining to the participant’s friends or family members. This was a common observation with participants using Facebook as well, which was often accompanied by social interactions such as comments or tagging [4].

Fieldnotes for Colin: Colin found it interesting how his friend managed to capture Michael Owen taking a goal at Wembley Stadium, as well as photos of players celebrating the goal. He had never been to the stadium before so it was nice to see inside. There were very close shots of the player and other players celebrating. He didn’t think you could get so close to the station. It made it feel very exciting.

Images for Connecting to Personalities

This involved multiple photos of a given location showing various perspectives of the space, particularly from a perspective seen from someone else (his friend). Details like the seat numbers, close-up photos of the players, and scoreboard helped Colin to visualize the experience of being at the stadium.

Some participants used images in order to connect to people they didn’t know personally, such as celebrities— not as gossip, but to somehow feel closer to them, or understand them. For instance, during our study Michael Jackson passed away, and several participants used viewing photos as a way of understanding his personality and the stories or situations surrounding his life.

Images for Re-living Past Experiences

We also found that images helped as a way of reconnecting people to a past experiences for familiar places from their past. This was the case even though they were not personal photos as most work on reminiscing assumes [25]. For example, Florence was informed of a snowstorm that hit the town of Dunedin, New Zealand, where she had previously lived:

Fieldnotes for Sue: She found that Michael Jackson died when everyone was talking about the news on Facebook… While she read the news, she looked at the pictures that accompanied the photos, all the while asking – “Did he really die? How did he die?”, reflecting that she was learning about this personality through the photos as well… So the photos helped her see his condition just before his death… Finally, she said she was satisfied from piecing together a believable story after browsing through all those photos.

Fieldnotes for Florence: Florence had heard from a friend that there was a snowstorm in Dunedin, New Zealand, where she used to live. She enlarged every thumbnail because it was a better experience, saying that it was more fun and satisfying and that she could see more details, such as bits of grass, and being able to identify which building that was. She missed Dunedin, and the photos helped her re-live the experience.

The above illustrates how some images have narrative qualities that are also valuable in helping users connect to the experiences of people. Here the participant was looking at the details of Jackson’s face to understand the discussions surrounding his plastic surgery and skin condition—fidelity was critical to the task at hand. As users sometimes hypothesize about personalities’ lives, these photos help users build believable stories surrounding these figures.

Another example showed how Elle was browsing through photos of a hotel she stayed in once: Fieldnotes for Elle: Elle loved looking at the photos of the hotel she stayed at one time, saying how it was “such a beautiful place”. She says she keeps looking at photos like these to remind herself of her goals in life.

Images as the Objects of Communication

The final way in which images were used was to facilitate communication and social interaction. This occurred in two ways—one where images did the work of communication, and the other where images facilitated social interaction.

Colin browsed through his friend’s photos of a concert he also attended to connect with that experience again: Fieldnotes for Colin: He was viewing other's people's Blur concert photos. He joyfully reminisced about being there as well, and the images helped him connect back with that experience.

Images to Do the Work of Communication

Images can sometimes be more effective than words in communicating. This example of a conversation between

108

Full Papers

Proceedings: NordiCHI 2010, October 16–20, 2010

Florence and her flatmate involved learning that Van Gogh resembled someone they both knew:

to speculate on why this might be. The fact that image search tools are designed to direct users to unfamiliar sites via keyword queries may go some way to explaining why they were used so infrequently during our study. Most of the image-based activities we logged tended to involve routine and familiar activities (61% overall). When seeking out particular images, users may be more adept at finding them quickly through familiar sites. In addition, given that images were often used in the course of other activities, rather than as end goals in themselves, suggests that a model of “search” is simply not a good fit in most cases where images are used on the Web. This leads on to a second main finding, which is that domain-specific Websites seemed to be the most important way that the images were engaged with (42% overall). Here it was clear that what was important was the whole array of features that these Websites deliver, often specialized for the kinds of activities users want to carry out where images happen to play a role. The use of retail sites, property sites, social networking and map sites are all examples of where images most effectively accessed in the context of other kinds of activities.

Fieldnotes for Florence: Florence’s flatmate found out that Florence didn’t know who Van Gogh was, and said that Van Gogh bore a striking resemblance to someone they both knew and asked her to do a search for the artist. Florence did a quick scan at the Google Image search results and she pointed to a Van Gogh self-portrait and asked her flatmate, "is it that one?" and the flatmate said yes and they both laughed at it because it did remind her of someone that they both knew.

Here, the fact that no explanation was needed of who the image resembled points to the effectiveness of images for some kinds of communication. Of course, the goal was purely to amuse, but the example shows how images can sometimes speak differently than words. The example of William’s search for pigs (previously discussed) also illustrates this pattern, as his intent was to email the pig photos to his colleague to humor her. Fieldnotes for William: He found two images which he planned to send over the next few days... Another picture [he found] was one of a pig bikini - he knew this would make his friend laugh. It was very amusing because it fit "her sense of humor".

This, in turn, is why understanding specific needs, tasks, or motivations is so important if we want to understand image use, leading us to discover a whole diversity of reasons why people used Web-based images. Some of these did involve finding answers to specific questions via unfamiliar sites, but most did not. Rather than question-answering, we found it more useful to think about how images were used to build on ongoing learning or interests, how they could spark discovery or inspire, how they could connect people to remote experiences, or could be used for quick navigation or communication. While prior work has aimed to create better tools for searching [13] and browsing [5, 19, 29], these types of interactions reported in this study, we would argue, point to more diverse kinds of tools, and highlight other kinds of important criteria for assessing these tools beyond keyword relevance or ranking accuracy. We now briefly sketch out some other directions we might explore on the basis of our framework.

In these last examples, the funny images sent were intended as messages in and of themselves, because of their shared meaning between two friends. Images for Social Interaction

As a final sub-theme, images were seen to be used by participants in social games or social features online. Besides common examples like Facebook comments and wall posts, some scenarios showed social interactions taking place in online forums, such as this guessing game that involved locating interesting images of personalities and sharing them to encourage a response: Fieldnotes for Tom: The object of the game, played twice weekly, is to identify an obscure person based only on a photo. The person with the correct guess then uploads the next picture. While Tom was on the site, he clicked on other thumbnails of obscure people that looked interesting. There was one of Mae Busch, which looked interesting because you could barely recognize the person. In fact, he enlarged the photo so he could identify the person better.

Designing for Learning and Research

There were various ways in which we found that images were used to support learning and research. The main way was via familiar sites that were routinely visited, and often the goal was to see whether there was anything new to look at. Important here were not just new images, but the surrounding content, helping participants to keep abreast of any news, to be entertained, and to add to their already considerable knowledge around a topic. Such behavior suggests not necessarily the need for better keyword search, but the support of quick navigation to familiar sites, perhaps triggered by new content appearing on those sites. The analogy is almost like channel-surfing on TV, where interesting content can grab attention. It suggests that Web-based tools might be better at alerting users to changes in content (whether image-based or not) on sites that are frequently visited.

The social interaction here was indirect—the participant was entertained by other members’ interactions around images, with no direct communication between them. This shows how images can spark interest in unique ways beyond starting points for conversation. DISCUSSION

This study has shown that the term “image search” may hide the rich and interesting complexity of reasons why users engage with images on the Web, and how they go about these activities. The results of this study derive a number of interesting findings. First amongst these was the fact that image-specific search tools (e.g. Google Images) were not the main way that images were accessed and used (less than 10% of the time). While it is difficult to make too much of this on the basis of a small sample, it is interesting

109

Full Papers

Proceedings: NordiCHI 2010, October 16–20, 2010

The desire to amass new knowledge on particular topics (such as new artists, or new work by a particular model) also suggests that more flexible tools for clipping and saving the results of Web browsing would be valuable. Tools that act as digital scrapbooks or repositories for information around a theme or topic could be effective ways of not just visiting sites, but collecting information from multiple sites, an issue which has been commented on before when looking at the predominance of the use of the Web for ongoing research activities [23]. A good example here would be the use of such “scrapbook” tools for putting together sources of ideas and inspiration, much like designers do when they keep notebooks.

Users also analyzed images in these scenarios from spatial and historical perspectives, thus we can speculate that the ability to position images in those dimensions can improve this experience. For example, geotagging images, allowing for panoramic views, rendering images in virtual 3D and being able to place images within a wider geographic context may be important. Likewise, re-connecting to the past might benefit from linking images to a timeline, for example, personal history. In both cases the ability to juxtapose multiple high fidelity views or perspectives is desirable, as participants often viewed all photos within a given set to get a better sense of the observed space.

Finally, learning and research sometimes did involve keyword search and the emergence of discovery through scanning large collection of images. Here, playing with different keywords, and analyzing images visually supported learning. Even though not a frequent activity, this suggests that tools that flexibly sort along different visual and semantic dimensions might be both fun and valuable to inspire and create new insights.

Images as understanding people or personalities both require related contextual information, such as timelines or links to more information. Activities involving image access to re-live past experiences would benefit from a feature that allows users to keep track of meaningful images for future reference (e.g. in Elle’s hotel scenario). Unlike traditional bookmarks, which facilitate information retrieval, here bookmarks are more akin to a digital keepsake box that allows the images to be re-experienced.

Designing for Image Access as Secondary Goals

Designing for Communication

When designing to support image access as a secondary goal, efficiency, accuracy and consistency are critical dimensions to consider. For example, one interesting finding was the way in which images could serve as an additional way of finding answers during keyword search. This suggests that, rather than having to always choose between a text search (with a ranked lists of text) or an image search (with a matrix of images), a mix of text and images in the returns might offer even more efficient ways of finding information or navigating to certain sites.

Images used in social interaction often occur over social platforms such as blogs, online forums, instant messaging, and social networking Websites. While some features exist (e.g. commenting, tagging) these could be extended, such as allowing users to be more creative with images in the course of interaction. For example, editing tools to modify images, create cartoons and storyboards, or simply to personalize an image for someone else are suggested. Further, general image search tools might more seamlessly connect with communication tools and social networking sites, recognizing that these are common follow-on activities.

Additionally, it is important to consider whether the image is part of the end goal. For instance, in the case of Gracia’s recipe search, the photo of the dish was a simply a step along the way to obtaining the real goal –the recipe. In other cases, images might be used more systematically as navigational indexes, such as the case of William locating a product by visually browsing thumbnails of DVD album covers on Play.com.

CONCLUSION & FUTURE WORK

Research going forward will include looking at the relative frequency of these new forms of image use, the context in which they are preformed, and the implications for search. However, this study has extended our understanding of the contextual and behavioral details of users’ interactions with Web-based images as they occur in the course of everyday life for a small but diverse sample of users. In doing so, we have shown how both the “how” and the “why” of these activities rarely conform to the model of user interaction which appears to inform most image search tools. Users do sometimes want to issue queries in terms of keywords and receive answers in terms of images, but of interest is both why this would be so, and the myriad of other ways in which images play into people’s experience of the Web. We have found that images are often intertwined with the ongoing process of learning about the world, that they provide a compelling source of entertainment and inspiration, that they help create connections to other people and remote places, that they help us reminisce about our personal past, and that they provide a way of navigating both the Web and the physical world.

Both cases suggest that rather than treating images as end goals, such as is implied in search tools, they might also be treated as steps toward better, more flexible forms of navigation. Not only could images be more integrated with text in the presentation of search results, but clicking on images might offer up new navigational pathways that help users to explore sites or products that they are connected to Designing for Recreating or Connecting to Remote Experiences

In this third category of image use, participants often remarked on how they valued the detail in photos they observed during tasks to recreate or connect to remote experiences. Thus, higher fidelity photos can be used to help users in the task of analyzing visual detail and contextual relevance; aiding recall, recognition and stimulating emotions.

110

Full Papers

Proceedings: NordiCHI 2010, October 16–20, 2010

12.Kirk, D., Sellen, A., Rother, C., & Wood, K. Understanding photowork. In Proc. CHI 2006, ACM P. (2006), 761-770.

Our hope is that a small but in-depth study of this sort can serve to complement and deepen our understanding of larger scale, quantitative clickstream and query data analyses in bringing to bear an alternative set of findings more firmly from the user’s perspective. Integrating these approaches, we believe, would be a valuable next step. Meanwhile, by beginning to look at the rich array of human values expressed within these findings, we hope to have shown that we can open up new opportunities for design to encourage reflection, connection, socialization and play with images.

13.La Cascia, M., Sethi, S., & Sclaroff, S. (1998). Combining textual and visual cues for content-based image retrieval on the world wide Web. Tech. rep., Boston, MA, USA. 14.Lerman, K., & Jones, L. Social browsing on Flickr. (2006) URL http://arxiv.org/abs/cs.HC/0612047 15.Marchionini, G. Information Seeking in Electronic Environnments. (1995.) Cambridge, Cambridge UP. 16.Markkula, M. & Sormunen, E. (2000). End-User Searching challenges indexing practices in the digital newspaper photo archive,. Information retrieval, 1, 4, 259-285.

ACKNOWLEDGMENTS

We would like to thank Xiang Cao, Darren Smith and Richard Harper for their comments throughout this project, and the Socio-Digital Systems group at Microsoft Research Cambridge for funding this research. We also would like to thank George Buchanan and Xia Lin for comments on drafts.

17.Miller, A.D. & Edwards, W.K. Give and take: A study of consumer photo-sharing culture and practice. In Proc. CHI 2007, ACM P. (2007), 347-356. 18.Negoescu, R. A., & Perez, D. G. Analyzing Flickr groups. In Proc. CIVR 2008, ACM P. (2008), 417-426. 19.Porta, M. Browsing large collections of images through unconventional visualization techniques. In Proc. AVI 2006, ACM P. (2006), 440-444.

REFERENCES 1.Paul André, Edward Cutrell, Desney S. Tan, Greg Smith, Designing Novel Image Search Interfaces by Understanding Unique Characteristics and Usage. In Proc. INTERACT 2009, ACM P. (2009), 340-353.

20.Pu, H. T. An analysis of failed queries for Web image retrieval. Journal of Info. Science 34, 3 (2008), 275-289. 21.Rieman, J. The diary study: a workplace-oriented research tool to guide laboratory efforts. In Proc. INTERCHI 1993, ACM P. (1993), 321-326.

2.Armitage, L. H, & Enser, P. G. B. Analysis of user needs in image archives. Journal of Info. Science 23, 4 (1997), 287-299. 3.Brewer, J., Kaye, J., Williams, A., & Wyche, S. (2006). Sexual interactions: why we should talk about sex in HCI. In CHI '06: CHI '06 extended abstracts on Human factors in computing systems, (pp. 1695-1698). New York, NY, USA: ACM.

22.Rodden, K., & Wood, K. R. How do people manage their digital photographs? In Proc. CHI 2003, ACM P. (2003), 409416. 23.Sellen, A.J., Murphy, R. M., & Shaw, K. (2002). How knowledge workers use the Web. Proceedings of CHI 2002, Minneapolis, MN. New York: ACM Press, pp. 227-234.

4.Burke, M., Marlow, C., & Lento, T. (2009). Feed me: motivating newcomer contribution in social network sites. In CHI '09: Proceedings of the 27th international conference on Human factors in computing systems, (pp. 945-954). New York, NY, USA: ACM.

24.Spink, A. (2004). Web searching for sexual information: an exploratory study. Information Processing & Management, 40 (1), 113-123.

5.Cai, D., He, X., Li, Z., Ma, W. Y., & Wen, J. R. (2004). Hierarchical clustering of www image search results using visual, textual and link information. In Proc. MULTIMEDIA '04, (pp. 952-959). New York, NY, USA: ACM.

25.Stevens, M. M., Abowd, G. D., Truong, K. N., & Vollmer, F. (2003). Getting into the living memory box: Family archives & holistic design. Personal and Ubiquitous Computing, 7 (3), 210216.

6.Choi, Y., & Rasmussen, E. M. Searching for images: the analysis of users' queries for image retrieval in American history. Journal of the American Society for Info. Science and Technology 54, 6 (2003), 498-511.

26.Süsstrunk, S., & Winkler, S. (2004). Color image quality on the internet. In Proc SPIE, vol. 5304, (pp. 118-131). 27.Toyama, K., Logan, R., & Roseway, A. (2003). Geographic location tags on digital images. In MULTIMEDIA '03: Proceedings of the eleventh ACM international conference on Multimedia, (pp. 156-166). New York, NY, USA: ACM.

7.Cunningham, S. J., & Masoodian, M. Looking for a picture: an analysis of everyday image information searching. In Proc. JCDL 2006, ACM P. (2006), 198-9. 8.Datta, R., Li, J., & Wang, J. Z. Content-based image retrieval: approaches and trends of the new age. In Proc. MIR 2005, ACM P. (2005), 253-262.

28.Van House, N., Davis, M., Ames, M., Finn, M., & Viswanathan, V. (2005). The uses of personal networked digital imaging: an empirical study of cameraphone photos and sharing. Ext. Abstracts CHI 2005, ACM P. (2005), 1853-1856.

9. Dumais, March, Schraefeumais, S. T. 2009. An interdisciplinary perspective on information retrieval. In Proc. of SIGIR '09. ACM, New York, NY, 1-2.

29.Wang, C., Zhang, L., & Zhang, H. J. (2008). Learning to reduce the semantic gap in Web image retrieval and annotation. In Proc SIGIR '08, (pp. 355-362). New York, NY, USA: ACM.

10.Goodrum, A., & Spink, A. Image searching on the excite Web search engine. Info. Processing and Management 37, 2 (2001), 295-311.

30.Westman, S. & Oittinen, P. (2006). Image retrieval by endusers and intermediaries in a journalistic work context. Information Interaction in Context, IIiX, 102-110.

11.Graham, A., Molina, H. G., Paepcke, A., & Winograd, T. Time as essence for photo browsing through personal digital libraries. In Proc. JCDL 2002, ACM P. (2002), 326-335.

111

Full Papers

Proceedings: NordiCHI 2010, October 16–20, 2010

HCI & Sustainable Food Culture: A Design Framework for Engagement Jaz Hee-jeong Choi Queensland University of Technology Brisbane QLD 4059, Australia [email protected]

Eli Blevis Indiana University Bloomington, IN, USA [email protected]

ABSTRACT

sustenance [17]. Humans simply cannot exist without food intake. Essentially, sustainability in this sense is based on ‘securing’ food: having access to stable availability and use of quality food [18]. However, the current food production and consumption practices do not ensure food security for the future but rather seriously threaten it [17]. One of the identified causes for this situation is the widening divide between the urban and rural environments. While people living in urban environments – including urban, suburban, and peri-urban (all of which are generally referred to as urban/metropolitan/city in this paper) regions – increasingly see food as objects of consumption or the ‘final products’ to be consumed. This situation deteriorates with the unprecedented scale of urban growth in recent years, broadening the segregation between the rural and urban environments, thereby further limiting knowledge about and access to fresh produce. Global urban population has been increasing rapidly, which suggests that availability of food resources in urban areas is also increasingly becoming scarce. The UN Population Fund [37] predicts that the urban population will grow further to reach 60% of the entire global population by 2030. Therefore, as seen in recent global economic and ecological turmoils, change is necessary in how we live our daily lives, in order to create a positive outlook for global food security. Now is a crucial time for HCI researchers to create actionable knowledge through identifying, testing, and building on technical opportunities that can be augmented and realised to cultivate urban food cultures that are environmentally, socially, and health-wise sustainable.

The current food practices around the world raises concerns for food insecurity in the future. Urban / suburban / and peri-urban environments are particularly problematic in their segregation from rural areas where the natural food sources are grown and harvested. Soaring urban population growth only deteriorates the lack of understanding in and access to fresh produce for the people who live, work, and play in the city. This paper explores the role of HumanComputer Interaction (HCI) design in encouraging individual users to participate in creating sustainable food cultures in urban environments. The paper takes a disciplinary perspective of urban informatics and presents five core constituents of the HCI design framework to encourage sustainable food culture in the city via ubiquitous technologies: the perspective of transdisciplinarity; the domains of interest of people, place, and technology; and the perspective of design. Author Keywords

Food, Sustainability, HCI, Urban Informatics, Design, Transdisciplinarity ACM Classification Keywords

H5.0. Information interfaces and presentation (e.g., HCI): General. K.4.2 Social Issues. INTRODUCTION

Assuring a sustainable future has become one of the main concerns for HCI researchers in recent years [14]. While sustainability as a term warrants multi-fold interpretations, recent years have seen a particular emphasis on environmental sustainability as the ‘climate change’ became part of the global political and social consciousness. One imminent consequences of the global warming is related to food, the vital foundation of human

In considering ‘how’ to design such HCI experience, we argue that the answer is ‘engagement’: engagement across disciplines, engagement with and amongst users (and nonusers), and engagement for sustainability, or rather, ‘sustained usability.’ In this paper, we will discuss each of these three main aspects of engagement: • Engagement across disciplines: The perspective of transdisciplinarity – transcending disciplinarity and values orientation

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. NordiCHI 2010, October 16–20, 2010, Reykjavik, Iceland. Copyright 2010 ACM ISBN: 978-1-60558-934-3...$5.00.

• Engagement with and amongst users/non-users: The domain of urban informatics at the intersection of people, place, and technology [19] – participatory, context-aware, and interactive networks

112

Full Papers

Proceedings: NordiCHI 2010, October 16–20, 2010

• Engagement for sustained usability: The perspective of design – design criticism and critical design

reductionism/holism, diversity/unity. This duality is transgressed by the open unity that encompasses both the universe and the human being [29].

More specifically, two perspectives and three domains of interest are introduced, which together constitute the main five sections of this paper: perspective of transdisciplinarity, domain of interest of people, domain of interest of places, domain of interest of technology, and perspective of design. While these five facets form a frame that is general and applicable to any inquiry about or practice within a values-oriented design space, we intend to use it specifically to serve as a theoretical underpinning for our exploration of HCI design for sustainable food culture. This frame is not intended to be complete. We hope to expand and refine this frame as a theoretical foundation and also apply it to our study of sustainable food culture (http://www.urbaninformatics.net/projects/food) as a matter of theoretically informed practice. Similarly, our practice will further inform this theoretical frame, making the development an iterative and incremental process. The goal of this paper is to initiate this process by defining the theoretical frame in the context of sustainable food cultures as a basis to apply to future study.

Transdisciplinarity differs from similar integrative research approaches such as multi- and interdisciplinarity. While multidisciplinary research refers to a study in which several partners representing various disciplinary perspectives share their respective methods and domains as outputs and inputs one-with-another, interdisciplinary research refers to a study in which several partners representing various disciplinary perspectives pool their methods and domain expertise. Both approaches ‘overflow’ the discipline and remain within the framework of disciplinary research; transdisciplinary research is ‘at once between the disciplines, across the different disciplines, and beyond all discipline’ [29]. Therefore, transdisciplinarity is an orthogonal concept to interdisciplinarity and multidisciplinarity, and helps transcend disciplinarity by focusing on goals that emerge from a focus on values, issues, as well as ontological and epistemological perspectives, appealing to disciplinary notions of method and domains of expertise as needed and in the service of these larger goals. Methodologically, transdisciplinarity allows the researcher to flexibly yet rigorously explore and integrate research methods from various disciplines to result in coherent knowledge rather than united knowledge across disciplines [33]. Emerging research fields such as urban informatics [19] reflect a need to create such holistic body of knowledge in order to understand and tackle challenging global issues such as the climate change, governance, and education. Given the diversity of role that food plays in everyday life and its impact that is also diverse in type and scale, designing HCI for urban food sustainability thus must seek ways to utilise ubiquitous technology’s flexibility in scale of application (for example, on the continuum of individual / collective, private / public, and local / global) to improve the health, social, and environmental bottom-lines of everyday human-food interaction at the intersection of people, place, and technology. Transdisciplinarity for food sustainability research is an essential perspective as it provides dialogic knowledge development that can tackle real life problems that are inherently multi-faceted.

TRANSDISCIPLINARITY: TRANSCENDING DISCIPLINARITY & VALUES ORIENTATION

As mentioned earlier, sustainable HCI has been growing as a unique—nevertheless broad—domain of study in recent years. However, despite the complexity of what constitutes the notion of sustainability, the majority of studies remain within specific disciplines – for example, environment [2, 11], health [10, 30, 31], and social [28]. As DiSalvo et al. [14] argue, there is an opportunity and necessity to add to these efforts through HCI design by recognising and embracing the innate interconnecting threads across various scholarly disciplines, domains of research and practices. Recently emerging movements such as Take a Bite Out of Climate Change (www.takeabite.cc) highlight the confluence of domains that are germane to sustainability, thereby reasserting the need for creating a broader, holistic understanding of current issues in food culture. From the scholarly perspective, engage the issue of food sustainability through HCI design therefore must extend beyond disciplinary compartmentalisation or didactic information dissemination.

THE DOMAIN OF INTEREST OF PEOPLE: PARTICIPATORY CULTURE

Since early 1970s when Jean Piaget coined the term, transdisciplinarity has evolved into a field or more pragmatically, a conceptual tool that help researchers create ‘innovative research strategies and methods tackling complex objects and contexts’ [25]. Transdisciplinarity is not a method, but a principle or approach that seeks to create new overarching knowledge through integrative research [25, 5]. Nicolescu maintains:

In many parts of the world, the means to engage in participatory culture are no longer limited to the technically versed or the civically inclined. Scholars such as Jenkins [21] and Hartley [20] have identified socio-technical trends towards participation in digital culture shaped by individual expressions of creativity. Participatory or DIY culture is fast becoming a core element of contemporary society (at least in developed nations) in which citizenship is construed through ‘practice’ of self identification by individuals rather than a ‘contract between state and subject’ [20]. As evident in many grassroots initiatives such as the Local Food

Transdisciplinarity transgresses the duality of opposing binary pairs: subject/object, subjectivity/objectivity, matter/consciousness, nature/divine, simplicity/complexity,

113

Full Papers

Proceedings: NordiCHI 2010, October 16–20, 2010

Movement—a collaborative effort to build more locally based, self-reliant food economies [15] and Slow Food International—a non-profit group focusing on preservation of the cultural, culinary, and artistic local traditions [22], significant transformations arise from large-scale consensual participation of individuals identifying with the value of a sustainable lifestyle both conceptually and pragmatically. For such transformations to have sustainable impact, they need to be incremental and self-initiated as opposed to abrupt and externally enforced [8, 7].

too-familiar places—via mass media such as print and TV, or micro-communications such as sensor networks—does not necessarily trigger sufficient motivation for behavioural change towards an ongoing health- and earth-friendly lifestyle. Instead, it is necessary to develop a better understanding about how to go beyond just informing and into motivating and encouraging positive changes in action and perception. As such, innovations in understanding of what makes technology persuasive and motivational [13, 23, 16, 1] for each user at any one given moment are likely to bring about sustained behavioural changes amongst individuals, and further increase the ideological impetus to enhance societal values. However, maintaining a food practice that upholds health, environmental, and social ideals is a time- and energy-demanding task. It currently requires an advanced understanding of a variety of sometimes conflicting information sources and a determination to commit the necessary time and effort to always know where to buy health- and earth-friendly produce, according to seasonality as well as to one’s differing tastes and lifestyle needs. While existing media channels such as television shows, magazines, and blogs on the topic of eating, cooking, and growing food may offer educational information—including nutritional, cultural, aesthetic—that could assist in this regard [24, 12], we can find additional and perhaps more effective benefit in ubiquitous technology’s capacity to streamline the vigorous management process particularly through real-time context awareness.

Therefore, usable and efficient human-computer interaction (HCI) design to bring about changes needs to be innately supportive and persuasive in guiding user’s actions rather than exerting control and leverage. Such design must be scalable in context awareness (from micro to macro – from the individual to community context, for example) and adaptive to provide the optimal solution according to the given context. Many examples of how participatory culture is enabled by recent technological innovation rely on socalled Web 2.0 applications such as Wikipedia, YouTube, Flickr, and social networking sites, which are arguably more customisable and open (though in many cases they impose regulatory boundaries but nevertheless allow some ways for interaction and modification though external means). In this regard, Burgess [9] has introduced a useful notion of vernacular creativity, which refers to ‘a productive articulation of consumer practices and knowledges (of, say, television genre codes) with older popular traditions and communicative practices (storytelling, family photography, scrapbooking, collecting)’ reducing the cultural distance between production and everyday experiences. Can the features that give rise to participatory culture on the web and significant changes in economy, politics, and socio-cultural domains— as seen in crowd sourcing phenomenon as notably explained by Shirky [34]—be harnessed to encourage people to further their sustainability awareness and actions in the context of food practices? The question for HCI designers is to find efficient and engaging ways to utilise technical resources to allow for collaborative information sharing, knowledge production, and user-led innovation. Some current popular services such as Urbanspoon (www.urbanspoon.com), Beanhunter (www.beanhunter. com), and Foursquare (foursquare.com) function as repositories of crowd-sourced recommendations for places to eat, coffee shops, and general places of interest by urban location. The users transform the virtual ‘space’ into ‘places’ by embedding values and meanings through social interaction [36].

The contexts of food culture vary widely from one place to another. Just as each continent and country may have its own values and practices associated with food, each place however small it may be, has its own atmosphere and rules of engagement for each entity in order for it to be counted as a valid constituent of the place. A brief observation around eaters in streets of any metropolis such as London, Tokyo, and New York would effortlessly show how, for example, an Italian pizzeria differs from a Korean barbeque restaurant that may be located nearby. As Steel asserts [35], ‘[f]ood shapes cities, and through them, it moulds us’ and the city is thus ‘partly shaped by food.’ Therefore, the opportunities that network technology imparts for cultivating sustainable urban food culture is essentially twofold: first, assistance in understanding and navigating through the food-layer of the given place; and second, encouraging sustainable food practices, which in turn shapes the city. Building a food culture toward a more sustainable future is thus an iterative and evolutionary process involving interactions amongst people, place, and technology.

THE DOMAIN OF INTEREST OF PLACES: CONTEXT AWARENESS & CONTEXT SPECIALISATION

An example of such system is Bonanni et al.’s [6] SourceMap (http://www.sourcemap.org). Both as an issue of food safety and minimizing food miles as a matter of sustainability, SourceMap is an example of a system that promotes geospatial context awareness in the domain of food. The system is targeted at providing small business

Prior to the rise of network technologies, likely places for placing advertisements for healthy lifestyle were television, billboards, and on the back of the cereal box. However, as evident in food culture today, simply providing people with environmental data and educational information in now all-

114

Full Papers

Proceedings: NordiCHI 2010, October 16–20, 2010

owners with the means to calculate the distances that the supplies they use need to travel and the means to make this information available to their clientele as a matter of demonstrating conscientious, socially responsible business practice. The system produces ‘receipts’ that denote the carbon footprint of a small business’s products, showing total carbon footprint and carbon footprints of various stages of transport of various constituent product elements.

There are a number of possible interactive systems that may play a positive role here. For example: (i) Food production and source tracking: As an issue of both sustainable supply and food safety, interactive systems can allow various groups of people to adapt to changing suitability of particular regions for growing particular crops and other forms of food production. (ii) Dashboard earth: As an issue of preparation and adaptation to changing climate, interactive systems can help ensure planning and preparation for how people in urban environments can respond to changes in food and water supplies and even to threats to the inhabitability of— particularly—coastal urban and other environments.

THE DOMAIN OF INTEREST OF TECHNOLOGY: NETWORKS & INTERACTIVITY

With respect to the domain of interest of technology, there are many forces in place, which create likelihood for changes with implications for food cultures that are yet to be understood. For each of these changes, there are opportunities for technologies—particularly in terms of networks and interactivity—to play a role with both positive and negative implications.

(iii) Orderly absorption: Urban environments are in some sense dynamically changing places—interactive systems can offer infrastructural support that allows urban policy makers to provide for the orderly immigration to or emigration from urban environments in the face of the effects of climate change.

The Intergovernmental Panel on Climate Change (IPCC) most recent complete report in 2007 predicts that climate change will have massive implications for global food production and conditions of production, as well as for water, coastal inhabitations, health, and ecosystems [32 as reported in 4]. For food production, ‘complex, localised negative impacts on small holders, subsistence farmers, and fishers’ is virtually certain, and the latitudes at which cereal production is viable will change depending on the amount of warming. Another salient effect is the likelihood of ‘increased damage from floods and storms’ and the possibility of coastal flooding and wetlands loss, depending on the amount of warming. These two factors and others in the IPCC report have implications for the many urban areas that are located in coastal environments, not just in terms of food, but also potentially in terms of habitability.

(iv) Living with fewer resources: Interactive systems can assist people in urban and other environments with understanding and practicing how to live with fewer resources, either as a matter of sustainable practices or as a matter of adapting to climate change or both. (v) Saving life: Social mechanisms—especially those that rely on interactive technologies—can play a role in fostering relationships between people at various levels of organization in order to ensure that as many people as possible have access to safe environments, with food and drinking water and that people are actively engaged in helping others. THE PERSPECTIVE OF DESIGN: DESIGN CRITICISM & CRITICAL DESIGN

Some respected scientists—such as James Lovelock—have made claims that the IPCC predictions are too conservative [27 as reported in 4]. Lovelock believes that we have already reached the point at which limiting anthropogenic production of green house gasses is no longer a viable mechanism for preventing climate change; and further, that positive feedback mechanisms are already in place which make dangerous global warming a certainty. Lovelock presents a catastrophic picture that the earth’s population— now approaching 7 billion—will be reduced to under 1 billion, before the end of the century.

This paper has outlined critical concepts and examples that are germane to conceptualizing HCI design for disciplinary and user engagement in sustainable food cultures in urban environments. The paper has also outlined the three core domains of interest for user-engagement: of people, place, and technology.

A changing climate and its associated implications per the two paragraphs above implies a need for the design of digital networking and interactive technologies that can help people at various levels—as individuals, small groups, governments, and global bodies—plan for the orderly adaptation to these effects. Such networks will need to make available data, visualizations, tracking, and predictive simulations about changing locations of food production and threats to particular urban environments.

Figure 1. Core domains of interest for user engagement

115

Full Papers

Proceedings: NordiCHI 2010, October 16–20, 2010

REFERENCES

Perceptively, technologies provide useful ways to help cultivate sustainable food culture from the individual to the global level. However, we must at the same time remember that network technologies are part of the broader contemporary influences shaping the ecology. Environmental impact is one of the main such influences. Recently Alexander Wissner-Gross at the Harvard University Center for the Environment has made a statement that has provoked much discussion amongst general public as well as in academic and commercial sectors: that ‘performing two Google searches from a desktop computer can generate about the same amount of carbon dioxide as boiling a kettle" or about 7g of CO2 per search’ [26]. This ‘statistic’ may in fact be controversial at best – nonetheless, the point that not even Google searches are without environmental impact holds.

1. Aleahmad, T., Balakrishnan, A.D., Wong, J., Fussell, S.R. & Kiesler, S., Year. Fishing for Sustainability: The Effects of Indirect and Direct Persuasioned. CHI conference, Florence, Italy. 2. Bang, M., Torstensson, C. & Katzeff, C., 2006. The PowerHhouse: A Persuasive Computer Game Designed to Raise Awareness of Domestic Energy Consumption. Persuasive Technology. 123-132. 3. Blevis, E., 2007. Sustainable Interaction Design: Invention & Disposal, Renewal & Reuse. Proceedings of the SIGCHI conference on Human factors in computing systems. San Jose, California, USA: ACM. 4. Blevis, E. & Blevis, S., 2010. Hope for the Best and Prepare for the Worst: Interaction Design and the Tipping Point. Interactions, 17.

For the ‘sustainability of’ a HCI design for sustainable food culture, the design needs to ensure participation of users in perpetual recreation of the technology and must also incorporate anticipation for the technology’s socio-cultural, health, and environmental impact. To achieve this, a continual examination of the technology’s effective is an imperative part of the overall development of the technology. Design theory is a rich and varied art with an extensive literature too broad to list here. Within design theory, one idea in particular that has elsewhere been applied to the notion of how interaction design is implicated in sustainability is the notion of design criticism and critical design as complimentary activities that are necessary companions in design processes [3]:

5. Blevis, E. & Stolterman, E., Year. The Confluence of Interaction Design & Design: from Disciplinary to Transdisciplinary Perspectivesed. Design Research Society Biennial Conference, Sheffield, UK. 6. Bonanni, L., Hockenberry, M., Zwarg, D., Csikszentmihalyi, C. & Ishii, H., 2010. Small Business Applications of Sourcemap: A Web Tool for Sustainable Design and Supply Chain Transparency. Proceedings of the 28th international conference on Human factors in computing systems. Atlanta, Georgia, USA: ACM. 7. Brinol, P., Rucker, D.D., Tormala, Z.L. & Petty, R.E., 2004. Individual Differences in Resistance to Persuasion: The Role of Beliefs and Meta-Beliefs. In E.S. Knowles & J.A. Linn (eds.) Resistance and Persuasion. Mahwah, N.J.: Lawrence Erlbaum Associates, 83-105.

We may distinguish here between design criticism – what is needed to understand and interpret present ways of being, and critical design – what is needed to ensure that our actions lead to sustainable future ways of being.

8. Brock, T.C., 1967. Communication discrepancy and intent to persuade as determinants of counterargument production. Journal of Experimental Social Psychology, 3.

Time being what it is, critical design takes place in the absence of complete understandings of present ways of being. Thus, design criticism and critical design are mutually dependent, ongoing, and co-evolving acts. Design without design criticism is unlikely to create critical design and criticism without critical design is unlikely to create design criticism. Design criticism is strategic. Critical design is tactical.

9. Burgess, J., 2006. Hearing Ordinary Voices: Cultural Studies, Vernacular Creativity and Digital Storytelling. Continuum, 20, 201 - 214. 10. Consolvo, S., Everitt, K., Smith, I. & Landay, J.A., 2006. Design requirements for technologies that encourage physical activity. Proceedings of the SIGCHI conference on Human Factors in computing systems. Montreal, Qubec, Canada: ACM.

In applying our framework to our future work on our project (http://www.urbaninformatics.net/projects/food), we expect to take this bi-valent approach—that is we expect to equally (i) look for and apply design criticism to various forms of the design of ubiquitous technologies for sustainable food culture that we may uncover by observation or from secondary sources, and (ii) suggest the design of new forms of interactivity that inspire sustainable food practices—while still preserving the joy of urban experiences.

11. Cornelissen, G., Pandelaere, M. & Warlop, L., 2006. Cueing Common Ecological Behaviors to Increase Environmental Attitudes. Persuasive Technology. 39-44. 12. De Solier, I., 2005. TV Dinners: Culinary Television, Education and Distinction. Continuum: Journal of Media & Cultural Studies, 19, 465 - 481. 13. De Young, R., 2000. Expanding and Evaluating Motives for Environmentally Responsible Behavior. Journal of Social Issues, 56, 509-526.

116

Full Papers

Proceedings: NordiCHI 2010, October 16–20, 2010

14. Disalvo, C., Sengers, P. & Brynjarsdottir, H., 2010. Mapping the landscape of sustainable HCI. Proceedings of the 28th international conference on Human factors in computing systems. Atlanta, Georgia, USA: ACM.

26. Leake, J. & Woods, R., 2009. Revealed: the environmental impact of Google searches. Times Online. 27. Lovelock, J., 2009. The vanishing face of gaia : a final warning New York: Basic Books.

15. Feenstra, G., 2002. Creating space for sustainable food systems: Lessons from the field. Agriculture and Human Values, 19, 99-106.

28. Marc, A.S., 2000. Some social implications of ubiquitous wireless networks. SIGMOBILE Mob. Comput. Commun. Rev., 4, 25-36.

16. Fogg, B.J., 2003. Persuasive Technology: Using Computers to Change What We Think and Do Amsterdam: Morgan Kaufmann Publishers.

29. Nicolescu, B., 2002. Manifesto of Transdisciplinarity New York: State University of New York Press.

17. Food and Agriculture Organizataion of the United Nations, 2008. Climate Change, Water and Food Security [online]. Food and Agriculture Organization of the United Nations. Available from: ftp://ftp.fao.org/docrep/fao/010/i0142e/i0142e07.pdf.

30. Papadaki, A. & Scott, J.A., 2006. Process Evaluation of an Innovative Healthy Eating Website Promoting the Mediterranean Diet. Health Education Research, 21, 206-18. 31. Parmar, V., Keyson, D. & Debont, C., 2008. Persuasive Technology for Shaping Social Beliefs of Rural Women in India: An Approach Based on the Theory of Planned Behaviour. Persuasive Technology. 104-115.

18. Food and Agriculture Organizataion of the United Nations, 2010. FAQ: What is Meant By Food Security [online]. Food and Agriculture Organizataion of the United Nations. Available from: http://www.fao.org/spfs/about-spfs/frequently-askedquestions-spfs/en/.

32. Parry, M.L., Intergovernmental Panel on Climate Change. & Intergovernmental Panel on Climate Change. Working Group Ii., 2007. Climate change 2007 : impacts, adaptation and vulnerability / Contribution of Working Group II to the Fourth Assessment Report of the IPCC Intergovernmental Panel on Climate Change [online]. IPCC Secretariat. Available from: http://www.ipcc.ch/publications_and_data/publications_ ipcc_fourth_assessment_report_wg2_report_impacts_ad aptation_and_vulnerability.htm

19. Foth, M., 2009. Handbook of Research on Urban Informatics: The Practice and Promise of the Real-time City Hershey, PA: Information Science Reference. 20. Hartley, J., 1999. Uses of television London ; New York: Routledge. 21. Jenkins, H., 2006. Fans, bloggers, and gamers : exploring participatory culture New York: New York University Press.

33. Ramadier, T., 2004. Transdisciplinarity and its challenges: the case of urban studies. Futures, 36, 423439.

22. Jones, P., Shears, P., Hillier, D., Comfort, D. & Lowell, J., 2003. Return to traditional values? A case study of Slow Food. British Food Journal, 105, 297-304.

34. Shirky, C., 2008. Here comes everybody : the power of organizing without organizations New York: Penguin Press.

23. Kaplan, S., 2000. Human Nature and Environmentally Responsible Behavior. Journal of Social Issues, 56, 491508.

35. Steel, C., 2008. Hungry City: How Food Shapes Our Lives London: Vintage.

24. Lappalainen, R., Kearney, J. & Gibney, M., 1998. A pan EU survey of consumer attitudes to food, nutrition and health: an overview. Food Quality and Preference, 9, 467-478.

36. Tuan, Y.-F., 2007. Space and Place: The Perspective of Experience Minneapolis: University of Minneapolis Press.

25. Lawrence, R.J. & Després, C., 2004. Futures of Transdisciplinarity. Futures, 36, 397-405.

37. Unfpa, 2007. State of World Population 2007: Unleashing the Potential of Urban Growth. New York: U.N.P. Fund.

117

Full Papers

Proceedings: NordiCHI 2010, October 16–20, 2010

Distributed Usability Evaluation: Enabling Large-scale Usability Evaluation with User-controlled Instrumentation Lars Christensen & Erik Frøkjær Department of Computing, University of Copenhagen Njalsgade 128, building 24, 5th floor, DK-2300 Copenhagen S, Denmark {larsc,erikf}@diku.dk ABSTRACT

evaluations, and the communication and prioritization necessary for transforming redesign proposals into reality are complicated [20,23].

We present DUE (Distributed Usability Evaluation), a technique for collecting and evaluating usability data. The DUE infrastructure involves a client-server network. A client-based tool resides on the workstation of each user, providing a screen video recording, microphone input of voice commentary, and a window for a severity rating. The idea is for the user to work naturalistically, clicking a button when a usability problem or point of uncertainty is encountered, to describe it verbally along with illustrating it on screen, and to rate its severity. These incidents are accumulated on a server, providing access to an evaluator (usability expert) and to product developers or managers who want to review the incidents and analyse them. DUE supports evaluation in the development stages from running prototypes and onwards. A case study of the use of DUE in a corporate environment is presented. The study indicates that the DUE technique is effective in terms of low bias, high efficiency, and clear communication of usability issues among users, evaluators and developers. Further, DUE is supporting long-term evaluations making possible empirical studies of learnability.

We have tried to ease some of these difficulties by designing a technique called Distributed Usability Evaluation (DUE) with the following objectives: • Support the detection of usability problems covering aspects of effectiveness, efficiency and satisfaction. • Minimize the evaluation workload for users, evaluators and developers. • Enable evaluation of short- and long-term usage. • Strengthen the quality and validity of identified usability problems, for example by enabling realistic user-context and large-scale evaluation with many users. • Strengthen the communication of usability problems between users, evaluators and developers. To realize the objectives it was clear that an instrumentation tool was needed. We wanted automated screen video capture of user interaction and recording of voice commentary, while the user was using the evaluated application. Like in beta testing, this enabled us to have the user in charge of problem detection, without quite losing the information-rich context known from think-aloud evaluations.

Author Keywords

Usability evaluation, instrumentation, automation, remote, distributed, screen video, voice commentary, case study, software industry, evaluator effect, think-aloud, beta test, learnability.

To investigate the DUE technique we carried out an extensive case study. Here DUE was used to evaluate the usability of production software under active development in a business environment with 16 users during one month.

ACM Classification Keywords

H.5.2 Information Interfaces and Presentation (e.g., HCI): User Interfaces–Evaluation/Methodology; D.2.2 Software Engineering: Design Tools and Techniques–User Interfaces.

The next section summarizes the key challenges that served as background for our design. Then we present DUE with its Instrumentation Framework and how it can be used by users, evaluators and developers. Next follows a section describing the case study and the strategy for data analysis. Finally, we summarize and discuss the main results.

INTRODUCTION

Usability evaluations in industry are limited by a number of challenges, for instance: evaluations are very labor demanding, it can be difficult to establish clear goals for

BACKGROUND

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. NordiCHI 2010, October 16–20, 2010, Reykjavik, Iceland. Copyright 2010 ACM ISBN: 978-1-60558-934-3...$5.00.

This section will summarize and briefly discuss some of the key challenges in usability evaluation, which we aim to address by DUE.

118

Full Papers

Proceedings: NordiCHI 2010, October 16–20, 2010

Measuring Usability

Emerging services, like for instance www.usertesting.com [26], offer internet-based remote think-aloud testing of websites. These services make it fast and easy to set up think-aloud tests and find participants, but many of the limitations and labour demanding tasks related to thinkaloud testing still remains, e.g. to extract, analyse and consolidate information from the videos.

A definition of usability, as the one given by ISO [15], gives us the initial step towards analyzing usability in applications. The next step, identifying and implementing reliable measures of usability, remains a challenge. Hornbæk [10] points out that usability cannot be measured directly and as a result of this, we find aspects of usability to measure and draw conclusions upon. Frøkjær et al. [4] make it clear that measures used to identify usability problems often do not take into account all three aspects of usability, i.e. effectiveness, efficiency and satisfaction; and a strong correlation between these aspects are often wrongly assumed. Identifying reliable measures remain an open-ended and commonly discussed challenge. Really useful measures cannot be identified at a general level, but have to be based on an understanding of the use situations within the particular application domain [4]. Users often have this understanding.

Thus, in spite of the assistance from various tools, usability evaluation in general is still a tedious task for evaluators, and often also for the developers. It involves extensive work to identify measures, set up evaluations with or without instrumentation, collect large amounts of data, analyse data, detect problems among the often vague measures, and communicate the problems [20,23]. The Evaluator Effect and Insufficient Analysis

The evaluator effect denotes that usability evaluators identify substantially different sets of usability problems when applying the same evaluation technique on the same application [17]. Facing large amounts of usability data, some of them vaguely documented, can make evaluator assessments complicated and unclear, thus increasing the evaluator effect.

Instrumentation and Automation

Instrumentation and automation within software evaluation have received some attention as a means to collect measures and supplement software evaluation, but have according to Ivory & Hearst [16] been underexplored.

Poor or insufficient analysis of usability data in practice, as observed by Nørgaard & Hornbæk [22], is partly a result of the complexity of the evaluator’s work when handling large amounts of usability data.

Early work by Hartson et al. [2,8,9] indicated that many challenges in usability evaluation could be met by using semi-instrumented remote evaluation and critical incident identification by users. However, Hartson et al. did not demonstrate the effectiveness of their approach in field studies.

These difficulties might be eased by tool support and stronger involvement of the users.

Recently instrumentation has received renewed focus as a means of collecting measures. Kim et al. [19] shows us that instrumentation can be a powerful tool to help detection of usability problems, especially when users are more involved than typically seen. User initiated events are seen as useful measures. Unfortunately the type of instrumentation used by Kim et al. requires program intrusion of the evaluated application, which limits practical usage in many situations.

Communication

The large amount of complicated data raises a huge communication challenge between people with quite different backgrounds: users with domain knowledge, developers with their technical insight, and evaluators with usability insight. Actually, this challenge is so large that it is often avoided in practice by more or less leaving out some of the stakeholders. However, studies indicate that involving developers, managers as well as users in the evaluation process can improve results [5,6,11,12].

Evaluation Tools and Workload

One possibility of strengthening the communication among stakeholders is to support the identification and recording of critical usability data in order to better understand and establish relationships among usability data [14].

Bateman et al. [1] recognize that setting up application intrusive instrumentation is a time-consuming task requiring technical insight by the evaluators. They argue that benefits can be achieved by being less intrusive; and propose a more intuitive way of adding instrumentation to an application.

Validity

The large workload on primarily the evaluators is probably the main reason for carrying out usability evaluations with few users over short periods of time. This makes it difficult to ensure external validity. The few users can make it doubtful that results are representative for all of the user population [3,25]. Further, a short evaluation period will only give snapshot insight in usability problems leaving out the users’ learning curve [7].

Other tools for usability evaluation, which offer some degree of instrumentation and are fully non-intrusive to the application, such as TechSmith’s Morae software, also support the task of usability evaluation. Here the evaluator is still left with the responsibility for problem detection and, as Howarth et al. [14] point out, usability evaluation is a tedious task even with this kind of software.

119

Full Papers

Proceedings: NordiCHI 2010, October 16–20, 2010

To strengthen the ecological validity, the circumstances of the usability evaluations should be as realistic as possible, while keeping attention to the risk of introducing disturbing experimental factors [24].

which have to be analysed thoroughly to identify usability issues, the focus in DUE is shifted to the user, who is given the responsibility to detect the usability issues. Thereby the primary instrumentation lays in the recorded video of the user’s screen, the recording of the user’s explanation, sound from surroundings, plus the collection of timestamps, and severity ratings of the user reported issues.

DISTRIBUTED USABILITY EVALUATION (DUE) The Instrumentation Framework

We have built an Instrumentation Framework, consisting of an Instrumentation Client and an Instrumentation Server, to realize the instrumentation and process automation necessary for meeting the DUE design objectives.

The user is instructed to have focus on elements which can have correlation to effectiveness, efficiency and satisfaction, but most importantly to use common sense and domain knowledge to report when something seems troublesome in actual use.

When doing a DUE evaluation the Instrumentation Client is installed on the computer of each participating user. The only requirements are that the computer must have a certain cpu power, a microphone, and a reasonable fast Internet connection. The Instrumentation Client manages user evaluation sessions. When an evaluation session ends, the Instrumentation Client transfers all collected data to the Instrumentation Server via the Internet.

In practice, when experiencing something as troublesome, the user can press a button to report and assess the severity of the issue (red, yellow or green) in the small Instrumentation Client visible on-screen, Figure 2 and 3.

All collected data are processed and stored on the Instrumentation Server for future use. The Instrumentation Server offers a web interface for evaluators and developers where they can log in and fulfil their role in the DUE workflow. See Figure 1 for the workflow of the four roles involved in DUE. These roles will be described in the following sections.

Figure 2. The small Instrumentation Client to the right of a web browser. Transparency enables placement in front of the application, if wanted.

Figure 1. Workflow in DUE. (1) The user starts with reporting issues, (2) the evaluator assesses these, (3) the development manager prioritizes, and (4) the developers respond. Finally the user receives feedback (1).

Figure 3. Close-up of the Instrumentation Client. The user’s severity rating of a usability issue is recorded by pressing one of the three colored buttons to the right.

After pressing the button, the user has to think out loud and explain the experienced issue vocally to the microphone, and maybe demonstrate further on screen by using the application if the issue was not evident from what was recorded before the button press.

The User Role

The instrumentation in DUE is centered on video recorded from the user’s screen and voice recorded from a microphone placed at the screen, while the user is using the application. The recording is continuously running while the Instrumentation Client is active, making it unnecessary to reproduce issues after discovery.

A timestamp at the user’s button activation is collected in the Instrumentation Client and is later used to create a bookmark in the recorded video of the user’s screen activity and voice commentary.

Instead of focusing on pre-selected measures and collecting large amounts of instrumented data from the application,

120

Full Papers

Proceedings: NordiCHI 2010, October 16–20, 2010

The Evaluator Role

on the user’s ability to report when they experience something as troublesome and primarily have to decide if the reported issue is a usability problem or not.

When data is received on the Instrumentation Server the evaluator can log into the Instrumentation Server web interface from anywhere through a web browser, and review new issues (see Figure 4).

The Development Manager and Developer Role

Development manager and developers run through a similar workflow as the evaluator, prioritizing and responding to the issues that are approved as new problems by the evaluator. Summary of Characteristics

the

Instrumentation

Framework

This type of user-controlled instrumentation has the advantage that developers do not have to build anything into their application, and companies need not to be concerned about how the instrumentation will affect their application. Figure 4. Two new issues ready for evaluation on the Instrumentation Server

The Instrumentation Client can be run automatically on the user’s computer at specified times, taking care of data collection, and letting the user work and use the application without disturbances. The user presses a single button when experiencing something as troublesome, shortly explains and demonstrates the experienced problem, and carries on work.

The Instrumentation Server’s web interface ensures that the evaluator can run through all new issues in an efficient workflow. For each issue the evaluator can jump to where the user experienced an issue—20 seconds before the user pressed the button—review what happened in the video of the user’s screen, hear the user’s explanation, and see the user’s severity rating (see Figure 5).

The Instrumentation Server automatically prepares data for evaluators and developers, making it possible for them to jump directly to the issues in the video, and concentrate on reviewing and assessing these. Further, the issues are conveniently stored for communication and future use in the development cycle. CASE STUDY

As this was the first practical use of DUE, an exploratory quasi-experimental case study was chosen as the research strategy [27]. Propositions

Based on the design objectives we identified a series of propositions to be investigated in the case study: A1: DUE supports catching usability issues (effectiveness, efficiency and satisfaction) and other types of problems. A2: DUE reduces the workload needed for usability evaluation of software for users, developers and evaluators.

Figure 5. Issue evaluation on the Instrumentation Server. User screen-video running to the left, evaluations written to the right. Full-screen viewing of video is possible.

A3: DUE enables both short-term and long-term evaluations of software, with novice users and expert users.

The evaluator then decides whether the issue is a usability problem or not. If it is judged to be a problem, it is described and rated for its severity, or classified as a sameas problem, that is a problem found before, which then is linked to the prior description of the same problem.

A4: DUE strengthens the validity of identified problems:

The evaluators do not have to use time on analysing large amounts of instrumented data and reviewing many hours of video to identify usability problems. Instead evaluators rely

121

-

By the natural context of usage.

-

By minimizing disturbing experimental factors.

-

By enabling many test users.

-

By the possibility of long-term evaluation.

Full Papers

Proceedings: NordiCHI 2010, October 16–20, 2010

A5: DUE provides an information-rich communication of the usability problems among users, evaluators, and developers.

Because of confidentiality agreements with the developing company we cannot show detailed pictures of F-Man, describe it in detail, or mention company names.

A6: DUE contributes to minimize the evaluator effect, and support a more thorough problem analysis.

F-Man has been on the market for a couple of years. It is under constant development and has never before been evaluated for usability, but is generally by customers considered to be user-friendly.

Strategy for Data Analysis

Following the general case study recommendations by Yin [27], we centered the case study on research questions designed to generate evidence that could clarify the propositions.

The users of F-Man were 16 users at a transportation company, who had been using F-Man for up to 8 months before the evaluation took place. The case study involved only one physical test site with the 16 users distributed in three offices.

Building a chain of evidence (Figure 6) ensured traceability all the way from evidence to final conclusions.

The role as evaluator was carried out by an external, highly experienced usability expert from industry. This evaluator has a Ph.D. degree in computer science and human computer interaction. The development manager role was carried out by the IT manager at the transportation company, and the role as developer was carried out by two developers at the developing software company.

Evidence  fact  proposition  conclusion

Figure 6. Chain of evidence, making it traceable how conclusions are supported by evidence.

The case study ran over one month, in total 18 work days as it was a period with many holidays.

All qualitative and quantitative evidence from the available sources were collected, see Table 1.

Installation and test of the Instrumentation Client on each participating user’s computer was done by a researcher, as the Instrumentation Client was a prototype without automatic configuration implemented yet. After installation the user was instructed in approximately 5 minutes.

Evidence was triangulated to support identified facts in the way Table 2 shows. In total 58 facts were identified. Each fact was assessed as supporting or opposing each of the propositions, see Table 3.

Due to very busy periods and customer service being a main concern for the transportation company, management preferred that users started the Instrumentation Client manually instead of this being done automatically. Because of this users were instructed to start the Instrumentation Client every day at different hours, or when they experienced an issue which could be reproduced. The Instrumentation Client automatically shut down after one hour. To help users in remembering this, a small gift was promised to the user being best at remembering the daily starts.

Alternative explanations that could challenge the propositions (rival explanations [27]) were investigated in a similar way. On this basis our conclusions are drawn. These are described in the Findings and Discussion section. The Case Study: Production Software under Active Development

To get a thorough investigation of DUE we teamed up with a software company developing and selling a fleet management and navigation product, here called F-Man. Source

Evidence

Usability issues

37 issues reported by users and evaluated by the evaluator. 30 issues prioritized by the development manager and responded to by developers. 37 issues reviewed for usability relevance by researcher.

Logged usage data

83 DUE sessions run by users, 57 hours total.

Observation

77 observations registered. 80 hours of user observation, 1 hour of evaluator and 4 hours of developers.

Timing of usage

13 user reports timed. 1 evaluator session timed. 1 developer session timed.

Interview

3 users interviewed in a total of 1:07 hours with 72 topics discussed. Evaluator interviewed for 32 minutes with 21 topics discussed. Development manager interviewed for 21 minutes with 27 topics discussed. 2 developers interviewed for 1:14 hours with 59 topics discussed.

Survey

8 users have answered the survey for users who have reported issues. 5 users have answered the survey for users who have not reported issues. 3 users chose not to answer any survey.

Documentation

1 e-mail from development manager to users. Table 1. Collected evidence in the case study.

122

Full Papers

Proceedings: NordiCHI 2010, October 16–20, 2010

Fact 43 (A2, A5, A6)

Issues are easy to understand for evaluator, development manager and developers.

Evidence

Source

ID

All users who have reported issues state that the evaluator has understood these correctly.

Survey

S16

Evaluator understands issue at first review of video in 11 out of 13 issues.

Observation

O35

Development manager states that it is easy to understand issues.

Interview

I100

Developers state that it is easy to understand issues.

Interview

Proposition A5

DUE gives an information-rich communication of the usability problems among users, evaluators, and developers

Support

Fact

Description

X

30

Structure and documentation of issues is beneficial.

X

37

Real problem can be hidden for the user, but developers easily recognize the problem.

X

43

Issues are easy to understand for evaluator, development manager and developers.

X

9

DUE is supplemented or overridden if serious problem prevents work at all.

X

15

Some users find reporting to the microphone unpleasant.

-

-

etc.

Oppose

I159

Further evidence: O40, O50, O73, I28, I77, I79, I107, I115. Table 2. Example illustrating how Fact 43 was triangulated by the collected evidence.

To protect user privacy only the screen where F-Man ran was recorded. Most users had two screens. Furthermore users were able to stop the Instrumentation Client at any point and start a new session later.

-

Table 3. Example of proposition with supporting and opposing facts.

Login information to the Instrumentation Server was emailed to the evaluator who also received a short instruction by phone. The development manager and developers also received login information by e-mail and were given a 15 minutes demonstration. Their reviews of issues were done from work or at home via a web browser.

were primarily of technical nature. A single problem could be attributed to lack of knowledge among users. The evaluator, the development manager and developers agreed that these issues were valid problems. 13 problems were considered as “same-as” by the evaluator. Thus in total 19 unique problems were caught. Of the technical problems one was a “same-as” problem, thus 3 unique technical problems were caught.

Automatic e-mail notification had not yet been implemented at the Instrumentation Server, and a researcher called the evaluator, development manager and developers when a suitable amount of new issues reported by users were ready for assessment. This happened four times during the case study.

Through the researcher’s observations and interviews 4 problems were identified, which were not reported by the users. Three of these are considered relevant usability issues. One was cosmetic and only observed once during 80 hours of observation. A second was severe, but very close to other reports (slow search facilities). The third problem was unclearly described and occurred for one user during very busy periods. The fourth was of internal technical nature, and did not show up in the F-Man user interface.

A researcher was present at the transportation company observing half of the days. This set-up enabled analysis of potential influence by the researcher. One session with the evaluator was carried out at an external usability laboratory under video-recorded observation, and one session with the developers was carried out at the company under video-recorded observation. At the end of the case study video-recorded interviews and online user surveys were carried out, see Table 1. The case study ran without any technical problems.

Our analyses indicate that the identified problems primarily are problems concerning efficiency and satisfaction, while questions concerning effectiveness and general workflow issues get less attention. To get more attention to effectiveness problems it will probably be necessary to adjust the DUE technique by having users that are less habituated with the evaluated application. Other factors influence the DUE technique’s ability to catch problems, for instance the periodic activation of the Instrumentation Client, and personal differences among the users such as willingness to speak to a microphone without personal contact with the receiver of the message.

Collected data were transcribed, analysed and documented by the researchers for use as evidence and for the identification of facts. FINDINGS AND DISCUSSION A1: DUE supports catching usability issues and other types of problems

Severe usability problems were uncovered with DUE, for example a seemingly simple problem with the pick-up and

Users reported 37 issues through the Instrumentation Client. Of these issues 32 were usability problems, 4 problems

123

Full Papers

Proceedings: NordiCHI 2010, October 16–20, 2010

A3: DUE enables both short-term and long-term evaluations, with novice users and expert users

delivery ordering not being visible in F-Man. This had a huge impact in busy hours, as it resulted in users having to call drivers constantly over the phone. The problem resulted in poor service for customers calling to hear when they could expect pick-up or delivery, and on a daily basis many work hours were wasted. Finding such a problem was heavily dependent on the evaluation taking place in the real use context.

This proposition has not been directly tested in this case study, because all the participating users were experienced daily users of F-Man. The experiment did show, however, that using DUE was easy; the experiment was running for 4 weeks without any suspension. Testing with novice users would make it possible to follow how usability problems change as users move from novice to expert. Such studies can be very useful in research of learnability of complex software, an important aspect that we know only little about [7].

An important quality gained by DUE was that the developers got a very direct and sometimes completely new insight into intricacies of the users’ work situation. In summary, the DUE technique has shown its ability to capture a significant share of the problems that the users of the application perceive as important, problems that the developers and the external usability expert agree to recognize as important.

A4: DUE strengthens the validity of identified problems

To investigate this proposition we are distinguishing between internal, external, and ecological validity. These types of validity can be clarified by discussing the following characteristics of the DUE technique: (1) natural context of usage—this influences ecological validity; (2) minimizing disturbing experimental factors—this influences internal and ecological validity; (3) enabling many test users—this influences external validity; (4) the possibility of long-term evaluation—this influences external validity.

A2: DUE reduces the workload for users, developers and evaluators

The total amount of time used by users, evaluator, development manager and developers for this DUE evaluation is for instruction 2 hours 20 minutes, and 8 hours 3 minutes to locate, understand, describe and rate the 37 reported issues.

NATURAL CONTEXT OF USAGE: All users were able to use DUE in the form of the Instrumentation Client at their workplace. A few users with older PC’s experienced a decreased performance, but only one found this to be relatively disturbing.

All the empirical data indicate a small workload induced on the participants in carrying out the evaluation. Test tasks are unnecessary, as a broad range of tasks follow by the user’s ordinary work processes. The reporting of an issue is activated by a mouse click and includes the user’s spoken explanation. The average reporting time of an issue in this experiment was 1 minute 15 seconds. The users found it easy to use the Instrumentation Client. The evaluator reported that he found the issues unusually clearly documented, and thus easy and fast to understand and assess with an average time of 2 minutes 45 seconds spent per issue. The development manager and developers had a similar experience and on average spent a similar amount of time to understand and assess the issues.

The users’ busyness did influence the reporting activity, but we found no evidence that this has reduced the validity of the reported issues. Reporting activity was influenced if users heard others report what they thought to be the same issue. Thus some under-reporting of problems in this study took place, which undermines the possibility of using the frequency of a problem report as indicator of seriousness. The fact that the case study took place at just one location in a partly open office landscape made this factor important. If an evaluation can be distributed over different physical locations this factor will play a less important role.

As mentioned, the transportation company who participated in the experiment wanted the Instrumentation Client to be turned off at busy periods. In these periods this caused a few seconds waiting time to start up the Instrumentation Client and report an issue. Also some of the periodical activations, which was part of the experimental setup, were forgotten by users. This indicates that automatic starting of the Instrumentation Client would have reduced the workload even more and would have been more convenient for the user.

The developers and the evaluator emphasized that the reported issues have given them an insight into the users’ natural work settings, including a number of especially challenging situations, which otherwise had been difficult to achieve. This confirms that DUE has contributed important contextual information which has strengthened the ecological validity of the usability problems. MINIMISING DISTURBING EXPERIMENTAL FACTORS:

In summary, the case study shows how DUE induces a small workload on the users, the evaluator and the developers.

Compared to conventional think-aloud testing the evaluator found DUE recordings to be much less influenced by irrelevant and disturbing factors. The most influencing disturbing factors seem to have been

124

Full Papers

Proceedings: NordiCHI 2010, October 16–20, 2010

the discomfort for some users of (a) speaking to a microphone, and (b) being monitored by the Instrumentation Client. Some users say that they find it difficult to describe an issue; but in only one case a description was found to be unclear and not really understandable to the developers and the evaluator. In this case the user has afterwards explained that she—in the situation of reporting—was fiery and thus unable to explain herself.

Observed factors Evaluator, development manager, and developer’s issue review takes 2:45 minutes per issue on average. User report rate is 0.35 issues per hour on average. User issue reporting takes 1:15 minutes per issue on average. Assumptions common to example 1 and 2 Evaluator, development manager, and developers each have 30 hours available for evaluation. 30 hours / 2:45 minutes per issue ≈ 654 issues in total can be evaluated. The Instrumentation Client runs for 1 hour each workday. One month is 22 workdays. Example 1: Number of users possible (Evaluation can run for one month) 22 workdays x 1 hour x 0.35 rate = 7.7 issues per user on average. 654 issues / 7.7 issues per user ≈ 84 users. 7.7 issues x 1:15 minutes ≈ 9:37 minutes spent per user on reporting.

Concerning both being monitored and speaking to a microphone, it must be noted that the majority stated that they did not feel any discomfort. For some users, however, this discomfort can have influenced their reporting activity. In the case study, the Instrumentation Client was running for one hour a day; and only one of the user’s two screens was video recorded. One could go further in providing the integrity of the user’s privacy, for instance by letting the user go through all of his/her video recordings, and delete clips that are undesirable.

Example 2: Time span possible (10 users are representative of the end user population) 10 users x 1 hour x 0.35 rate = 3.5 issues per workday on average. 654 issues / 3.5 issues per workday ≈ 186 workdays ≈ 8.5 months. 186 workdays x 1 hour x 0.35 rate x 1:15 minutes ≈ 81:23 minutes spent per user on reporting.

Table 4. Two hypothetical examples to illustrate number of users and time span possible in large-scale DUE evaluations. The average figures in the assumptions are based on measures from the case study.

ENABLING MANY TEST USERS: Based on measures of

“time used” for users, developers, and the evaluator from the case study, we have estimated that a DUE evaluation with 84 users can be done during a period of one month with an effective work effort of one week (30 hours) by an evaluator (Table 4), a development manager, and the group of developers. The participating users will on average contribute 10 minutes each. The instruction of how to use the instrumentation client will take further 5 minutes per user. The calculation is just a rough indication as it is based upon measurements of time used from this single case study. Time spent for a DUE evaluation will be heavily dependent of the number of issues reported, which again will be dependent of the software being evaluated and the involvement of the participating users. Note, that a DUE evaluation can be scaled, even after it has been started, by connecting or disconnecting users.

A5: DUE provides an information-rich communication among users, evaluators, and developers

For better communication, it is important that the usability problems are described as accurately as possible along with information-rich contextual information. Some of the developers told how they, before DUE, often experienced that they—after having spoken with a user about a problem on the phone—had not really understood the problem. The development manager added that he and the developers also experienced difficulties in their communication about usability problems. By collecting usability issues at one easily accessible place, the Instrumentation Server, the participants had one common information repository to service everybody. The study showed that the problems were easily understood by developers and the evaluator. They emphasized the effective way of combining video clips, and the user’s spoken explanations along with their mouse pointing. Often these descriptions of the issues contained contextual information, that were valuable in understanding the importance of an issue, for instance situations with interruptions by telephone calls, and difficulties in taking up work after being interrupted.

In this case study the users could work productively during the evaluation. If the software is less mature and work only is carried out for evaluation purposes, time spent by the user must be fully accounted to the evaluation cost. In the described example in Table 4 this would be 1 hour per day, totaling 22 hours per user. POSSIBILITY OF LONG-TERM EVALUATION: The case study lasted 4 weeks demonstrating that a DUE evaluation can go on for a long time. Based on the measures of time spent by the different participants in the current case study we can estimate that a DUE evaluation with 10 users lasting 8.5 months can be carried out with an effective work effort of one week by the evaluator, the development manager, and the group of developers (Table 4). Each of the 10 users can be expected to use 1 hour 21 minutes for reporting usability issues during these 8.5 months. Again, these numbers are just rough indications.

As mentioned above, some participants were not comfortable talking to the microphone or being video recorded during work; this can limit the usage of DUE, see the paragraph Minimizing disturbing experimental factors.

125

Full Papers

Proceedings: NordiCHI 2010, October 16–20, 2010

A6: DUE contributes to minimize the evaluator effect, and support a more thorough problem analysis

instrumentation with capture of screen video and voice recordings. The tool can be used for evaluation of prototypes and running versions of application software.

The detection of problems is entirely left to the users, and therefore the contribution to the evaluator effect from the evaluator’s problem detection activity is avoided. The rest of the evaluator effect, that is: (1) the evaluator’s severity rating of usability problems, and (2) the matching of similar reported usability issues as "same-as", is still a challenge. Instead we are left with a "user effect", in the sense that users identify different sets of usability problems when exposed to the same test setting [21]. With DUE we expect to meet this by involving a higher number of users, in a larger time span, than would be possible in conventional usability testing based on think-aloud protocols.

In a case study in industry DUE was demonstrated to be effective for catching important usability problems in a complex software product. The usability issues found related mainly to efficiency and satisfaction aspects, while issues related to overall effectiveness were rarely discovered. This indicates that DUE evaluations should be combined with evaluation techniques directed at identifying effectiveness issues. The contribution to the evaluator effect from the problem detection activity was partly eliminated by letting the users detect the problems. The involved stakeholders: users, developers, a development manager, and a usability expert, all expressed a high degree of satisfaction with the DUE technique. Information-rich descriptions of usability findings were stored in a common repository, which was easily accessible by all stakeholders. This served to clarify the understanding of problems and made the communication about possible solutions more concrete, avoiding many misunderstandings. The developers emphasized the usefulness of the direct and sometimes completely new insight into intricacies of the users’ work situation.

In the current study we have observed a tendency of the evaluator to rate the problems as more serious than the developers and the users. The evaluator, who was invited as an external usability expert, did only have a very brief introduction to the application domain and the users workflow. It could have been interesting to involve evaluators with a solid domain insight [6]. The task of matching usability problems as "same as" is an important aspect of usability analysis and far from an easily solved matter [13]. The repository of usability problems can to some extent support such analyses, for instance by grouping problems according to where in the application they were reported; also annotations to the problems might be to some use. But such use of the repository has not been experimented in this study.

Overall, the stakeholders assessed the quality and the validity of the DUE evaluation to be high. The important factors were the natural use context, few disturbing experimental factors, the ability of engaging many test users, and running the evaluation for a long period of time. Some users, however, reported discomfort by speaking to a microphone and by being monitored while working.

The evaluator found it easy to use the Instrumentation Server and emphasized that the issues were easy to understand—with one exception, the user reporting while fiery as mentioned above. The evaluator found it especially handy that he usually did not have to forward or rewind in the video. Starting the video clip 20 seconds before the user actually clicked for reporting an issue, established a sufficient understanding of the user’s situation. The 20 seconds worked fine and matches the “specious present” described by William James [18]. The evaluator found that the DUE evaluation seemed to have even more practical value than think-aloud testing. The provided information was vivid, informative, and convenient to use, he said.

The small workload to run a DUE evaluation makes it practically possible to perform large-scale usability evaluations, potentially revealing usability problems related to learnability as the user goes from novice to expert. Further, investigating the usability of a broad range of functionality in large and complex software becomes possible. ACKNOWLEDGMENTS

We would like to thank Dina Friis, Guðrún Hulda Jónsdóttir Johannessen, Esben Warming Pedersen, Kasper Hornbæk, Claus Topsøe-Jensen, Mikkel R. Jakobsen and Jakob G. Simonsen for valuable comments to an earlier version of this paper. Also anonymous reviewers gave many insightful comments. Special thanks go to Tobias Uldall-Espersen, the transportation company, and the software developing company—without their insight and generous cooperation this study had been impossible.

CONCLUSION

Usability evaluation draws heavily on scarce resources such as expertise in interaction design, programming, insight into the application domain and the company’s needs. Further, usability evaluation is time consuming and complex in many ways. In an effort to show that these difficulties might be reduced without compromising the quality of the evaluation work, a technique called Distributed Usability Evaluation (DUE) was designed. DUE is based on a tool with user-controlled problem detection, automation and non-intrusive

126

Full Papers

Proceedings: NordiCHI 2010, October 16–20, 2010

REFERENCES 1. Bateman, S., Gutwin, C., Osgood, N., and McCalla, G. 2009. Interactive usability instrumentation. In Proceedings of the 1st ACM SIGCHI Symposium on Engineering interactive Computing Systems, 45-54.

techniques for matching of usability problem descriptions. Interact. Comput. 20, 6 (Dec. 2008), 505-514.

14. Howarth, J., Smith-Jackson, T., and Hartson, R. 2009. Supporting novice usability practitioners with usability engineering tools. Int. J. Hum.-Comput. Stud. 67, 6 (Jun. 2009), 533-549.

2. Castillo, J. C., Hartson, H. R., and Hix, D. 1998. Remote usability evaluation: can users report their own critical incidents?. In CHI 98 Conference Summary on Human Factors in Computing Systems, 253-254.

15. ISO 9241-11. Ergonomic requirements for office work with visual display terminals (VDTs) – Part 11: Guidance on usability. International Organization for Standardization, 1998.

3. Coolican, H., Research Methods and Statistics in Psychology, 3rd ed. Hodder & Stoughton, 1999.

16. Ivory, M. Y. and Hearst, M. A. 2001. The state of the art in automating usability evaluation of user interfaces. ACM Comput. Surv. 33, 4 (Dec. 2001), 470-516.

4. Frøkjær, E., Hertzum, M., and Hornbæk, K. (2000). Measuring usability: are effectiveness, efficiency, and satisfaction really correlated? In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, CHI Letters, 2(1), 345-352.

17. Jacobsen, N. E., Hertzum, M., & John, B. E. (1998b). The Evaluator Effect in Usability Tests. Conference Summary of CHI’98 Conference on Human Factors in Computing Systems, 255-256.

5. Frøkjær, E. & Hornbæk, K. Cooperative Usability Testing: Complementing Usability Tests With User-Supported Interpretation Sessions, Proc. CHI '05 extended abstracts on Human factors in computing systems, ACM Press (2005), 1383-1386.

18. James, W. 1890. Principles of Psychology. Henry Holt & Co., (Chapter XV, The Perception of Time)

19. Kim, J. H., Gunn, D. V., Schuh, E., Phillips, B., Pagulayan, R. J., and Wixon, D. 2008. Tracking real-time user experience (TRUE): a comprehensive instrumentation solution for complex systems. In Proceeding of the 26th Annual SIGCHI Conference on Human Factors in Computing Systems, 443452.

6. Følstad, A. Work-Domaine Experts As Evaluators: Usability Inspection of Domain-Specific Work-Support Systems, IJHCI, 22, 3 (2007), 217-245.

7. Grossman, T., Fitzmaurice, G., and Attar, R. 2009. A survey of software learnability: metrics, methodologies and guidelines. In Proceedings of the 27th international Conference on Human Factors in Computing Systems, 649658.

20. Kjeldskov, J., Skov, M. and Stage, J. Instant Data Analysis. In Proc. Nordichi 2004 (2004), 233-240.

21. Lewis, J. Sample size for usability studies: Additional considerations. Human Factors, 36 (1994), 368-378.

8. Hartson, H. R., Castillo, J. C., Kelso, J., and Neale, W. C.

22. Nørgaard, M. & Hornbæk, K. What Do Usability Evaluators

1996. Remote evaluation: the network as an extension of the usability laboratory. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, 228235.

Do in Practice?: an Explorative Study of Think-Aloud Testing, Proc. of the 6th conference on Designing Interactive systems, ACM (2006), 209-218.

23. Nørgaard, M. & Hornbæk, K. Working together to Improve

9. Hartson, H. R. and Castillo, J. C. 1998. Remote evaluation for

Usability: Exploring Challenges and Successful Practices. International Journal of Technology and Human Interaction, 6(1) (2010), 33-53.

post-deployment usability improvement. In Proceedings of the Working Conference on Advanced Visual interfaces (L'Aquila, Italy, May 24 - 27, 1998). T. Catarci, M. F. Costabile, G. Santucci, and L. Taranfino, Eds. AVI '98. ACM, New York, NY, 22-29.

24. Preece, J. et al. Human-Computer Interaction, AddisonWesley, 1999.

10. Hornbæk, K. 2006. Current practice in measuring usability:

25. Spool, J. and Shroeder, W. Testing web sites: five users is

Challenges to usability studies and research. Int. J. Hum.Comput. Stud. 64, 2 (Feb. 2006), 79-102.

nowhere near enough. In Proc. CHI 2001 (2001), 285-286.

26. www.usertesting.com. Retrieved 2010 August 8th.

11. Hornbæk, K. and Frøkjær, E. (2005). Comparing usability

27. Yin, R. K., Case Study Research: Design and Methods, 4th ed.

problems and redesign proposals as input to practical systems development. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (Portland, Oregon, USA, April 02 - 07, 2005). CHI '05. ACM, New York, NY, 391-400.

Sage Publications Inc., 2009.

12. Hornbæk, K. and Frøkjær, E. (2008). Making use of business goals in usability evaluation: an experiment with novice evaluators. In Proceeding of the Twenty-Sixth Annual SIGCHI Conference on Human Factors in Computing Systems, 903912.

13. Hornbæk, K. and Frøkjær, E. (2008). Comparison of

127

Full Papers

Proceedings: NordiCHI 2010, October 16–20, 2010

A Study of Mobile Mood Awareness and Communication through MobiMood Karen Church

Telefonica Research Via Augusta 177, 08021 Barcelona, Spain

[email protected]

∗

Eve Hoggan

Dept. of Computer Science University of Tampere Tampere, Finland

[email protected]

ABSTRACT

Nuria Oliver

Telefonica Research Via Augusta 177, 08021 Barcelona, Spain

[email protected]

HCI domain. Our moods, however, are complex. They are aﬀected by many dynamic factors and can change multiple times throughout each day. Our mood can have signiﬁcant implications in terms of our experiences, our actions and most importantly on our interactions with other people. For example, it has been shown that when we are happy we are more likely to communicate with others, while when we are sad we tend to distance ourselves from friends and family [16]. Furthermore, it’s been shown previously that moods are often a common theme in conversations with friends, highlighting the extent to which mood plays an important role in our daily life [4]. There has been lots of research over the past decade which explores the detection, monitoring and communication of moods through physiological sensing [9, 17, 25], through automatic detection in online blogs1 ,2 ,[8, 13], status updates3 and online diaries[24]. Each approach oﬀers a range of trade oﬀs in terms of validity, accuracy and privacy[7]. More recently, a variety of mobile services and applications have surfaced designed to help users communicate presence information in the form of short textual messages which often convey their mood[14]. However, the majority of existing approaches to the presentation and detection of moods do not facilitate the sharing of mood information within one’s social circle or within mobile environments. Existing mobile devices allow us to sense a range of contextual factors such as location, time and activity of users. Mobile handsets provide a great opportunity to capture information in-situ and gather feedback from users while onthe-move. Given the increased popularity of social mobile awareness applications such as Loopt4 and BrightKite5 , mobile users may also be interested in sharing moods with each other and may actually be able to positively impact each other’s moods. Furthermore, given their role as personal communication devices, mobile phones are a natural choice for sharing and communicating moods with others. In an eﬀort to address the gaps in existing literature and to answer a series of research questions related to explicit mood sharing, communication and mobile awareness, we present the results of a live ﬁeld study of a mobile application called MobiMood. MobiMood is a social mobile awareness application which allows users to submit and share moods and their associated mobile contexts with friends and others

Recent research shows that there has been increased interest in investigating the role of mood and emotions in the HCI domain. Our moods, however, are complex. They are aﬀected by many dynamic factors and can change multiple times throughout each day. Furthermore, our mood can have signiﬁcant implications in terms of our experiences, our actions and most importantly on our interactions with other people. We have developed MobiMood, a proof-of-concept social mobile application that enables groups of friends to share their moods with each other. In this paper, we present the results of an exploratory ﬁeld study of MobiMood, focusing on explicit mood sharing in-situ. Our results highlight that certain contextual factors had an eﬀect on mood and the interpretation of moods. Furthermore, mood sharing and mood awareness appear to be good springboards for conversations and increased communication among users. These and other ﬁndings lead to a number of key implications in the design of mobile social awareness applications.

Keywords Moods, emotions, mobile computing,awareness, social context, location, mobile interaction, ﬁeld study

Categories and Subject Descriptors H.5.2 [Information interfaces and presentation (e.g., HCI)]: User Interface - Evaluation/methodology

General Terms Design, Experimentation, Human Factors

INTRODUCTION Recent research shows that there has been increased interest in investigating the role of mood and emotions in the ∗This project was carried out while Eve Hoggan was an intern in Telefonica Research.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. NordiCHI 2010, October 16-20, 2010, Reykjavik, Iceland. Copyright 2010 ACM ISBN: 978-1-60558-934-3...$5.00.

1

See See 3 See 4 See 5 See 2

128

http://nyti.ms/baswaG, last retrieved Aug 2010 http://www.wefeelfine.org, last retrieved Aug 2010 http://bit.ly/c4h64j, last retrieved Aug 2010 http://loopt.com, last retrieved Aug 2010 http://brightkite.com/, last retrieved Aug 2010

Full Papers

Proceedings: NordiCHI 2010, October 16–20, 2010

the market. Loopt7 or BrightKite8 enable social serendipitous encounters and keeping track of what your friends are doing. Google and Yahoo also provide generic location sharing services (Latitude9 or Fireeagle10 ), along with readily available APIs that support development of location-aware mobile applications.

while on-the-move. It supports mood sharing in a similar way to microblogging services such as twitter6 , which allows status updates to be submitted and shared easily and quickly in the form of short snippets of text. In this paper we describe the MobiMood application and present the results obtained from a 2-week ﬁeld study involving 15 users, split across 5 social groups. The ﬁndings of our ﬁeld study lead to a number of important implications in future mobile social awareness applications.

Mood awareness

RELATED WORK MobiMood supports explicit mood sharing and awareness among groups of friends while on-the-go. Aside from mood, MobiMood also allows users to share other forms of context including location, time and social context (i.e. who I’m with). As such we have identiﬁed a number of related threads of research that belong to the general category of mobile social presence [3].

Location awareness While there has been several research projects that aim to connect electronic information to a physical location, e.g. GeoNotes [18], ActiveCampus [10], and UrbanTapestries [12]. In more recent times, focus has switched from tagging of locations to location sharing. For example, researchers working at Intel’s PlaceLab project have explored explicit sharing of location information in social communication using a prototype called Reno[22]. The goal of the Reno prototype was to understand how people leverage social context in mobile environments to share location. The authors carried out a small-scale pilot study with 7 early adopters using Reno over a period of 5 days. Their preliminary results showed that Reno facilitated more eﬀective communication using a simple approach to disclose place. In [1], Barkhuus et al. described Connecto, a locationaware mobile application that allows location tagging and sharing among social groups. In a live ﬁeld study the authors found that Connecto was used to express moods, lifestyle and events as well as location information. The authors highlight how the application allowed participants to ‘tell a story’ and express feelings relating to each others perceptions of each other. The WatchMe prototype[15] is a watch-based mobile communications device designed to support awareness and initiate communication among users. WatchMe supports multiple communication modes (text messaging, voice, etc.) and uses the display of photographs as emotional references to show users that others are thinking about them. The prototype supports sharing of location and activity information. In [2], Belloni et al. focused on mobile social awareness in the context of commuting. Through a web-based mobile interface users could explicitly self-report their current location, receive a list of train times in real time and ﬁnd out which of their friends are in the same train as them. Through an informal study involving a group of 4 people, the authors found that users exhibited very diﬀerent behaviours depending on their level of acquaintance with the person in question. For example, in some cases, users openly encouraged more interaction with their friends, in other cases, they avoided any form of interaction. More recently, commercial applications have appeared on 6

See

http://www.twitter.com

129

Microblogging services such as twitter provide insights into how and what people are doing. Although such services were not designed with mood in mind, recent results show that a high percentage of entries are in fact mood related, (e.g. in Twitter see: [5]). There have also been some projects that support explicit mood sharing. For example, the MoodJam project11 from the Human Computer Interaction Institute at Carnegie Mellon University provides a visualization of moods and other peoples moods based on colored strips and words. Users choose colours that feel most appropriate to them, thus creating a personalized mood representation. The website allows users to keep a record of their moods, to learn about mood trends and to share moods with others. The goal of the project is to increase mood awareness among users and groups, however, it should be noted that the project is desktop and no consideration has been given to mobile users. Nokia developed the Context Watcher application [11] with the goal of making context tracking, storage and use easy for users. The application runs on series 60 phones and allows context sharing with family, friends, colleagues, etc. The application supports explicit mood input as one form of context (along with location, activities, body data, etc.). The eMoto [23] prototype developed by Sundstr¨ om et al. supports mobile emotional messaging and communication through a series of emotion related gestures which are then mapped to a series of shapes, colours and animations. The authors carried out a user study with 5 participants and 5 friends as spectators, in which users reported on their friendships, the social experience and reactions to the emotional messages they received. Their results showed that emotional communication was not simply transferring information plus emotion from one person to another, but that the system must provide support for the sometimes fragile communication pattern that occurs among friends. They also found that users wanted their own personal ways of expressing themselves to one-another. Shirazi et al.[21] focused on sharing emotions through selfcomposed ringtones on mobile handsets. These melodies are used to share information about emotion is an easy and nonobtrusive manner. The authors carried out a preliminary user study involving 12 participants, the results of which highlight that these personal melodies have a stronger impact than pre-composed or downloaded melodies in terms of conveying mood or emotion. However, at the receiver end, users did express concerns regarding (1) misinterpretation of the melodies and the message it was trying to convey, (2) social embarrassment and (3) a lack of control. There are also a number of commercial mobile applications that support manual mood input but the majority do not support sharing of moods among friends/groups, e.g. 7

See See 9 See 10 See 11 See 8

http://loopt.com, last retrieved Aug 2010 http://brightkite.com/, last retrieved Aug 2010 www.google.com/latitude, last retrieved Aug 2010 http://fireeagle.yahoo.net/, last retrieved Aug 2010 http://moodjam.org/, last retrieved Aug 2010

Full Papers

Proceedings: NordiCHI 2010, October 16–20, 2010

Mood of the Nation12 , My-Mood13 , and MoodSense14 . Although there has been some work that explores mood sharing in mobile environments, work to date has been limited with little focus on the role that mood awareness plays on communication in dynamic mobile environments or the impact that explicit mood sharing in social groups has on increasing awareness among users. By creating an application solely for the purpose of sharing moods within social groups we can study: (1) the impact on awareness and (2) the role of explicit mood sharing has on communication and in particular whether communication is initiated based on knowing the mood of others. As such the key contributions of this paper are as follows:

some degree. Each mood category is also associated with a diﬀerent colour based on those established by Wexner: bluesad, green-energetic, purple-tense, red-angry, yellow-happy [26]. We chose orange for the custom mood (see Figure 1b). To input a mood, the user presses one of the buttons at the bottom of the screen and a bubble will begin to grow in the rounded box in middle of the screen (Figure 1a). The longer the user holds their ﬁnger on the button, the bigger the bubble will grow. The size of the resulting bubble is mapped to the intensity of the mood (1 to 10, where 1 is represents the lowest intensity and 10 represents the highest intensity). The intensity is also displayed on a progress bar so that more absolute feedback is provided. For example, if the yellow button is pressed and held until the bubble grows to its largest size, the participant’s mood is considered to be ‘happy’ with an intensity of 10.

1. An examination of how explicit disclosure of mood increases awareness and communication among groups of mobile users through a live ﬁeld study in-the-wild.

Contexts

2. A set of design implications for future social mobile awareness applications.

Once the user has selected a mood and intensity level, he/she clicks ‘next’ and is taken to the context-input screen. On the context-input screen users record their situational and social contexts. The situational context allows us to determine more about the location of the user and is given by selecting one of four options: at home, at work, commuting, other. The social context 17 allows us to determine more about who the user is in the presence of when submitting a new mood. The social context is introduced by selecting one of six pre-deﬁned options: friends, family, alone, partner, colleague, other. In addition, we log the device ID, the date and time and the physical location (in latitude/longitude form) of each user. By logging the physical location of the user we can map meanings to the situational contexts.

MOBIMOOD MobiMood is a proof-of-concept research prototype that enables groups of friends to share their moods with each other while on-the-move. The prototype consists of two components: (1) an iPhone application that allows users to record and share moods as well as comment on the moods of others; (2) a server that synchronizes and stores all mood details in the MobiMood database15 . The server forwards an up-to-date list of all moods to the mobile application. The server also consists of an email and SMS notiﬁcation facility that informs members of the appropriate social network about new moods and new mood comments from friends. In addition, the server logs all interactions between the user and the iPhone application for oﬀ-line analysis of behaviour.

Mood Lists After submitting a mood, the user is presented with a list of the last thirty moods of their friends. The mood lists shows the name of the user who submitted the mood, the mood (in both textual and colour format), the date and time and the situational context. By using the tabs at the bottom of the screen, the user can also view their own previous moods (My Moods tab) or the moods of everyone else (Everyone tab) (see Figure 1c). Users can view the details of any of the listed moods by double-tapping on the mood in question.

Mood Entry When users launch the MobiMood application, they are presented with a mood input screen. The mood input screen shows six diﬀerent coloured buttons at the bottom of the screen, each representing a diﬀerent mood. Users can choose from one of ﬁve standard moods (sad, energetic, tense, happy and angry) or they can input their own custom mood, e.g. ‘bored’, ‘very excited’, etc. The standard moods are derived from a subset of moods found in Russell’s Circumplex of Aﬀect [19] and XMPP [20]. Russell’s Circumplex is widely used in psychology and XMPP is a common standard used in Web based social applications. Both contain a similar set of moods16 . The Circumplex Model of Aﬀect proposes that all aﬀective states arise from two fundamental neurophysiological systems: one relates to valence (a pleasure - displeasure continuum) and the other relates to arousal or alertness. Each mood can be understood as a linear combination of these two dimensions, i.e. varying degrees of both valence and arousal. We chose a set of standard moods from each of these dimensions. By allowing users to set numerical values for each mood we could still capture opposing moods to

Supported Interactions This mood detail screen (Figure 1d) provides more details about each mood. The detail screen lists the user’s name, the mood, the date and time submitted, the intensity of that mood as well as any comments submitted by the users friends about the mood. This screen also includes 3 buttons that allow a user’s friend to interact with the mood entry. These buttons include an ‘add comment’ button, a button which initiates a phone call to the user in question and a button which initiates an SMS. The inclusion of these buttons allowed us to assess if mood sharing increased awareness and communication via comments, SMS messages and phone calls within the application.

12

See http://www.moodofthenation.net/, last retrieved Aug 2010 See http://bit.ly/dCFRlQ, last retrieved Aug 2010 14 See http://bit.ly/9uKEVZ, last retrieved Aug 2010 15 We use Apache for the server requirements and all data is stored in a MySQL database. 16 Related moods by Paul Ekman: http://en.wikipedia.org/ wiki/Paul_Ekman, last retrieved Aug 2010 13

USER STUDY In this section, we describe the results of a live ﬁeld study of the MobiMood prototype with a focus on designing social 17

130

These labels were chosen based on informal pilot studies.

Full Papers

Proceedings: NordiCHI 2010, October 16–20, 2010

Figure 1: Screenshots of the MobiMood application. sionals working and socialising together in Swansea, UK18 .

mobile awareness applications. Before conducting the ﬁeld study, several pilot tests tool place in-house to evaluate the application design and the experimental methodology. The results of which led to several iterations of the MobiMood prototype with the ﬁnal version shown in Figure 1. Note that in this study we were more focused on understanding the interesting and dynamic behaviours that emerged from using the MobiMood prototype in-the-wild rather than examining the usability/interface design of the prototype.

Group # Users # Male # Female Country

1 3 2 1 Scotland

2 2 2 0 USA

3 3 2 1 Italy

4 2 1 1 Scotland

5 5 4 1 Wales

Table 1: Details on the MobiMood participants

Participants

Procedure

To take part in the user study, participants needed to own an iPhone and be part of a group willing to participate in a 2-week study where they input and share their moods with friends. We chose to study the use of MobiMood in ﬁve close-knit groups who lived, worked and studied in diﬀerent countries around the world. In total, 15 participants took part in the study (11 male and 4 female), ranging in age between 23 and 43 years (avg=28.6). The participants had a diverse set of occupations, including a journalist, a teacher, solicitors, IT professionals and students.The participants were given a small incentive of £20 for taking part and a £200 raﬄe was held at the end of the study and given to one participant. Table 1 shows more details about each of the ﬁve groups: number of users per group, the ratio of males to females and the country of origin for each group. Group 1 consisted of three young professionals from Glasgow, UK, two of which knew each other after having been ﬂatmates previously, and the other was the partner of one participant (and well acquainted with the other). Group 2 consisted of two computing science PhD students from Indiana, USA, who work together in the same lab and often socialise with each other. Group 3 consisted of three professionals from Rome, Italy. Two of these are married and the other is a family member. Group 4 was made up of two young professionals in a longterm relationship with each other from Glasgow, UK. Lastly, group 5 included ﬁve members all of whom are young profes-

Before the ﬁeld study began, users completed a pre-study questionnaire and installed the MobiMood application. The pre-study questionnaire was used to gather basic demographic information, details about their use of online social network sites as well as information about their moods and the factors that contribute to their moods. The live ﬁeld study took place over two weeks in August 2009. During the study, we collected a series of log data for post-task analysis which included: listings of the calls and SMS sent between participants19 , the time, location, type and intensity of each submitted mood (participants could choose to omit their location data), what moods were viewed, comments on moods as well as whom the participants were with at the time of mood entry. Finally, participants were asked to complete a post-study survey to gather subjective information on their 18

Although the groups in this study represent relatively small samples, we focused on groups of users with close friendships rather than larger sample groups with less intimate relationships. If such an application was integrated with larger online social networks such as Facebook, we might expect the interaction to increase, especially given that users of online social networks would be able to select only the peers they want to interact with within an application like MobiMood. See http://bit.ly/awjxuu, last retrieved Aug 2010 19 Note that we could only count calls and SMS message initiated from within the MobiMood application

131

Full Papers

Proceedings: NordiCHI 2010, October 16–20, 2010

Figure 2: Distribution of moods and average number of moods per user across custom and standard mood entries.

Figure 3: Moods submitted per user per group.

experiences with the application20 . In an eﬀort to maintain the motivation amongst participants, the ability to view moods was restricted. MobiMood only allowed users to view other moods and associated comments once they had submitted a mood. One of the issues with evaluating social applications is that often they need to reach some mass level of usage before the system can be useful to users. Our study was relatively short and we wanted to ensure that enough mood entries were collected and shared. Given the natural imbalance that exists between producers and consumers of content within social systems its unlikely that these behaviours could emerge so quickly/naturally in a short time-frame. Furthermore, participants had to submit at least 15 moods before being able to see the submitted moods and comments of everyone in the study. Prior to this, participants could only view and comment on their own moods and those of their friends. At the end of study, we provided two visualizations of submitted moods to all users: one visualization at the group level and a second visualization showing the moods of all 15 participants over the 2-week period. We used email and SMS notiﬁcations to keep users informed of the interactions of others within the study21 . Whenever a participant submitted a new mood, all of the participant’s friends received an email notiﬁcation. When a comment was added to a particular mood, the participant who originally entered that mood received an SMS to inform her of the new comment.

RESULTS Basic Usage Results In total, participants submitted 311 moods over the 2week period. 112 were standard moods (36%), while 176 were custom mood entries (56.6%)22 . Figure 2 shows the

Figure 4: Custom moods classiﬁed according to Russell’s Circumplex Model of Aﬀect. distribution of mood entries per type as well as the average number of moods per user. Of the custom moods, 103 (58.5%) were unique custom mood entries, indicating that diverse sets of custom moods were submitted by participants. The most common standard mood was “happy” and the least common was “sad”, suggesting that users tend to share their positive emotions more easily than their negative emotions. We found an average of 20.1 submitted moods per participant (stdev=9.4) and an average of 62.2 moods per group (stdev=35.4). Figure 3 shows the number of moods submitted per user per group. We can see that even groups with only 2 participants (e.g. group 4) generate a high number of mood entries (> 70 moods submitted).

Custom Moods We found that 14 of the 15 participants submitted at least one custom mood, resulting in an average of 12.6 custom moods per user. When we looked more closely into the custom moods, we found 3 types of custom mood entries: (1) basic moods, e.g. “tired”, “excited” and “rushed”. 77% of custom moods fell into this category; (2) status or activity related moods (16% of the custom moods), e.g. “hating busy

20

Both pre and post-study surveys were carefully designed to include questions that would allow us assess participate reactions to an application such as MobiMood. 21 Emails are a common notiﬁcation mechanism used in many social applications e.g. Facebook informs its users when friends comment on status, tag photos, etc. Note the email did not include a link to the application so if users did access MobiMood after receiving an email it was to interact in other ways, e.g. to post comments. 22 The type of the remaining 23 mood entries (i.e. standard or

custom) is unknown. The type is unknown for mood entries where the user did not want the application to track their location. We will discuss this later in the paper.

132

Full Papers

Proceedings: NordiCHI 2010, October 16–20, 2010

hot slow bus in the rain” and “looking forward to yoga”; and (3) combination moods, made up of a combination of a basic mood followed by a description of why the participant was in a particular mood: e.g., “grumpy (about going back to work)”, “glad, Monday done”, and “bored (of big brother!)”. 7% of custom moods fell into this category. In an attempt to better understand the custom mood entries, we manually classiﬁed all custom entries according to Russell’s Circumplex Model of Aﬀect. Figure 4 shows the percentage of custom moods that fall into these dimensions. We ﬁnd a total of 62.5% of the moods are actually unpleasant, and 62.2% fall into the deactivation dimension. Note that although we earlier found a very low occurrence of the explicit sad mood, we do ﬁnd that users expressed more negative moods in their custom mood entries. In future work, we plan to investigate why and how users are ready – or not – to communicate these more negative moods with the goal of understanding how users might be persuaded to share their mood even when they are sad. We also ﬁnd 50 (almost 30%) of the custom mood entries (out of the 176 that could be classiﬁed according to Russell’s model) are related to tiredness, implying that there are some patterns in the custom moods submitted. This is an area we’d like to explore further as part of future work.

Figure 5: Percentage of mood types by location.

Context and Mood We explored two diﬀerent types of context in the MobiMood study: (1) location and (2) social context23 . The most popular location context chosen by participants was “home”, with 41.6% of all moods submitted at this location by 14 of the 15 users (average 9.3 moods submitted per user at this location). According to the answers from the post study questionnaire, users accessed the MobiMood application at home because there were often “at a loose end” and had more time to interact with their device. It was diﬃcult to assess fully the eﬀect of location on the moods experience by users given the volume and diversity of “custom” moods, however, as shown in Figure 5, we did identify a number of trends and correlations. For example, when the majority of participants recorded their location as “at work”, energetic, sad and angry levels were considerably lower (1.6% of moods at work were energetic, sad and angry), while “tense” (9.8%) was chosen more often. Whereas when location was set to “home”, energetic levels were higher. When we asked users if the location labels (i.e. at home, at work, commuting, etc.) attached to moods helped them to understand more about their friends’ moods, 11 users answered yes while the remaining 4 said no. Of the users who answered yes, their reasons included: “I can see how these eﬀect my friends moods”, “they gave clues to what might be causing peoples moods”, and “if it was just angry it would be hard to know why they were angry without asking why but by adding commuting you would know that it was either delays/too many people etc.”. The users who reported no felt that not enough context was provided with just these labels, e.g.“too much variety of possible scenarios”. In terms of the social context associated with mood entries, we found that users submitted most of their mood entries when they were alone (35.5%). We found that 13 of the 15 users submitted moods while alone, with an average 8.5 moods submitted per user in this social context. Fig-

Figure 6: Percentage of mood types by social context. ure 6 shows the social contexts associated with individual moods. Interestingly, there were absolutely no recorded sad moods when participants were with friends or their partners and no angry moods submitted with family or their partners. Furthermore, a considerably higher number of custom moods were submitted when participants were alone (49% of custom moods). When we asked participants if the social context helped them understand more about their friends’ moods, only 7 (46.7%) answered yes. Of the users who said no, most felt that not enough additional detail was provided with these labels, e.g. “They did slightly but if at work i would assume they were with colleagues or at home it would usually be alone” while other users reported not noticing these labels as often, e.g. “didn’t really notice”. Based on initial user responses, it appears that the social context had less of an eﬀect than location in terms of understanding why someone was in a particular mood. However, as mentioned previously, given the large volume of custom moods this is more diﬃcult to assess. As part of future work we’d like to explore alternative interfaces for conveying such context to the end user which would enable us to assess fully the impact and eﬀect of location and social context on mood sharing and mood awareness.

Sharing and Viewing Moods

23

One of the core features of MobiMood is to allow users to share moods among friends. In the pre-study question-

Social context in this regard refers to who the person is with when they submit their mood.

133

Full Papers

Proceedings: NordiCHI 2010, October 16–20, 2010

ing that I could see everyone’s moods”, the results of the study show that all participants frequently attempted to view the moods submitted by strangers. On average, each participant attempted to view the list of everyone’s moods 13 times (min=1, max=61, stdev=13.9). Of the users who accessed the list of everyone’s moods, we found that 7 participants clicked on individual mood entries to access further details. On average, these 7 participants viewed an average of 6 speciﬁc moods submitted by strangers (min=1, max=26, stdev=26). These results indicate that our participants were quite curious to know what other users where doing, despite not being in the same social group.

naire we asked participants how they felt about the prospect of sharing their moods with others, while in the post-study questionnaire we gathered qualitative feedback about their experiences in sharing their moods during the study. Interestingly, we found diﬀerences in responses before and after using the MobiMood application. In the pre-study questionnaire we found users more reluctant towards sharing mood information with others. For example, in the pre-study questionnaire we asked users if they would like their mobile phone to be able to track their mood. 7 of the participants responded yes for reasons such as curiosity and the ability to track changes over time; 1 participant said no, while the remaining 7 participants said that it would depend on factors such as privacy and accuracy of the mood captured. When asked if they would like to be able to see their friend moods via their mobile phone, 10 participants said yes, mentioning curiosity again as a reason for it. One participant said, “It would make me feel more connected to them”; 3 participants said they would not like to track their friends’ moods with mobile phones, while 2 participants said that it would depend on the speciﬁc mood of their friend. We also asked participants if they would like their friends to be able to see their moods. In this case, the responses were less positive: Only 5 participants said yes. One participant stated: “I like talking about moods to people. There’s usually a story behind them!”; 8 participants said it would depend on factors such as the particular mood they were in; while 2 participants said no indicating that they would not wish to reveal more negative moods to their friends. For example, one participant commented: “sometimes hiding a bad mood is a good idea”. In the post-study questionnaire, users reactions to mood sharing were more positive. For example, in the post-study questionnaire, all 15 participants rated “viewing their friends’ moods” very highly (median=524 ) and likewise participants rated “liking the fact that friends could view their moods” also highly (median=4). Furthermore, based on the usage statistics we found that each participant viewed an average of 10.6 speciﬁc mood entries submitted by their friends (min=1, max=31, stdev=7.8). These positive ﬁndings are strengthened further when we asked users to tell us about their experiences in using MobiMood. Overall, participants indicated that sharing their moods and viewing the moods of friends’ as a positive feature of the application. For example: “I like being able to see how my friends are feeling.” and “Seeing how other people were doing was interesting, and is often lacking from other social networking apps”. As mentioned previously, the MobiMood application also allows users to view the moods of everyone within the application, i.e. the moods of people who are not within their social circle. We found that the users’ reactions to this feature was quite negative both before and after the user study. Almost the same number of participants (13 vs. 12 in the pre and post-study questionnaires respectively) stated that they would not like strangers to be able to view their moods. In both cases, participants raised privacy as their main concern: “because it’s something that is open to misinterpretation and abuse”, “mood its private or for close friends” and “it’s not really any of their business!”. However, although participants did not wish strangers to see their own moods and indicated that they were indiﬀerent in terms of “lik-

Mood and Communication As part of this research, we were interested in determining whether the expression and sharing of moods prompts communication between users. That is, if you know your friend is in a particular mood are you more inclined to contact them? Most of the participants indicated that they liked being able to associate comments with moods (median=4). However, despite very positive subjective reactions to the comments facility, at the end of the 2-week study, participants only entered a total of 28 comments on submitted moods – an average 1.9 comments per person. Some of the questionnaire responses indicated that an improved mechanism for commenting may have facilitated more comments: For example, “I like being able to see how my friends are feeling. I also like the ability to comment on other people’s moods”. We found a similar trend in terms of the SMS/Call communication facility provided by Mobimood. For example, one user commented the following as what they liked most: “The integrated SMS/commenting feature. It meant that when I saw someone in a sad or negative mood I could instantly send them a text to oﬀer support. It made me feel closer to my friends”. However in terms of actual communication, there were only 17 calls and 7 SMS messages25 recorded between friends throughout the 2-weeks. This suggests that users did not use MobiMood to communicate with each other regarding their moods. Given that we used email and SMS notiﬁcations informing participants when their friends submitted new moods and comments to existing moods, we hypothesized that users may carry out communication about their moods outside of the MobiMood application. In order to determine if this was the case, we asked participants in the post-study questionnaire if they initiated or experienced any communication about MobiMood outside of the application. Overall, 12 (80%) participants indicated that they utilised communication regarding their moods outside of the MobiMood application: 3 participants said they spoke about their moods via email, 3 participants phoned their friends, 4 sent SMS messages, 2 communicated through instant messaging systems and 1 participant engaged face-to-face conversations. Therefore, it appears as though sharing moods was a successful facilitator for communication outside of the MobiMood application. One participant said, “I liked being able to see my friends’ moods, and due to the fact that moods are generally one word (‘excited’, ‘happy’, ‘sad’, etc), it made me intrigued as to why they were feeling that way and what they were doing. It was a good catalyst for a conversation 25

We did not log any communication (e.g. phone or SMS messages) that took place outside of the MobiMood application.

24

Ratings were on a 5-point likert scale, where 1 indicates strongly disagree and 5 represents strongly agree.

134

Full Papers

Proceedings: NordiCHI 2010, October 16–20, 2010

with a friend, to see how they were”. Another user commented: “one of my friends using mobimood was a colleague, so it kind of became a ‘back-channel’ form of oﬃce communication, with the moods (‘frustrated’, ‘accomplished’, ’‘annoyed’ etc) having a shared meaning”. Also, “It made me think about their context. From my knowledge of what they were doing I was trying to guess why they applied a particular mood. From some mood messages we started conversations and this was nice”. We also explored whether the users context impacted a users willingness to engage in communication. We found that location context does have some eﬀect on the willingness to communicate with no calls or SMS messages sent when location was set to “work”. All communication occurred when users set their location to “home” or “other”. However, 11 participants (73%) indicated that the location of a user did not aﬀect the type of communication they would use. If a user‘s location was set to “work”, only 4 participants stated they would not call or SMS the person but they would choose a more subtle form of communication through comments or SMS. Conversely, if the location was set to “home”, participants felt comfortable calling their friend to discuss their mood. In the words of one participant, “if at work I would know not to call, a text or comment would be best or if at home would be ﬁne to call”. The eﬀect of social context was less important: 13 users said that the social context would not alter their willingness to interact. In other words, they would still call or send an SMS to friends even if their friends were in the company of others. However, one participant mentioned that “if (friends were) alone I would prob(ably) call”. Overall, participants found sharing moods to be a good springboard for conversation.

Figure 7: Sample visualization provided to participants at the end of the study. ticipants were provided with two mood visualizations; one for their group and a second visualization for all users across the 2-week study period. Figure 7 shows the visualization for everyone, i.e. all participants averaged over the 2 weeks. When asked how participants felt about this visualization, 13 participants said it was useful and some of the explicit comments made by participants related to increased awareness. For example, “It was cool to compare it to the public one. Also interesting to see that we all used the custom mood the most”, “I liked the look of it, and it’s fun to see how you ﬁt in.”, and “I seen that the moods most used was custom and that suggested me that everyone likes to tell precisely how they feel”. Thus the usage results combined with answers to the questionnaire suggest that not only did MobiMood help users to become more self aware of their moods; it also supported awareness among friends and family.

Mood and Awareness Over the past two years there has been explosive growth in status update and microblogging tools that provide users with the ability to share private information about their daily lives, their experiences, their interactions and their feelings. Recent work has look explicitly at whether we can discern “moods” or “feelings” from social awareness tools such as twitter. For example, researchers from the University of Vermont have looked at using twitter as a type of collective mood ring, by analyzing tweets and deciphering the general mood of the public26 . Instead of focusing on implicit detection of moods in this manner, the goal of the MobiMood user study was to see how explicit mood sharing while on-the-go could increase awareness among groups of mobile users. In the post-study questionnaire, we asked users to tell us in their own words about their experiences with other peoples’ moods. Interestingly, when asked this question, some users made explicit connections between mood sharing and increased awareness. For example, “It made me more aware of how my friends were feeling and more likely to drop them a text if need be.”, “It was in some cases insightful to know how someone was feeling” and “It added a bit of awareness. I think its nice to know when friends are happy or stressed”. These results indicate that explicit mood sharing in-situ appears to have impacted on awareness in a positive manner. As well as attempting to increase awareness with the MobiMood tool, we also used external visualizations as an means for increasing awareness. At the end of the user study, par26

See

http://nyti.ms/1I0Ern,

KEY IMPLICATIONS Designing social applications is a diﬃcult task[6] and when social applications are deployed in a mobile setting the environment becomes even more challenging. The live ﬁeld study of MobiMood has led to a number of key implications in the design of mobile social awareness applications.

Self-expression is important In order to increase awareness within mobile groups, users must ﬁrst feel like they can express themselves fully within their group and within the application. It’s important to remember that in social applications, people are the content 27 , and users of social services like to control how they express themselves and how that expression is interpreted by others. As mentioned earlier, a large proportion of mood entries were “custom” moods (almost 60%). The participants’ tendency to express their moods through the “custom” option suggests a need to express more information. In terms of expressing moods, the post study questionnaire revealed that participants would have liked to have more control over mood categories. One approach in future version of MobiMood could be to allow users to submit a combination of moods or to include a description with each mood to enable users to explain their choice of mood. Although this ﬁnd27

“A social interaction design primer”, See: last checked Aug 2010

last retrieved Aug 2010

135

http://bit.ly/YOyC,

Full Papers

Proceedings: NordiCHI 2010, October 16–20, 2010

ing is relevant to many social systems, providing innovative approaches to self-expression is particularly important when sharing information such as mood due to the intimate and complex nature of the information behind shared. One of the key challenge in providing rich self-expression capabilities in mobile environments relates to interface and interaction limitations. Touch-based interaction is a very personal modality and as such may be ideal for communicating certain moods or emotions. Another form of enriching the self-expression experience would be to incorporate sounds, haptics and images so users really feel like they have control over what they to reveal and to who.

easily ﬁnd the content of interest and easily act upon that content. And providing notiﬁcations in social applications of this nature is important because it helps to keep users informed of the interactions of other users. We used visualizations at the end of the user study to help users reﬂect upon the mood of others within the study. Ideally, the MobiMood application would have included support for provide end-users with real-time, interactive visualizations of their moods and the moods of their group. Although this feature is not implemented in the current prototype, such visualizations could potentially increase awareness and self-reﬂection.

The Need to Highlight Context

Enhancing communication Given that mobile phones are used primarily as a communications device, social mobile awareness applications provide a unique opportunity to provide end users with an arena to collaborate and communicate with friends and other users while on-the-move. Perhaps the most interesting results from our user study arose from the use of MobiMood as a communication facilitator. We found that MobiMood generated a lot of conversations between users, with 12 users communicating even outside the application through email, SMS, phone calls, faceto-face conversations and instant messages. Furthermore, by providing users with a range of communication options we’re bridging the gap between these virtual and real worlds, between indirect communications like commenting towards direct communication and increased interaction. An important point to consider when facilitating communications within social mobile awareness applications is that the choice of communication channel used by end users tends to reﬂect intent and the type of awareness desired. This is especially true when communicating more intimate and personal information such as ones mood. For example, email is considered a good form of communication that supports one-to-one and one-to-many communications where the recipient can respond at their own convenience. SMS is mostly one-to-one, quick and easy to use and tends to be less invasive than phone calls. SMS supports a type of “I want to let the other person know I’m here but I don’t want to bother them”. Although phone calls can be invasive to the receiver they support more social dynamics of face-to-face type conversations and tend to be used in situations were an immediate responses is desired or needed or in situations that are more personal in nature. As such when designing social mobile services, designers should clearly identify the communications channel they wish to support, taking into account the intent of end users, the intimacy of the information being shared, as well as the advantages and disadvantages of that choice of communications channel.

MobiMood supports two explicit forms of context to go alongside the mood entries; (1) location and (2) social context. The location context in particular provided users more information so they could interpret or understand more about their friends’ moods. However, it was clear from some user comments that even with these contextual labels, not enough detail/clariﬁcation was provided and in some cases the labels were not explicit enough. Once again, although this ﬁnding is relevant to many context-aware applications, providing rich contextual information is even important when sharing moods among users due to the complex nature of moods and emotions. Users expressed a need and desire to know more about the context of their friends moods so they could better interpret and empathize with the moods expressed by their peers. The goal behind awareness applications is to provide users with a deeper understanding of a remote persons actions, activities or in the case of MobiMood, feelings. One of the key challenges is how to provide novel, rich and fun interfaces/interaction modalities to enable users to convey contextual information to their friends. There is room for improving the interface and input mechanism for conveying such context in the MobiMood prototype which would in turn enable us to assess fully the impact and eﬀect of location and social context on mood sharing and mood awareness. As mentioned earlier haptics, gestures and persuasive UIs could all be explored to highlight context and help users interpret the moods fully thus improving awareness.

Interactions & Social Visibility One of the key ﬁndings of the MobiMood user study is that curiosity about our peers’ whereabouts and activities seems to be part of human nature. Participants were not only interested in their friends mood; they were also interested in the moods of strangers. MobiMood relied on users updating or sharing their moods with one another. Some users commented on forgetting to update their moods while busy. These users explicitly indicated that it was not that they didn’t want to update, they just forgot until an email reminder appeared. For example, “I did have trouble remembering to update it while being busy. If I saw more updates from my friends in my email I would have been reminded to update more often I think. I think in the future if you have larger groups so that more updates are sent people will remember to update more often.” To overcome the typical cold-start problem associated with social applications, there must be some social purpose (in our case mood sharing), there must be interaction and there should be social visibility. Mobile social awareness applications must engage users. Users need to be able to easily interact with the application,

SUMMARY AND FUTURE WORK We have presented MobiMood, a proof-of-concept research prototype which supports sharing of moods among groups of friends while on-the-move. We carried out a 2-week live ﬁeld study (involving 15 participants split across 5 groups) that focused on explicit mood sharing and it’s impact on increasing awareness and communication among users. Our results highlight that mobile users enjoyed the ability to share moods with friends and that this sharing process facilitated a number of interesting conversations outside of the application. We also found that certain contextual factors

136

Full Papers

Proceedings: NordiCHI 2010, October 16–20, 2010

had an eﬀect on mood, willingness to interact and the interpretation of mood. Based on our results we highlighted a number of important implications in the design of future social mobile awareness applications. We are currently investigating a number of avenues of future work related to the MobiMood prototype. We believe there is more interesting work to be done in the area of mood awareness and it’s aﬀect on our social interactions. We are developing an improved version which incorporates some of the lessons we have learned. We plan to carry out a longitudinal ﬁeld study involving more participants and more social groups. We would like to explore the social context of moods in more detail, to investigate how to facilitate more fruitful communications and conversations among users and how improving self-expression can impact on awareness within social circles.

12.

13.

14.

15.

ACKNOWLEDGEMENTS We would like to thank all the participants of the study and thanks to the NordiCHI reviewers who provided interesting insights and improvements for the ﬁnal version of the paper.

16.

REFERENCES

17.

1. L. Barkhuus, B. Brown, M. Bell, S. Sherwood, M. Hall, and M. Chalmers. From awareness to repartee: sharing location within social groups. In Proceedings of CHI’08, pages 497–506. ACM, 2008. 2. N. Belloni, L. E. Holmquist, and J. Tholander. See you on the subway: exploring mobile social software. In Proceedings of CHI’09 Extended Abstracts, pages 4543–4548. ACM, 2009. 3. F. Bentley and C. J. Metcalf. The use of mobile social presence. IEEE Pervasive Computing, 8(4):35–41, 2009. 4. K. Boehner, M. Chen, and Z. Liu. Reﬂector: An emergent impression of collective experience. In Proceedings of the CHI Workshop on Providing Elegant Peripheral Awareness. ACM Press, 2003. 5. J. Bollen and A. P. H. Mao. Modeling public mood and emotion: Twitter sentiment and socio-economic phenomena. In Proceedings of WWW ’10, pages 1–3. ACM, 2009. to appear. 6. C. Crumlish and E. Malone. Designing Social Interfaces: Principles, Patterns, and Practices for Improving the User Experience. O’Reilly, 2009. 7. A. Geven, M. Tscheligi, and L. Noldus. Measuring mobile emotions: Measuring the impossible? In Proceedings of MobileHCI’09, pages 1–3. ACM, 2009. 8. A. J. Gill, D. Gergle, R. M. French, and J. Oberlander. Emotion rating from short blog texts. In CHI ’08: Proceeding of the twenty-sixth annual SIGCHI conference on Human factors in computing systems, pages 1121–1124. ACM, 2008. 9. A. Gluhak, M. Presser, L. Zhu, and S. Esfandiyari. Towards mood based mobile services and applications. In Smart Sensing and Context, 4793/2007:159–174, 2007. 10. W. Griswold, P. Shanahan, S. Brown, and R. Boyer. Activecampus: Experiments in community-oriented ubiquitous computing. IEEE Computer, 37(10), 2003. 11. J. Koolwaaij, A. Tarlano, M. Luther, and P. Nurmi. Context watcher - sharing context information in

18.

19. 20.

21.

22.

23.

24.

25.

26.

137

everyday life. In Proceedings of Web Technologies, Applications and Services, 2006. G. Lane, S. Thelwall, A. Angus, V. Peckett, N., and West. Urban tapestries: Public authoring, place and mobility, 2005. Project ﬁnal report, Proboscis, UK, London, UK, 2005. G. Leshed and J. J. Kaye. Understanding how bloggers feel: recognizing aﬀect in blog posts. In CHI ’06: CHI ’06 extended abstracts on Human factors in computing systems, pages 1019–1024. ACM, 2006. M. Luther, S. Bohm, M. Wagner, and J. Koolwaaij. Enhanced presence tracking for mobile applications. In ISWC ’05: Proceedings of the International Semantic Web Conference, 2005. N. Marmasse, C. Schmandt, and D. Spectre. Watchme: Communication and awareness between members of a closely-knit group. In Ubicomp, volume 3205 of Lecture Notes in Computer Science, pages 214–231. Springer, 2004. C. McIntyre, L. Clark, and S. Cross. The eﬀect of induced social interaction on positive and negatives aﬀect. Bulletin of the Psychonomic Society, 29:67–70, 1991. M. Morris. Technologies for heart and mind: New directions in embedded assessment. Intel Technology Journal, 11(1), 2007. P. Persson and P. Fagerberg. Geonotes: a real-use study of a public location-aware community system, 2002. Technical Report SICS-T–2002/27-SE, SICS, University of G¨ oteborg, Sweden, 2002. J. Russell. Aa circumplex model of aﬀect. Journal of Personality and Social Psychology, 39:1161, 1980. P. Saint-Andre. Extensible messaging and presence protocol (xmpp): Instant messaging and presence, 2004. IETF RFC 3291. A. S. Shirazi, F. Alt, A. Schmidt, A.-H. Sarjanoja, L. Hynninen, J. H¨ akkil¨ a, and P. Holleis. Emotion sharing via self-composed melodies on mobile phones. In Proceedings of MobileHCI’08, pages 1–4. ACM, 2009. I. Smith, S. Consolvo, A. Lamarca, J. Hightower, J. Scott, T. Sohn, J. Hughes, G. Iachello, and G. D. Abowd. Social disclosure of place: From location technology to communication practices. IEEE Pervasive Computing, 3468:134–151, 2005. P. Sundstr¨ om, A. St˚ ahl, and K. H¨ o¨ ok. In situ informants exploring an emotional mobile messaging system in their everyday practice. International Journal of Human-Computer Studies, 65(4):388–403, 2007. J. R. Vittengl and C. S. Holt. A time-series diary study of mood and social interaction. Motivation and Emotion, 22(3):255–275, 1998. H. Wang, H. Prendinger, and T. Igarashi. Communicating emotions in online chat using physiological sensors and animated text. In Proceedings of CHI ’04 extended abstracts, pages 1171–1174. ACM, 2004. L. Wexner. The degree to which color (hues) are associated with mood-tones. The Journal of Applied Psychology, 38(6):432–435, 1954.

Full Papers

Proceedings: NordiCHI 2010, October 16–20, 2010

Engaging Spect-actors with Multimodal Digital Puppetry Céline Coutrix1, Giulio Jacucci12, Anna Spagnolli3, Lingyi Ma1, Matti Helin1, Gabriela Richard4, Lorenza Parisi5, Stefano Roveda6, Prayag Narula1 1 Helsinki Institute for Information Technology HIIT, Aalto University [email protected] 2 Department of Computer Science, University of Helsinki, [email protected] 3 HTLab Dept. of General Psychology University of Padova, [email protected] 4 Educational Communication and Technology, New York University [email protected] 5 Facoltà di Scienze della Comunicazione, Università di Roma “La Sapienza” [email protected] 6 Studio Azzuro, [email protected] ABSTRACT

spectators are invited to act in this fictional space, a truly engaging experience co-created by the puppeteer and the spectators can arise. Thus, spectators are not just passive recipients of the storytelling, but become active characters in it – spec-actors in short.

We present Euclide, a multimodal system for live animation of a virtual puppet that is composed of a data glove, MIDI music board, keyboard, and mouse. The paper reports on a field study in which Euclide was used in a science museum to animate visitors as they passed by five different stations. Quantitative and qualitative analysis of several hours of videos served investigation of how the various features of the multimodal system were used by different puppeteers in the unfolding of the sessions. We found that the puppetry was truly multimodal, utilizing several input modalities simultaneously; the structure of sessions followed performative strategies; and the engagement of spectators was co-constructed. The puppeteer uses nonverbal resources (effects) and we examined how they are instrumental to talk as nonverbal turns, verbal accompaniment, and virtual gesturing. These findings allow describing digital puppetry as an emerging promising field of application for HCI that acts as a source of insights applicable in a range of multimodal performative interactive systems.

The animation of digital objects or digital puppets does incite imagination of spectators in a particular way. Spectators interacting with such digital “beings” enter also a fictional space. Digital objects can be animated in many different ways and can be transformed in real time to provide novel possibilities for engagement and co-creation. Computer-mediated puppetry has been used mostly for animation production purposes and not for live public performances. In contrast, in this paper we focus on live digital puppetry with live audience using real-time multimodal animation as a promising area of application for engaging spectators. We describe a concrete system, Euclide, installed in a science museum in Naples, Italy. Euclide utilizes multimodal inputs: a data glove, a standard keyboard, a MIDI keyboard, and a mouse (see Figure 1, left). The system has a control center from which the puppeteer operates five different stations at the museum (see Figure 1, right). These are audiovisual output installations for the digital puppets, monitored by a camera and microphone. The puppeteer is able to switch between stations swiftly and choose to interact with passers-by. We are interested in characterizing the interactive and performative aspects of this application area by describing its three following elements:

Author Keywords

Multimodality, performative interaction, engagement, cocreation, virtual puppetry, field study, museum. ACM Classification Keywords

H5.m. Information interfaces and presentation (e.g., HCI): Miscellaneous. INTRODUCTION

Puppetry is a 30,000-years-old art [4]. Most puppetry involves storytelling, and its impact is determined by the ability to create a fictional space for the spectators that has aspects in common with magic and with play. If the

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. NordiCHI 2010, October 16–20, 2010, Reykjavik, Iceland. Copyright 2010 ACM ISBN: 978-1-60558-934-3...$5.00.

•

How the puppeteer makes use of the complex multimodal system,

•

The interaction sessions in their structure and lengths on this example of a non-work related interaction,

•

How engagement is co-constructed by the puppeteer and “spect-actors”.

Others elements that are of interest are left for future work, such as considering the social context and the meaning making process. Those elements compose are what makes

138

Full Papers

Proceedings: NordiCHI 2010, October 16–20, 2010

Figure 1. Puppeteer interacting with the multimodal system Euclide in the control room (left and center) and interactive station and young audience interacting with the system (right).

puppetry an interesting application field for HCI. In order to study the first three, we collected composite videos of puppeteer, spect-actors, and puppet over several days, with three professional puppeteers. The findings utilize video analysis employing quantitative and qualitative methods. The variables we considered arose form these research questions and the data, as we explain in the Study section.

Chinese shadow puppetry has been implemented in a system called I-Shadows [14]. The installation allowed children to create stories for an audience. In this theater, the user interacts with the system by controlling a physical puppet of either a hero or a villain, whose movements are interpreted by a vision system that sends the information to the autonomous character’s environment. With CoPuppet [6], a system for collaborative puppetry is presented. The CoPuppet project explores the possibilities offered by multimodal and cooperative interaction, in which performers, or even audience members, are called upon to affect different parts of a puppet through gestures and voice.

RELATED WORK Real-time Digital Puppetry

There are surprisingly few examples of digital puppetry, given the possibilities offered by current technologies. Computer puppetry generally is used to refer to mapping the movements of a human performer to an animated character in real time [32]. Dontcheva et al. [13] introduce a novel motion-editing technique that derives implicit relationships between the animator and character. This and other work are generally motivated by the development of tools for animation production. Not only human performers are used to this end. Mazalek and Nitsche [28] addresses production and performative challenges involved in creating machinima through the development of tangible interfaces for controlling 3D virtual actors and environments.

Some systems are not directly puppetry applications but include interactive aspects whereby digital objects are influenced by spectators. Affective Painting [33] supports self-expression by adapting in real time to the perceived emotional state of a viewer, which is recognized from his or her facial expressions. Cavazza et al. [8] introduce a prototype of multimodal acting in mixed-reality interactive storytelling, in which the position, attitude, and gestures of spectators are monitored and influence the development of the story. Camurri et al. [7] propose multi-sensory integrated expressive environments as a framework for performing arts and culture oriented to mixed-reality applications. They report an example in which an actress’s lips and face movements are tracked by the EyesWeb system and her voice is processed in real time to control music.

Other examples include development of such production tools for pupils and use in everyday play. Barnes et al. [2] present “Video Puppetry,” in which the puppeteer ﬁrst creates a cast of physical puppets, using paper, markers, and scissors. An overhead camera tracks the motions of the puppets and renders them on a new background while removing the puppeteer’s hand.

Frameworks for Performing Media and Spectators

Recently, several researchers have applied different performative or theatrical metaphors to describe the emergence of novel interaction formats and experiences that are related to real-time animated puppetry.

Liikkanen et al. [26] present PuppetWall, a multi-user, multimodal system intended for digitally augmented puppeteering. This application allows natural interaction to control puppets and manipulate playgrounds comprising background, props, and puppets. PuppetWall utilizes hand movement tracking, a multi-touch display, and emotional speech recognition for interfacing. Here, however, the idea is that the visitors are themselves having fun with the puppets and the puppeteer is not hidden backstage.

Dalsgaard and Koefoed Hansen [10] observe how the user is simultaneously operator, performer, and spectator. A central facet of aesthetics of interaction is rooted in, as they put it, the user’s experience of herself “performing her perception.” They argue that this three-in-one situation is always shaping the user’s understanding and perception of

139

Full Papers

Proceedings: NordiCHI 2010, October 16–20, 2010

the interaction, and they address the notion of the performative spectator and the spectating performer.

Figure 1 (left) shows a hidden animator controlling the movements and mimicry of a virtual character through a multimodal interface including a data glove. The animator’s hand movements “activate’’ the virtual character, controlling the mimicking, and digital effects alter the animator’s voice.

Reeves et al. [31] present a taxonomy with four broad design strategies for the performer’s manipulations of an interface and their resulting effects on spectators: the “secretive”, wherein manipulations and effects are largely hidden; the “expressive,” in which they tend to be revealed, enabling the spectator to fully appreciate the performer’s interaction; the “magical”, where effects are revealed but the manipulations that caused them are hidden; and, finally, the “suspenseful”, wherein manipulations are apparent but effects are revealed only as the spectator takes his or her turn. Benford et al. [3] extend the above framework for designing spectator interfaces with the concept of performance frames, enabling one to distinguish audience from bystanders. They conclude that ambiguity to blur the frame can be a powerful design tactic, empowering players to willingly suspend their disbelief.

The rendering of the character appears on a screen in a second space, the “stage” (see Figure 1). Five stages are scattered about the museum. The animator monitors the audience members via a microphone and a camera and reacts to them (see Figure 1). Therefore, the puppeteer can react and respond to people talking to the character.

Figure 2. Virtual character of the Euclide system.

Also central to the discussion is the framework of “Interaction as Performance” [17][18][19]. This framework is based on anthropological studies of performance that have roots in a pragmatic view of experience. In particular, the framework proposes a variety of principles aimed at describing performative interaction. One of these is that of accomplishment and intervention. The etymology of the term “performance” shows that it does not have the structuralist implication of manifesting form but, rather, a processual sense of bringing to completion or accomplishing. The concept of event and processual character is also key: performances are not generally amorphous or open-ended; they have diachronic structure, a beginning, a sequence of overlapping but isolable phases, and an end. Expression and experience is another element of import. According to pragmatist views, an experience is never completed until it is expressed. Also, in an experience there is a structural relationship between doing and undergoing.

The system offers 100 different features to the puppeteer for animating the character, among them jumping (see Figure 2) or dressing like Santa Claus (see Figure 4). To allow use of this great expressive power, with many elements sometimes utilized simultaneously, different modalities are proposed. The interface includes 11 screens, two computer keyboards, two mice, a data glove, a microphone, headphones, and a MIDI keyboard, all in the control room (see Figure 1, left and center). Among these devices, three screens, one computer keyboard, one mouse, the MIDI keyboard, the microphone, and the glove are dedicated to real-time puppetry. The other devices are dedicated to system launch, switching between interactive areas or setting the puppet to inactive in order for the puppeteer to take a break. Facial Expressions

Using a data glove, the puppeteer can horizontally open/close both eyes by bending the index finger of the glove at i2 (see Figure 3), vertically open/close both eyes by bending the index finger of the glove at i3, and move the eyebrows around the eye by bending the middle finger of the glove at m2.

These novel frameworks originate from concurrent trends in HCI, including the emergence of installations as a delivery platform for interactive experiences. Installations as also tangible interfaces have the property of providing a stage on which the user becomes at times a performer. Other trends include attention to the fact that performers in general have more and more technology to mediate their interaction with spectators. The system we present now is a novel multimodal puppetry application that aims at engaging spectators in performative sessions of interaction. THE SYSTEM: EUCLIDE

Euclide is a virtual puppet (see Figure 2) that has an engaging role in the visit of a science museum in Naples, Italy. The system offers a multimodal interface to the puppeteer in order to animate a virtual puppet and entice the audience.

Figure 3. Control points on the fingers of the glove.

To control the mouth of the puppet, the puppeteer can bend the thumb of the glove at t1 to open/close the mouth and

140

Full Papers

Proceedings: NordiCHI 2010, October 16–20, 2010

bend the thumb of the glove at t2 to raise/lower the corners of the lips. This enables, respectively, making the puppet talk and look happy or sad.

constantly. To overcome this difficulty pointed out by a puppeteer, the thumb now controls the opening of the mouth.

The color of the skin can be controlled by four of the MIDI keyboard sliders, on the basis of the three RGBA channels.

Also, expressive gestures of the puppeteer were mapped to the expression of the puppet. For instance, clenching the fist makes the puppet look angry, opening the hand makes the puppet look happy, and relaxing the hand makes the puppet seems neutral.

Some costume accessories can be added to the puppet’s face, such as hats, glasses, and mustache, by pressing keys in the left portion of the MIDI keyboard (see Figure 1).

THE STUDY

Body Movements and Expressions

We studied the system in the science museum in Naples, where it is used for engaging the audience in their visit.

The puppet has global freedom of movement in 3D space. For this, the puppeteer uses the wheel of the mouse to translate the puppet in the depth dimension. Two sliders on the MIDI keyboard enable translation of the puppet vertically and laterally. The puppeteer can also change the orientation of the puppet to, for example, have its head upside down, by pressing the “M” key on the computer keyboard (see Figure 1).

Data Collection

We video-recorded the use of the multimodal puppetry system installed in the museum in winter and spring 2009. We recorded several hours of interaction from four viewpoints synchronized in one video (see Figure 4): the audience from front left and front right, and the real-time image of both puppet and puppeteer from the front. The recording also included sound. In total, three puppeteers interacted with the system. In addition to the recording, there was an interview with the most experienced puppeteer, to examine his use of the system.

The body of the puppet can be moved locally by jumping, turning, and bending its spring. Jumping extent is controlled by the little finger of the glove (l3 in Figure 3). The lateral movement of the mouse controls rotation extent around Bit’s vertical axis. Bending forward/backward is controlled with the y dimension of mouse movement. Jumping, rotation, and bending maximums are each controlled by one of the keyboard sliders. As well as the face, some costume items can also be added to the puppet’s body, such as a Superman costume, by pressing keys in the left part of the MIDI keyboard. Background

The background behind the puppet can be modified, from black to live video of the puppeteer or to still images or prerecorded videos. The interface also allows fine-tuning the transparency of the background (via the wheel on the MIDI keyboard), the orientation of the background in the vertical plane (“N” on the computer keyboard), the zoom of the background image (controlled via a potentiometer on the MIDI keyboard), and the 3D position of the background (in the vertical plane and near to distant, each controlled via a potentiometer on the MIDI keyboard).

Figure 4: Composite video, in which the puppet is dressed as Santa Claus. Data Analysis: Procedure and Reliability Coding Scheme

We employed constant comparison analysis [15] to the video data collected. While initially conceptualized as an inductive process, whereby theory emerges from close reading and analysis of the data, contemporary practice asserts that deductive or abductive processes can also be used in constant comparison analysis [25]. For the purposes of this analysis, we used an abductive process, letting the theoretical constructs emerge from the data as well as from existing theory such as the PAD scale [30] for measuring emotion.

The design of this complex form of interaction has been driven by the requirements from the museum to use generic and flexible hardware, the puppeteers’ requirements for a large number of functionalities, and usability. For instance, the location of the controls we presented here has been studied, to allow the puppeteer to perform particular movements simultaneously. For instance, moving the mouth and the eyes of the puppet have to be enabled simultaneously for puppetry.

Constant comparison analysis as conceptualized by Glaser and Strauss [15] is suitable for the analysis of multiple data sources, including observation. Constant comparison analysis is a fundamental element of grounded theory [15] and is a means of systematically generating theory by working closely with the data and setting aside previous theoretical assumptions. The coding process is methodical

Early versions of the system were not as easy to use, and the interface has been improved with puppeteers’ assistance. For instance, opening the mouth was tiring before: it was controlled by the index finger and done

141

Full Papers

Proceedings: NordiCHI 2010, October 16–20, 2010

and starts with open coding (creating initial categories), continues with axial coding (parsing out relationships), and finally uses selective coding (coding around core categories) [34]. The theory is developed once core categories are repeated enough in the data for a point of theoretical saturation to be reached. We discussed the coding between three researchers and our final consensual scheme included 35 tracks for annotation. The audio was annotated using Praat software [5] and the video was annotated using Anvil software [23].

seldom-used features, such as costume or background change, are handled with less direct devices (e.g., the MIDI keyboard).

Selection of Clips

We selected clips via both a deductive and an inductive process. We applied a deductive process because clip selection was grounded in our research questions and objectives [12]. However, we were aware of a clip's narrative power [12] for understanding the phenomena. In other words, each clip was analyzed to understand whether it fit the scope of our research as well as how illustrative it was. Accordingly, the selection of clips also took an abductive approach, particularly influenced by Goldman's technique of negotiated clip selection [16]. The video material was evenly divided between the three coders.

Figure 5: Percentage of the duration of use for each feature (Logarithmic scale).

Figure 5 also shows that third most used features are the animation of the body (bending and rotation). Then comes the use of special effects, such as costume or background change used in specific situations when relevant. Least important features are more complex “physical” actions such as moving around.

Agreement Betweens Coders

For reliability of our annotation, we also assessed the intercoder agreement rate. As the basis, we use Cohen’s kappa index, which is a descriptive statistics that summarizes agreement as percentage of the cases on which the coders agree across a number of objects against bare chance. Index at 0.80 to 1.00 is considered very good agreement; 0.60 to 0.80 considered good agreement; 0.40 to 0.60 considered moderate agreement; 0.20 to 0.40 considered fair agreement; and 0.20 or less considered poor agreement [24]. In practice, if our assessed agreement was less than 0.6, we discussed between coders and made revision in the data, then continued testing and revising until the agreement was satisfactory, i.e. greater than 0.6.

Figure 6: Simultaneous use: no, individual, or multiple features.

In addition, we found that 2 to 4 features are used simultaneously 39% of the time (see Figure 6), whereas no features are used 27% of the time and a single feature 34% of the time. This highlights the actual significant simultaneous use by the puppeteers of the multimodal resources [27].

FINDINGS

In the presentation of the findings, we focus on how the puppeteers make use of the multimodal system, how the sessions are structured, and how engagement is co-constructed by puppeteer and spect-actors. These findings draw from the statistics distilled from the quantitative and qualitative analysis of interaction sessions.

More precisely, the most used set of features is speech alone, followed by speech and lips together characterizing the talking of the puppet and then lip movement alone to characterize expressions such as smiling. Speech without lip movements is the most frequent combination, meaning that the speech and lips are not fully in synchronization. This points to possible improvements in automating the lips’ synchronization.

Multimodal Puppetry

Several features’ use is important: the speech, lips, and eye movements, as well as bending and rotation (see Figure 5). This is related to the fact that these are distributed by microphone, different fingers of the data glove, and the mouse. Indeed, the interface for these features has been provided in the data glove and via the mouse in order for these to be easy to use simultaneously. These devices, respectively, are attached constantly to the puppeteer’s right hand and in the resting location for the left hand most of the time (i.e., when it is not required for operating a keyboard). These devices are used all the time to make to animate the speech, facial expressions and body of the puppet. Other

Figure 7 shows that the distribution of speech and betweenspeech durations for the puppeteer follows a power law. Indeed, the puppeteers keep segments of speech short, trying to make the audience react rather than monopolizing the conversation (see Figure 7a). In a similar manner, they try to show that the puppet is paying attention to the audience and therefore rarely let it remain silent for long (see Figure 7b).

142

Full Papers

Proceedings: NordiCHI 2010, October 16–20, 2010

Comparing speech and silent use of features, we found that the puppet is considerably active also when the puppeteer is silent. Consequently, the use of the features goes beyond a basic animated character as we further explain in section “Resourceful Co-constructing of Engagement”.

In all sessions we analyzed, the number of people in the audience tends to increase until it reaches a maximum near the middle of the session and then decreases towards the end. The increasing phase demonstrates that the system appeals to visitors. The audience also tends to use body movement such as waving in the second part of the session. Length of Sessions

(a)

The length of session seems to follow a power law: four clips last more than nine minutes, five are between two and nine minutes long, nine last 1–2 minutes, and seven are very short. However, we should note that some of the sessions are a continuation: the same group already interacted with the puppeteer. Moreover, in some cases the sessions are interrupted by teachers or parents who want to move on in the visit.

(b)

Figure 7: Distribution of the 35 speech (a) and 36 silence (b) durations for puppeteers, in seconds.

Phase

Audience members’ action

Puppet’s reaction

Approach

Enter (one or several people, or only a voice)

Stops activity

Testing

Present themselves

Presents itself

Laugh

Skips happily Asks what is funny

Say bad words or abuse a bit

Repeats in a mechanical way Cries, complains, and goes away

Say a keyword

Changes costume Tells a story Sings a song

Ask questions

Answers normally Answers as if crazy or slow Answers and asks the audience the same

Greet

Greets

Playing

Ending

Structure of Sessions

Drawing from previous work on interaction with entertainment-related interactive installations [20], we analyzed the structure of the sessions. Different phases were annotated: Approach, Testing, Playing, and Ending. Approach is the phase in which participants enter the interaction area, observing. Testing is that in which they start trying to interact with the installation, by taking a particular action such as touching the screen in order to find out which actions have an effect on the installation. Playing is the phase in which participants interact with the installation in an aware, active, and involved way. This phase includes the climax or main action of the interaction session. Ending is the phase in which participants have their attention diverted from the installation before they leave. Figure 8 shows that these phases are balanced in the sessions, with the exception of Approach, which could not always be recorded. In this regard, the structure of the sessions as explained by the puppeteer during the interview confirms this distribution: more actions are proposed during the playing phase (Table 1).

Figure 8: Average relative length of phases in analyzed sessions (100% is the average duration of sessions).

Table 1. Summary of the structure of sessions as reported by the most experienced puppeteer.

Considering these four phases, we studied the evolution of the perceived emotion of the audience through pleasure and arousal (see Figure 9). For this we considered the facial expressions within the group, as well as their oral expressions and gestures. We found out that pleasure is high during the approach and testing phases while arousal is low. Pleasure is less important (p<0.01) during playing phase while arousal is more important (p<0.01). During the ending phase, the visitors show more pleasure (p<0.01) with less (p<0.01), but still positive, arousal. The evolution

Emergence of Performative Structures

We grounded the following analysis on the performative framework presented in the related work. Engaging the Audience Throughout the Sessions

People in the audience don’t talk to each other, 99% of the time, and they don’t pay attention to the area outside the interactive space, 98% of the time. They prefer to talk to the puppet (65% of the time). In addition, their pleasure and arousal was never annotated as negative.

143

Full Papers

Proceedings: NordiCHI 2010, October 16–20, 2010

of pleasure and arousal demonstrates that the audience starts interacting happily but calm, then gets excited and finishes the interaction happily and calm again.

visitors who would hardly be attracted by speech, such as very young babies or a far-away, busy audience. Nonverbal turns: In this case, nonverbal resources are used to make an individual contribution within a longer interaction that also features verbal exchanges. Examples include wearing some accessories on the special request of a visitor, and subsequently commenting on it, and explaining a detail by displaying a picture of it. Verbal accompaniment: This is the case in which nonverbal actions, such as acrobatic movements, are accompanied by speech. The communication relies on the nonverbal resource, which dictates its structure and meaning. An example is provided below: the puppet makes a jump commented upon by an accompanying onomatopoeic sound.

Figure 9: Evolution of pleasure and arousal in the sessions’ phases (annotated between -1 and 1).

We also noticed that in all sessions the audience’s members carefully take turns for saying goodbye to the puppet, regardless of the way they were interacting during the session, either through a leader or simultaneously. For instance, Figure 10 shows the audience interaction throughout time in a one-minute-long session: They mainly talk all at the same time in a chaotic way. At three occasions a leader emerge in the group. Before they leave, they speak one after the other in order to say goodbye. This ending interaction pattern is recurrent in every session we analyzed.

Example 1 (Disc 2, video 1, f 1:52) (Nonverbal events are in double parentheses. Extending over multiple columns indicates that the preceding sound is stretched.) Puppet:

Ho una molla I’ve got a spring Mi serve per saltare I need it to jump ⎡

Dong:: dong:: ⎤ Dong dong ⎣((Starts jumping))⎦

As the puppet starts jumping, the main resource to communicate with the visitor switches from verbal to visual: the onomatopoeic sound “dong dong” refers directly to the jump both in its duration and in its meaning. By producing it, the puppet continues the verbal communication with the audience even during the jump; exclusive room is not given to the visual resources, but the puppeteer signals the change in priority from verbal to visual.

Figure 10: Group verbal interaction with puppet in a oneminute-long session: The audience’s members mainly speak simultaneously, interacts through a leader three times, and take turns to talk to the puppet at the end.

Virtual gesturing: In the final case, priority is given to speech; visual features are used in synch with speech, following its structure and duration. The head movement in example 2 provides an instance of this.

Another interesting finding is that the structure of session can also be related to the use of the multimodal interface. Indeed, features like special effects (costume or background change) tend to be used during the Playing phase.

Example 2 (Disc 2, video 1, f 7:57) (The notation conventions are the same as for the previous example; in addition, events are in brackets when they overlap in time.)

Resourceful Co-constructing of Engagement The Puppet’s Multimodal Resources

The puppeteer acts by using visual effects (eyeglasses, masks, clothes, etc.), and performing virtual movements (jumping, rotating, etc.); the different ways in which these nonverbal resources are used can be distinguished on the basis of their relation to talk, as is done with natural gestures [22]. The following four cases have been observed.

Visitor:

Dove sei? Where are you?

Puppet:

Sono⎡::((pause))⎤ I’m ⎣((Looking down))⎦ ⎡qui dentro,⎤ inside here ⎣((Looking around))⎦

Pure special effects: This is the case in which nonverbal resources represent the predominant modality for interacting with the visitors; they are deployed to address

non mi vedi? can’t you see me

144

Full Papers

Proceedings: NordiCHI 2010, October 16–20, 2010

Circular movement of the head is produced as the puppeteer starts replying to the question “Where are you?” and continues while the sentence is interrupted, filling the gap left by the search for an answer. The movement then stops and restarts with a different function when the answer “inside here” is finally produced: the puppet’s head, moving in a circle inside the monitor, shows what “here” refers to and performs a pointing gesture. In natural communication, verbal and nonverbal resources contribute to the creation of one, joint meaning and vary in their mutual dependence [22][29]. The four cases considered here suggest that this holds true for mediated communication as well, adapted to the specific resources of the communication medium – as is the case with the pointing gesture performed with the puppet’s head.

what it could do, in an explorative attempt that has already been observed with other performative technologies [20]. Since the visitor did not see any reaction except for a nonmotivated friendly look, she judged the character to be “dumb.” As the puppeteer started talking (and reacting to the specific, situated abuse), the abuse came rapidly to an end. The appearance of the puppet remained cartoon-like, yet the talk redefined the capacities of the puppet, reconfiguring its social presence. As is shown by De Angeli & Brahnam [11], visitors probe the cognitive abilities of a virtual character. The conversational agents studied by De Angeli & Brahnam, in this situation, mocked a human speaker and failed to show human conversational competence; in fact, the abuse often focused on the poor quality of the speech. A puppet instead can participate more properly in a verbal exchange, and this is probably responsible for the way in which abuse episodes are concluded. Let’s consider the case of irony. In extract 4, below, for instance, the puppeteer recognizes irony, which relies on the ability to understand implicit meaning [1], and is able to defeat the abuse.

Interactional Consequences

The multimodal resources do not just feed communication; they also define who the puppet is. While the repertoire of special effects per se is limited and cannot keep the visitor’s interest for a long time, talk can sustain the interaction longer. What the puppet does is entertain itself in conversations with the visitors, wherein talk is the main resource and the visual effects are instrumental to it. The visitors in our video recordings inquire about the puppet’s personal information (i.e., name, age, family, and sex), about what it can do, and about its general knowledge of the “real” world (TV shows, songs, and popular people). The puppet is so successful in this that most conversations are interrupted forcedly by an external intervention, from a parent or a teacher soliciting the young visitors to leave.

Example 4 (Disc 1, video 2, f 06:13) Child:

Ma::: chi e è tuo padre? But who’s your father

Puppet:

Ma io ne ho un sacco di papà Well, I have a lot of fathers

Child:

E allora tua mamma::: ((turns to a friend)) Then your mom:::

The puppet’s resources as a conversation partner distinguish it from other conversational agents, as is apparent from the cases of verbal abuse. In example 3, the puppeteer connects with a puppet located in a new room, with some young visitors trying to interact with it. Immediately after the puppeteer switches on, the conversation in example 3 unfolds.

Puppet:

↑Allora mia volume)) Then my mom

Child:

((pause)) ⎡che lavoro fa?⎤ What does she do for a living

Puppet:

⎣↑Allora mia ((increased volume)) Then my mum

Example 3 (Disc 1, video 3, f 00:20)

Child:

Che lavoro fa? What does she do for a living

Puppet:

AAAAAAH

Puppet:

mia mamma è la scheda madre del computer My mom is the mother board of the computer

(Same notations as in the previous examples; inaudible sound is between rounded brackets; non-verbal events are in double rounded brackets.) Child:

Vabbeh adesso basta. All right; let’s stop this

Child:

((turns back, surprised))

Puppet:

mamma?⎦

In this exchange, the visitor replies to the puppet’s announcement (“I have a lot of fathers”) with a question that implies offense to the puppet’s mother (“Then your mom?”). The puppeteer recognizes the implicit meaning in the visitor’s words before the sentence is even completed: he repeats the visitor’s ironic words twice at a higher volume and in an angry tone, responding to the indirect offense with an indirect threat. The visitor then completes the sentence (“What does she do for a living?”), the puppet reacts with a (funny) scream (“AAAAH!”), all visitors

Ogni volta che scemo, ogni volta che scemo. All the times dumb, all the times dumb. E tu invece come sei? And what about you instead, what are you?

Child:

((increased

Children: hahahah

(Scemo) Dumb

Puppet:

mamma?

((goes away))

The abuse takes place while the puppet is not animated. The visitor tried to interact with the puppet and to understand

145

Full Papers

Proceedings: NordiCHI 2010, October 16–20, 2010

laugh, and the conversation continues normally (“My mom is the motherboard of the computer”).

dedicated to the four inactive stations in the museum like presented in [9].

In synthesis, the different resources available to the puppeteer are combined together in multimodal communication with several specificities. The resulting social presence attracts the visitors effectively and avoids the low status that is attributed to other virtual characters.

Spect-actors and the Co-constructing of Engagement

The puppeteer uses nonverbal resources like visual effects and performing virtual movements. We examined the relation of these nonverbal resources to talk identifying types of use as nonverbal turns, verbal accompaniment, virtual gesturing. However we noticed that in the multimodal resources is talk that can sustain interaction longer. Talk is the main resource and the visual effects are instrumental to it. For example verbal abuses usually addressed to autonomic virtual characters (that generally are attributed a low status) are here resolved through irony by the puppeteer in an effective way. We also observed how the narrative of the sessions emerges from the interaction and contribution of both the puppeteer and the spect-actors.

DISCUSSION AND CONCLUSIONS

We described a system of digital puppetry and reported on a field study with the aim of characterizing a promising application field for HCI. The system we described featured digital puppetry using real-time multimodal animation. The attractiveness for HCI in this area lies in that it provides the possibility to introduce advanced interface techniques. Other examples of digital puppetry, such as those described by Liikkanen et al. [26] utilize expressive (emotional) feature-tracking from voice. This installation utilized a data glove and a variety of other input devices used in a multimodal way. In addition, puppetry provides a case for recent frameworks of interaction that focus on performative situations or installations. In particular, it illustrates a form of interaction where engagement is co-constructed by spectators and puppeteer through the interface.

This area of application is particularly engaging because the spectators are called upon to interact with the puppets in improvised sessions. They express themselves and therefore feel that they have the role of protagonist. Current analysis and frameworks for installations or performing media anticipate some of these themes. Dalsgaard and Koefoed Hansen [10] point to the multiple roles of the user operator, performer, and spectator (see [17], [18]). Jacucci et al. [19] point to a variety of elements characterizing interaction as performance, including the structural relationship between expression and experience. These frameworks ascribe to the user an important role in the construction of the resulting performance. We believe these frameworks can be useful in further analysis of this emergent field.

Multimodal Use of the System by the Puppeteer

We described the use of the system as truly multimodal since several features were used simultaneously. The use of features in ranked by duration served to (1) animate the speech, (2) give the puppet facial expressions, (3) animating the body, (4) use special effects, and (5) perform more complex “physical” actions such as moving around. The puppeteers used mostly the mouse and glove.

Beyond the role of the different features of the multimodal system, we showed how puppeteer and spect-actors accomplish engagement in the sessions. In particular, spect-actors have a key part in creating the narrative – the gags are often inspired by what the spectators say. While it is situated and emergent, we described how engagement is the product of particular performative strategies and skills and how it relies on collective accomplishments of the mediated puppet (puppeteer) and the spect-actors.

We inferred as implications for design the opportunity to automatically animate the lips and synchronize them with the speech of the puppeteer. This might free the puppeteer to concentrate more on expressive or symbolic acts. In addition, the expressive dimension of the interface, e.g. clenching the fist making the puppet look angry, can be further investigated. Performative Structures for Brief Interactions

In this case, the spectators were mostly pupils and teenagers and sessions lasted more than two minutes, on average. It must be noted, however, that some sessions are interrupted by teachers and others are a continuation of a previous interaction. While improvised, sessions conform to a general structure, which is also reported in interviews with the puppeteers. These structures have been observed to emerge in the use of installations [20, 21]. The groups of spectators generally are attentive to the installation (they did not talk to each other), actively interact with it, and show positive and growing interest as they interact. The puppeteer, therefore, is working with different resources, including a repertoire of gags, to be able to keep spectators engaged for several minutes.

REFERENCES

1. Attardo, Linguistic Theories of Humor. Berlin: Mouton de Gruyter, 1994. 2. Barnes et al., Video Puppetry: A Performative Interface for Cutout Animation, SIGGRAPH Asia 2008, 124:1124:9. 3. Benford et al., The Frame of the Game: Blurring the Boundary between Fiction and Reality in Mobile Experiences, CHI 2006 4. Blumenthal, Puppetry and puppets, Thames & Hudson, 2005. 5. Boersma, Weenink, Praat http://www.fon.hum.uva.nl/praat/

Implications of the found structure of the interaction lie in the design of an extended computer animated puppet

146

software,

Full Papers

Proceedings: NordiCHI 2010, October 16–20, 2010

6. Bottoni, et al., CoPuppet: Collaborative Interaction, Randy Adams, Steve Gibson and Stefan Müller Arisona eds Virtual Puppetry, In Transdisciplinary Digital Art. Sound, Vision and the New Screen, pp. 326-341.

information: designing for engagement at a public multitouch display . In: ACM CHI '10: Proceedings of the 28th international conference on Human factors in computing systems, Pp: 2267-2276, 2010.

7. Camurri et al. Communicating Expressiveness and Affect in Multimodal Interactive Systems, IEEE MultiMedia, 12(1), 2005, pp. 43-53.

22. Kendon, Gesture: Visible Action as Utterance. Cambridge, UK: Cambridge University Press, 2004.

8. Cavazza et al., Multimodal Acting in Mixed Reality Interactive Storytelling, IEEE MultiMedia, 11(3), 2004, pp. 30-39.

24. Landis, Koch, The measurement of observer agreement for categorical data in Biometrics. Vol. 33, pp. 159-174, 1977.

9. Coutrix et al., Interactivity of an Affective Puppet, Adj. Proc. of Ubicomp 2009. 10. Dalsgaard, Koefoed Hansen, Performing Perception— Staging Aesthetics of Interaction, ACM TOCHI, 15(3), 2008.

25. Leech, Onwuegbuzie, Qualitative Data Analysis: A Compendium of Techniques and a Framework for Selection for School Psychology Research and Beyond. School Psychology Quarterly, 23(4), pp. 587 -604, 2008.

11. De Angeli, Brahnam, I hate you! Disinhibition with virtual partners. Interacting with computers 20, 2008, pp. 302-310.

26. Liikkanen et al., Exploring emotions and multimodality in digitally augmented puppeteering, Proc. AVI 2008, pp. 339-342.

12. Derry et al., http://visa.inrp.fr/visa/presentation/Seminaires/Journees _inaugurales/Video_Gdlines_JLS5_09Derry.pdf, 2009.

27. Liikkanen, L., Jacucci, G. & Helin, M. (2009) ElectroEmotion: A Tool for Producing Emotional Corpora Collaboratively. In: the Proc. of Affective Computing and Intelligent Interfaces ACII 2009. Amsterdam, The Netherlands IEEE.

23. Kipp, ANVIL software, http://www.anvil-software.de/

13. Dontcheva, Yngve, Popović, Layered acting for character animation, ACM TOG, 22(3), 2003.

28. Mazalek, Nitsche, Tangible interfaces for real-time 3D virtual environments, Proceedings of the international conference on Advances in computer entertainment technology, 2007.

14. Figueiredo et al., Emergent stories facilitated, In Spierling, U., and Szilas, N., editors, ICIDS, Springer LNCS vol. 5334, 2008, pp. 218–229. 15. Glaser, Strauss, The discovery of grounded theory. New York: Aldine, 1967.

29. McNeill, Gesture and thought. Chicago: University of Chicago Press, 2005.

16. Goldman-Segall, Points of viewing children’s thinking: A digital ethnographer’s journey. Mahwah, NJ: Erlbaum, 1998.

30. Mehrabian, 1996. Pleasure-arousal-dominance: A general framework for describing and measuring individual differences in temperament. Current Psychology: Developmental, Learning, Personality, Social, 14, pp. 261-292.

17. Jacucci, Interaction as Performance. Cases of configuring physical interfaces in mixed media. Doctoral Thesis, University of Oulu, Acta Universitatis Ouluensis, 2004.

31. Reeves et al., Designing the spectator experience, Proc. of CHI 2005.

18. Jacucci, G. and Wagner, I., Performative Uses of Space in Mixed Media Environments, In Davenport, E., Turner P., Spaces, Spatiality and Technologies, Springer, London, 2005.

32. Shin et al., Computer Puppetry: An Importance-Based Approach, ACM Transactions on Graphics, 20(2), 2001, pp. 67-94.

19. Jacucci et al., A Manifesto for the Performative Development of Ubiquitous Media, Proc. 4th Decennial Conference on Critical Computing, pp. 19-28, 2005.

33. Shugrina et al., Empathic painting: interactive stylization through observed emotional state, Proc. NPAR 2006, ACM Press, 2006.

20. Jacucci et al., Bodily Explorations in Space: Social Experience of a Multimodal Art Installation, Proc. INTERACT 2009, Springer, 2009, pp. 62–75.

34. Urquhart, An Encounter with Grounded Theory: Tackling the Practical and Philosophical Issues. In Trauth, E. (Ed.) Qualitative Research in Information Systems: Issues and Trends. Idea Group Publishing, London, 2001.

21. Jacucci, G., Morrison, A., Richard, G.T., Kleimola, J., Peltonen, P., Parisi, L., Laitinen, T., Worlds of

147

Full Papers

Proceedings: NordiCHI 2010, October 16–20, 2010

Comparing User Interaction with Low and High Fidelity Prototypes of Tabletop Surfaces Jan Derboven†, Dries De Roeck†, Mathijs Verstraete†, David Geerts†, Jan Schneider-Barnes‡, Kris Luyten‡ † ‡ Centre for User Experience Research (CUO) Hasselt University – tUL – IBBT IBBT/K.U.Leuven Expertise Centre for Digital Media Parkstraat 45, 3000 Leuven, Belgium Wetenschapspark 2, 3590 Diepenbeek, Belgium {firstname.lastname}@soc.kuleuven.be {firstname.lastname}@uhasselt.be materials” that are different from the final product, such as paper, cardboard, etc. [20, 21]. High-fidelity, functional interactive prototypes are built at a later stage of the usercentred design process, and typically do use materials that you would expect to be in the final product, producing a prototype that looks and behaves much more like the final product [21]. In multi-touch technology and interaction, however, directness and visual interactivity is at the core of the experience and the interaction [16]. Because of that, it becomes hard to apply Wizard of Oz-like techniques to multi-touch directly, as problems with delay or obscured parts of the interface make it difficult for the „wizard‟ to quickly and accurately respond to user interactions. Some existing multi-touch developments deal with this by means of a „design by doing‟ approach, where ideas are implemented straight away and tested afterwards, without having a profound notion of the usefulness or chance to succeed at the start of the development [8]. From a developer‟s point of view, this is an understandable approach since the technical complexity of multi-touch hardware is relatively low. The downside of this is that implementing systems straight away is error-prone, and thus costs more time and money than creating low-fidelity prototypes. The required effort to update a system after it has been developed is significantly higher compared to creating several, rapid iterations of low-fidelity prototypes [23]. Furthermore, when implementation is started immediately, HCI practitioners coming from a nontechnical background are experiencing difficulties to fully participate in the process of creating prototypes [20], and exploring the potential of the new interaction paradigms.

ABSTRACT

This paper describes a comparative study between the usage of low-fidelity and a high-fidelity prototyping for the creation of multi-user multi-touch interfaces. The multitouch interface presented in this paper allows users to collaboratively search for existing multimedia content, create new compositions with this content, and finally integrate it in a layout for presenting it. The study we conducted consists of a series of parallel user tests using both low-fidelity and high-fidelity prototypes to inform the design of the multi-touch interface. Based on a comparison of the two test sessions, we found that one should be cautious in generalising high-level user interactions from a low towards a high-fidelity prototype. However, the lowfidelity prototype approach presented proved to be very valuable to generate design ideas concerning both high and low-level user interactions on a multi-touch tabletop. Author Keywords

Tabletop, multi-touch, creation, design.

prototyping,

fidelity,

content

ACM Classification Keywords

H5.2 User Interfaces: Input devices and strategies INTRODUCTION

In the field of HCI, methods for prototyping computer interfaces have quite a history [3, 7, 10, 20, 23]. For testing most graphical user interfaces, simulating interactivity can be done using low-fidelity prototyping methods: these are simple visualisation techniques of design ideas at very early stages of the design process, whose development does not need much time, such as paper prototyping. Low-fidelity prototyping methods typically involve “low-fidelity

To overcome these limitations, and stimulate the use of low-fidelity prototypes in the design of multi-touch tabletop interaction, we compare a low-fidelity and a high-fidelity prototype for the design of a multi-touch user interface.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. NordiCHI 2010, October 16–20, 2010, Reykjavik, Iceland. Copyright 2010 ACM ISBN: 978-1-60558-934-3...$5.00.

The paper is structured as follows. First, we will survey research in the area of prototyping, mostly focusing on multi-touch surfaces. We will then present the MuTable platform, which is used as a case for comparing low-fidelity and high-fidelity prototyping. The two test set-ups will be described in detail, before presenting the results which focus on the differences and similarities on different aspects

148

Full Papers

Proceedings: NordiCHI 2010, October 16–20, 2010

of the prototypes. We will conclude with findings about the value of multi-touch prototypes in various phases of development.

in a presentation. The scenario integrates searching and browsing for content with the creation of new content. A lot of work has already been done in the field of collaborative multi-touch applications, both on the side of detailed interaction design (for instance, gesture research as described in [12, 17, 25, 26]), as on a higher-level plane of application design and user collaboration [18]. Applications range from (collaborative) picture browsing and sharing [2, 5, 13], to gaming [1, 8, 15], storytelling [9, 22] and even specialized applications for hotels, restaurants and healthcare (e.g. the Sheraton Hotel, Resto‟Touch, and Patient Consultation showcased in [11]). However, significantly less attention has been paid to productivity tools and specifically content creation on multi-touch surfaces. The specific focus of the user tests described in this paper is on researching how people behave and interact with each other and the multi-touch work surface at a high level, while they are working on a creative assignment.

RELATED WORK

Low-fidelity prototypes can be used for different purposes. In a HCI context, paper prototypes are mostly used as tools for the evaluation of design/usability failures or successes [6]. The method is used to validate a prototype of which the findings are applied in a final product or application [1, 15]. Another approach to paper prototyping is proposed by Lim, Stolterman and Tenenberg [7], i.e. the use of a low-fidelity prototype as an explorative way to look at design ideas and as a way to think about design alternatives. Scott, Carpendale and Inkpen [19] used paper prototyping to categorize the type of interactions people have when they work collaboratively at a physical table. Findings of their study concern high-level interactions starting from a paper prototype, and do not focus on the relation with an actual multi-touch application. Research by Al Mahmud et al. [1] shows that interactivity and visualization is the core of a multi-touch experience, which is why the physical design of a low-fidelity prototype for multi-touch is something to be approached with caution. Therefore perhaps, few efforts have been made to experiment elaborately with „lowfidelity‟, physical interaction in a multi-touch development context. Examples are [14, 19, 24]. Both Piper [14] and Terrenghi [24] use a low-fidelity multi-touch oriented prototype in a focussed context, where the functionality of the tested application is relatively narrow. Furthermore, it is not always clear whether low-fidelity prototype studies allow for generalization to high-fidelity multi-touch products. Terrenghi [24] presents a study investigating this. In her study, a comparative analysis was made between a high and low-fidelity version of single-user puzzles and photo sorting tasks, focusing mainly on interface metaphors and lower-level interaction (such as one-handed vs. bimanual interaction).

The MuTable was conceived to be a multi-touch surface that is positioned horizontally, or at a small angle, convenient for people to work at while standing up. The surface was specifically intended for co-located collaboration when standing at one particular side of it, so all users face the screen in the same direction, unlike some other multi-touch hard- and software designed for access from all four sides of the surface (e.g. Microsoft Surface). This consideration also played its part in UI design, as the software was designed with users in mind standing in one specific orientation at the table. This setup makes the UI design of the MuTable software very suitable for touch tables that are placed against a wall, or that are wallmounted. When the table is positioned at a small angle in one direction, the table position will also be ergonomically better to work at, compared to a completely horizontal surface. Moreover, the MuTable design incorporates a fullscreen presentation mode in which slides created on the table can be presented. This functionality is aimed at wall projection, or wall-mounted tables that can be tilted from a small angle (in „working mode‟) to a vertical position, to serve as a presentation surface („presentation mode‟).

The approach taken in this paper focuses on higher-level interactions, such as interpersonal interaction between users. Specifically, the similarities and dissimilarities between low-fidelity prototypes and their high-fidelity counterparts in collaborative content-creation tasks are investigated.

METHOD

In our study, we performed two series of user tests: one focusing on low-fidelity prototyping and another on highfidelity prototyping. The two observation series, which are described in more detail below, were very similar in set-up. Different groups, equal in size and of the same age, worked on the same assignment. The main difference between the two tests was the material the participants worked with: the „low-fidelity‟ groups created a presentation with physical everyday objects (paper, scissors, etc.), while the „highfidelity‟ groups created a similar presentation using only digital content available on a multi-touch table.

THE MUTABLE PLATFORM

Our tabletop surface setup, conveniently called MuTable in the remainder of this paper, was used in a variety of public spaces, including museums, public events, libraries, and schools. While a lot of multi-touch applications typically focus on viewing video, browsing photographs and playing music, MuTable focuses on productivity tools for collaborative content creation. The usage scenario central in this paper is one where adolescents are asked to collaboratively use a tabletop interface for school-related work. Students are able to search for content (text documents, pictures, movies…), and integrate this content

When testing low-fidelity prototypes, and comparing them to their high-fidelity counterparts, it is important to choose the right prototyping focus. Arnowitz et al. [3] identify

149

Full Papers

Proceedings: NordiCHI 2010, October 16–20, 2010

several „prototype fidelity levels‟: a prototype does not have „just one characteristic of fidelity‟, but it has several fidelity levels. Arnowitz et al. distinguish between levels such as information design, interaction design, navigation model, visual design, etc. However, the focus aimed for in the MuTable low-fidelity prototype doesn‟t fit neatly into one of the levels described by Arnowitz et al. The aim is to look at a general level of interpersonal interaction and collaboration, and general interaction with the work surface and the material on it. In the test setup described below, the similarity in participants, group size, assignment, and tools should allow for a high-level comparison between the two series of user tests. The test setup does not, however, allow for detailed comparisons of specific interaction techniques, as the low-level interactions of handling scissors and picking up or folding paper in the low-fidelity prototype are not comparable to handling digital information in the highfidelity prototype. As also found in related work, such lowlevel comparisons would require different prototype setups, in which it can be more difficult to draw conclusions about high-level interaction.

mimics a user gradually exploring a system during first time use, opening and closing functionality.

Low-Fidelity Tests

The participants were asked to work at the prototype of the interactive table, standing up (see figure 2). Their task was to create a presentation in 45 minutes on the life and works of Leonardo da Vinci. This topic allowed the participants to create a presentation about the aspects of Leonardo they thought most interesting (arts or science), and it resembles a school assignment, as stipulated in the MuTable use scenario.

Figure 1: Low-fidelity test setup Test Design

For the test with the low-fidelity MuTable prototype, 16 adolescents aged 16-17 were recruited (7 female, 9 male participants). They participated in small groups (2 groups of 3, 5 groups of 2), in which all members of the groups already knew each other before the test: they were friends, classmates, or family.

Prototype Design

The low-fidelity prototype of the interactive table involved a table at which test users were asked to collaborate using only real-world, physical materials. The long side of the table (120x80x92 cm - LxWxH) was placed against the wall, in order to match the requirements of the envisioned touch table. At the start of the observations, only 3 sheets of A3 paper were on the table: these sheets were to be used to create a presentation on a given subject. In order to allow for a comparison between the low-fidelity and the high-fidelity prototypes, the former prototype‟s fidelity was increased by imposing a small number of restrictions. The materials to be used for the creation of the presentation were available in a separate material repository, which was hidden behind a curtain. The participants received a list of materials available for use in the presentation – they had to ask the test facilitators to bring them the material they needed. The repository included documentation materials, a typewriter to type text, a „recycle bin‟, and materials to paste (glue, tape, etc.), to cut (cutter knife, scissors, etc.), and to draw. In order to simulate screen estate usage, every material group (e.g. a set of scissors and knives) was in a separate container that had to be placed on the table. Participants were not allowed to stack the containers, or put them on the floor: all materials had to be put on the table, or given back to the observers.

Figure 2: Low-fidelity table usage

For analysis purposes, the test sessions were recorded using two video cameras. One of the test observers gave the assignment to the participants, in which the participants were asked to create a presentation for a travelling exhibition on Leonardo da Vinci. When finished, the participants presented their work and talked about the contents of their presentation. Before starting the test, participants were given a pre-test questionnaire asking about their background; after the test, they were given a post-test questionnaire asking how the assignment went. At the end of the test session, the participants received an incentive.

Since the materials were not immediately available, participants could not really anticipate what to expect when asking for specific work material (Figure 1). Test participants could ask the test observers at any time to bring them work materials from the repository, or to return these materials to the repository. In this way, the prototype

150

Full Papers

Proceedings: NordiCHI 2010, October 16–20, 2010

High-Fidelity Tests

members of the groups already knew each other before the test: they were friends, classmates, or family.

Prototype Design

The high-fidelity prototype offers similar functionality for creating presentations as the low-fidelity prototype discussed in the previous section. In general, the highfidelity MuTable platform offers a number of separate, small applications such as a typewriter, a presentation creation tool, a file browser, and a drawing application (see figure 3). The functionality is available on a surface of 110 cm x 55 cm, with a screen resolution of 1280x800 pixels. This allows two to four persons to collaborate at the MuTable in a comfortable way, depending on the specific tasks to be completed.

One of the test observers gave the assignment to the participants, which was the same assignment used in the low-fidelity tests: creating a presentation for a travelling exhibition on Leonardo da Vinci in 45 minutes. The test itself consisted of some ten minutes in which the participants were allowed to explore the table themselves. After some ten minutes, most participants arrived at a point where they needed some extra information and guidance on specific gestures and the functionality of the platform. To provide the test participants with extra information after their initial exploration, the complete MuTable functionality was explained to the participants in a short demonstration. Afterwards, they were allowed to continue working on their presentation for approx. 30 minutes. For analysis purposes, the test sessions were recorded using three video cameras. After completing the test assignment, participants presented the work they had created. Before starting the test, participants were given a pre-test questionnaire asking about their background. After the test, they were given a post-test questionnaire asking how the assignment went. At the end of the test session, the participants received an incentive.

All applications can be accessed through a central, circular menu, which can be personal (one menu per user) or common to several users: no restrictions have been imposed regarding „ownership‟ of content. The applications can be used together to create and search for content, and (re-)use that content in a presentation. The applications especially facilitate research on a specific subject, and creating new content (text, a presentation) on that subject. The highfidelity MuTable applications are the digital counterparts of the material used in the low-fidelity setup. Test participants in the high-fidelity tests, like the low-fidelity participants, could also do research and compile a presentation by reading documents, typing texts, using pictures, dropping unwanted material in a recycle bin, etc. In this way, both prototypes were very similar in their high-level functionality, regardless of lower-level, specific differences (such as the use of specific interaction methods, like gestures). 3.

6.

Comparing Low-Fidelity and High-Fidelity Tests

The observations and user feedback from both the lowfidelity and the high-fidelity test sessions were processed separately. In a first round, the test sessions were discussed and the common, recurrent themes/observations were logged. Based on this limited set of themes, the test sessions were reviewed a second time, fitting all relevant observations into the predefined themes. Afterwards, the observations were compared to each other, in order to find parallels in user behaviour between the two test settings.

1.

RESULTS

2.

5.

In this section, the results of the test sessions are divided into a number of key areas regarding high-level user interaction. The test results of both test sessions will first be discussed separately. After the separate discussion, an overall evaluation and comparison of the high and lowfidelity test results will be made.

4.

Table Layout

Figure 3: Main MuTable functionality, including 1. a central circular menu, 2. a Create submenu, 3. a typewriter, 4. a presentation creation tool (with a slide opened), 5. a file browser, 6. a piece of content (picture) opened.

Low-Fidelity Table Layout

Low-fidelity test participants tended to divide the table into several zones, similar to those described by Scott et al. [19] – for convenience sake, we will use the same terminology. One type of zone observed during the low-fidelity tests was the personal zone. This zone, typically the zone closest to a specific participant, was used only by that individual participant, and was primarily used for presentation creation, and to store ready-to-use documentation materials. The second type of zone was the group zone: this zone was used for common storage of the tools and unused documentation materials. Although some test groups clearly

Test Design

For the tests with the high-fidelity MuTable prototype, a test design similar to the low-fidelity test design was used. 16 adolescents aged 16-18 were recruited (9 female, 7 male participants – all were different participants from the ones that used the low-fidelity prototype). They participated in small groups (2 groups of 3, 5 groups of 2), in which all

151

Full Papers

Proceedings: NordiCHI 2010, October 16–20, 2010

were messier in their work than others, this division provided a structure for the test participants in which they knew where to look for a specific tool or content piece, even if one of the other participants used it last. E.g. the pair of scissors was always returned to the same storage/group zone of the table, regardless of which participant used it.

themselves, similar to the low-fidelity tests, in which participants also were free to divide the table according to their wishes. All test groups more or less naturally took a position between the two extreme approaches to table layout: a more fixed, clean layout emerged naturally from a task division between the test participants. Without a clear task division, the table layout tended to be messier.

The personal work zone consisted of the presentation being created, and numerous pages and scraps of content lying around, ready to be used in the presentation. The fact that the work zone consisted of current work materials and other content ready to be used, is consistent with the observations of Scott et al. [19] about storage zones, which states that „the location of a storage territory appeared to influence who utilized the resources contained within it‟. Shared items were indeed kept in the group storage zone, while „stored materials‟ ready to be used by one participant were kept close to that participant (see figure 4).

-

-

However, while Scott et al. only mention the use of piles to store content [19], the MuTable low-fidelity observations adds content layering to this framework. Instead of creating neat piles of content, participants tended to work on top of one or several layers of ready-to-use materials, randomly scattered on the table. In this way, most test groups effectively had more than one „layer‟ in their personal zones. The upper layer consisted of the presentation pages, and some tools they were actively using; the lower layer (often invisible, covered with the upper layer) consisted of content scraps lying around, waiting to be used. The participants lifted the upper layer in order to browse the underlying layer of content scraps.

Messy table layout. A messy table layout was in most cases the result of unclear task divisions between the test participants. Test groups that tended towards this side of the continuum mostly got annoyed with this arrangement after a while, commenting on it: „Shouldn’t we close some stuff first?‟, and „Maybe we should give the Recycle Bin a permanent place somewhere‟. Fixed table layout. This table layout mostly emerged from a more or less clear task division, e.g. one test participant typing text and creating slides, and another test participant searching for content such as images and documents. Four out of seven test groups evolved towards a more or less stable table arrangement in this way. For instance, one test group placed the keyboard in the lower left corner of the table, the presentation creation tool in the lower right corner, and the file browser in the upper right corner. The division leftright often demarcated the individual users‟ work spaces, while users kept the functionality they were working on most actively closest to them, and other functionality further away (division lower-upper half of the table – see also the section on interaction among test participants). One test group, however, also used a more or less stable table arrangement: they placed the typewriter centrally in the lower half, and the presentation creation tool centrally in the upper half. In this way, the table layout didn‟t really support simultaneous work: instead, they collaborated on the same tasks, the two test participants alternately working and observing.

Table Layout: Comparison

The table arrangements made by test participants during the low-fidelity and the high-fidelity tests were often very similar, in terms of global arrangement. Nearly all test groups divided the table in smaller vertical zones right in front of them, which were the users‟ personal zones. Apart from that, users also divided the space into horizontal zones: one personal work zone with material they were actively working on, and one common storage zone with material not used as actively, but still ready to be used.

Figure 4: Table layout emerging in most low-fidelity tests (three participants). The high-fidelity layout was often less clear-cut. High-Fidelity Table Layout

The MuTable high-fidelity prototype didn‟t impose any fixed table „layout‟. As in the low-fidelity prototype, applications as well as content could be placed anywhere on the table, and they could be resized (and, for content, also rotated) to any size. In the MuTable software, there are no defaults for window or content positioning, such as in graphical user interface (GUI) operating systems (e.g. Minimize and Maximize buttons in MS Windows, Split and Arrange All options in individual applications). Space and screen estate was to be divided by the test participants

Clear table layouts were somewhat more obvious in the low-fidelity tests, while layout in the high-fidelity prototype tended to be somewhat more „fluent‟. It is striking that in the high-fidelity prototype, a clear task division eventually lead to a more fixed layout, and an unclear task division lead to a messier table layout. Most test groups started out exploring with a messier layout – some groups later evolved into a more structured one. In the low-fidelity

152

Full Papers

Proceedings: NordiCHI 2010, October 16–20, 2010

prototype, this effect of task division was not that clear: the same table layout emerged, whether the participants had a clear task division or not. This leads to the conclusion that working with tangible materials that can be picked up and „layered‟, a suitable table layout emerges more fluently. It seems that the 2D, flat surface of the multi-touch table is somewhat limiting in this respect, an issue also explored by Terrenghi et al. [24]. Test groups that tried to work in an organic way, but weren‟t able to layer the material in front of them, often ended up with an unstructured mess – the more successful groups were „forced‟ into a more structured layout due to the two dimensionality of the content on the table. To make up for this, more explicit guidance on how to effectively divide the table into zones would be a good user support addition.

-

Turn-taking. Turn-taking collaboration was observed most often, interwoven by episodes of collaboration. Users didn‟t work individually, but took turns in actually interacting with the table, e.g. one user was just watching the other one interact with the table, or one user was reading text from a content PDF, while the other user was actually typing the text. These shifts between participants occurred mostly guided by negotiating, and comments such as „Let me try!‟ or „Can I type some text now?‟

The prevalent turn-taking behaviour obviously has an important effect on the results shown in graph 1. For instance, there was a 3-user test group in which the users didn‟t really take turns, but were actively working most of the time, dividing the table into 3 work zones (one zone per test user). With this in mind, it is understandable that two users of this group rate the table as „a bit too small‟, and only one user as „big enough, no room to spare‟.

Interaction among Test Participants Low-Fidelity Participant Interaction

When observing participant interaction, two general kinds of interaction could be distinguished, similar to the observations described in Gutwin et al. [4]. Test participants switched fluently between working individually on separate tasks and working together (e.g. working together creating content, in which one participant reads a piece of text out loud and the other one types out relevant items on the typewriter). Most of the time, participants worked individually. Even when they were all working on similar tasks, such as placing content on the presentation pages, participants tended to work individually (e.g. by each working on a different slide), merging their results afterwards. This phenomenon is described by Gutwin et al. [4] as „coupling‟: working individually in a group before „requir[ing] discussion, instruction, action, information, or consultation with another person‟.

The opposite observation is also true. Three groups out of seven mostly took turns in actively using the table; all users in these groups but one reported that the table was „big enough, with some space left‟, even though one of these groups had the maximum test group composition of three users.

High-Fidelity Participant Interaction

The interaction among the high-fidelity test participants generally went smoothly, apart from some minor incidents, such as users getting in each other‟s way while working on separate tasks (these incidents will be discussed in the section on territoriality).

Graph 1: Table size ratings in the post-test questionnaire. Participant Interaction: Comparison

Although overall, the participants in both sessions showed similar behaviour of alternating individual work with working together, an important distinction in the individual work was observed. While in the low-fidelity setting, people most of the time worked individually, in parallel, the high-fidelity setting showed people taking turns in working. In most groups, there was an active role („worker‟) and a passive role („onlooker‟), with participants constantly switching between the two roles. Since participants constantly had to work on the same surface, they tended to let one participant work at a task alone before doing work themselves. Only after some moments of on-looking, other participants started working themselves. „Tangible‟ work with 3D objects, as in the low-fidelity prototype, allowed more for individual work, without anyone looking. 2D interaction, on the other hand, often created situations in

In general, the same two kinds of interaction as described in the low-fidelity setting were observed: alternately working individually on separate tasks and working together. However, a distinction was seen between parallel, individual work, and turn-taking. -

Parallel, individual work. A minority of test users worked individually during the entire assignment, with participants working constantly on parallel, separate tasks. This parallel collaboration implies that all test users had a personal zone on the multi-touch table, in which they worked separately. When one participant found some content that could be used by another participant, it was exchanged in a fluent, natural way. Like in the low-fidelity tests, individual work was interrupted by episodes of merging results.

153

Full Papers

Proceedings: NordiCHI 2010, October 16–20, 2010

using too much space, use your own half!‟, etc.1 (see figure 5).

which participants tended to let another participant work, without „interfering‟ by starting to interact themselves. In this view, reducing interference would only be possible by increasing table size quite drastically. This behaviour impacted the way the table size was judged. With participants taking turns in working, the high-fidelity table was rated as „big enough, with some space left‟ by most participants. In contrast, most low-fidelity participants – working in parallel most of the time – rated their table as „big enough, no room to spare‟ (see graph 1). Territoriality Low-Fidelity Territoriality

In the low-fidelity test, users collaborated quite naturally. Participants exchanged material, passed material on to other participants, and often reached across the table to find the right tools to work with. Participants didn‟t really get into each other‟s way while doing this. These findings are similar to the results described in [19], where participants partitioned their interactions „with little to no verbal negotiation‟.

Figure 5: A high-fidelity test participant grabbing his head in frustration, when the other participant covers his work space.

This „territoriality‟ behaviour was, however, not related to the use of one common central, circular menu (see figure 3), or the use of individual personal menus per user. Both groups using only one common menu, and groups using more (individual) menus made remarks related to their „own‟ side of the multi-touch table.

Furthermore, users frequently exchanged tools and content material among each other. Participants passed on tools and showed photos and documents to each other. Larger tools, such as a typewriter, were not exchanged (which we thought was most likely due to physical properties such as weight). Instead, users changed places behind the table to type text on the typewriter. In general, users collaborated quite fluently, reached over to pick up the material they needed, and switched positions, e.g. to use the typewriter.

The territoriality issue is further complicated by users physically switching positions at the table. Often, when one user took over a task (such as typing) from another user, they did not move the applications on the table. Instead, they switched positions at the table, passing behind each other to reach the other part of the table in which, for instance, the typewriter was located. On the other hand, content such as pictures and books were resized and passed on, moving the digital content around instead of physically switching places.

High-Fidelity Territoriality

As already hinted at in the previous section, test participants often took turns working. Consequently, when only one user is working at the table, the user encounters no issues regarding the demarcation of his or her work territory. However, most groups worked simultaneously on more or less separate things at some point in the presentation creation (e.g. typing and browsing). This also means that the individual participants in these groups had their „own‟ functionality or content they were working on at any given moment. However, since there was no strict table division imposed, individual users‟ work sometimes got in the way of other participants‟ work. This was especially the case when resizing content and functionality on the table – not so much when moving functionality or content around the screen. In the last case, most participants appeared to „respect‟ each other‟s territory. When resizing content or functionality to get in the way of other users, remarks about it were made immediately: „Hey, this is MY part of the table!‟, „Keep your things in your own part, man!‟, „You‟re

In terms of effort, it is easier to move the functionality (which takes only one gesture), rather than to walk to the other side of the table. The position-switching behaviour suggests that territoriality doesn‟t mean „the side of the table with my content/functionality on it‟, as much as „the side of the table I am currently working on‟. While working on the collaborative assignment, users switched roles, and switched between working on different kinds of content – after switching, the „new‟ content or functionality users were working on quickly became their content or functionality. Furthermore, the „static‟ functionality and the „moveable‟ content suggest that table layout (see the section on table layout) has an important influence on territoriality, as well. Many participants seemed to consider the functionality on the table as part of the more static table layout, while content such as pictures etc. were not 1

Similar behavior has also been reported in [13]. In that experiment, however, users often didn‟t know other users interacting at the same multi-touch surface. Therefore, their reactions and comments often were more polite or covert.

154

Full Papers

Proceedings: NordiCHI 2010, October 16–20, 2010

considered to be part of that static layout. Therefore, functionality was not moved and passed on to other users.

were used only rarely were not returned, but stayed on the table, ready for use. Participants preferred to have all material on a rather crowded table, rather than use a cleaner, more organised table and go through the trouble of returning the materials and asking them back later.

Territoriality: Comparison

Although it was first thought that in the low-fidelity prototype, the typewriter was not moved around due to its weight, a similar observation was made in the high-fidelity tests. Although test users did exchange content (photos, articles,…), tools such as the typewriter tool or the content browser most often were left in its position, with the test users exchanging places.

Unused High-Fidelity Functionality

Like in the low-fidelity tests, test participants didn‟t use the recycle bin very often in the high-fidelity tests either. Instead, they bended the „rules‟ implicitly imposed by the platform, and used other, creative methods to dispose of unused materials. Participants resized the content until it was very small (and nearly invisible), or simply moved the unwanted content off-screen.

In addition, low-fidelity and high-fidelity prototypes results were quite different regarding territoriality conflicts: while in the low-fidelity setting, participants collaborated without any problems, the high-fidelity setting created a lot more territoriality issues. When comparing these territoriality issues to table size ratings (see graph 1), it becomes apparent that the type of prototype has an important effect on territoriality. While one would expect more problems to occur when participants judge the work surface as too small (i.e., in the low-fidelity prototype), the table size ratings given by participants show the opposite. In the low-fidelity setting, with a table that was rated as quite limited in work space, no territoriality issues ensued. In the high-fidelity setting, with a table that appeared large enough to most participants, several territoriality issues occurred. As in the participant interaction and table layout sections, it seems that the 2D high-fidelity vs. the 3D low-fidelity work space is an important factor here. Participants get in each other‟s way more easily on a 2D surface than in a 3D space.

As already mentioned in the table layout section, some groups tended to leave all material „lying around‟ on the table, like the low-fidelity groups. However, most of them soon got frustrated with the mess of content displayed on the table. Unused functionality: Comparison

Low-fidelity and high-fidelity tests were very similar in terms of functionality that was left unused by the participants: few test groups used the recycle bin. This suggests that low-fidelity prototypes potentially are adequate predictors of the perceived usefulness of functionality. The prototypes can predict which functionality users think of as useful, and which parts they think are redundant. However, it can be inferred that this predictive power will only hold when the interaction fidelity of the low-fidelity prototype is quite high on a more detailed level, as well. In the case of the recycle bin, lower interaction fidelity probably would have made the bin more useful. It can be assumed that the low-fidelity bin would have been used more often if users were allowed to place it underneath the table, or next to it, like a normal, real-world bin. Increasing prototype fidelity by only allowing the bin to be placed on the table made the bin less useful: it took up too much place on the table, compared to the actual benefit users had from having it on the table. Similarly, in the highfidelity prototype, the bin took up too much screen estate, given the limited amount of functionality it added.

To summarize, whereas the low-fidelity prototype allowed people to divide the available space in a natural way, as described also in the low-fidelity setting of [19], the highfidelity prototype tests in our study showed different results. Participants ran into territoriality conflicts and, depending on the nature of the items displayed (content vs. applications), they switched between passing on the digital content and physically switching places themselves to leave the table layout intact. This leads to the conclusion that specific issues, such as the 2D vs. 3D collaboration mentioned in the table layout comparison section, and a perceived difference between „static‟ functionality and „moveable‟ content (based on table layout), can have a profound influence on the way people experience territoriality in the real-world vs. in digital collaboration.

In sum, our findings suggest that, given the fact that interaction fidelity is high at a general level and at a lower, more detailed interaction level, low-fidelity multi-touch prototypes can be accurate predictors of the user‟s perceived usefulness of parts of the prototype.

Unused Functionality Unused Low-Fidelity Functionality

DISCUSSION

Although the low-fidelity tabletop prototype had a waste bin in which test participants could leave unused article scraps and other content material they did not use, it was not used as it took up too much table space. In the majority of user groups, unused scraps eventually made up the bottom layer in the work zone.

The results discussed above show that there are aspects to low-fidelity multi-touch prototyping that allow for prototyping, and aspects that resist prototyping. The way participants passed on content, but left functionality in place in the high-fidelity setting was similar to the lowfidelity setting, not because of the intrinsic properties of the functionality itself (as in the low-fidelity setting: the weight of the typewriter, etc), but because of the participant‟s

Participants tended to leave all material (both tools and content material) lying around on the table. Even tools that

155

Full Papers

Proceedings: NordiCHI 2010, October 16–20, 2010

implicit agreement on table layout. Similarly, the recycle bin was not used in both test series due to a comparable perception concerning its usefulness and the valuable screen estate it demanded.

interactions due to its physical presence on a table. This three-dimensional interaction allows users to easily „layer‟ paper documents, and retrieve them by picking up the paper. On a 2D surface, on the other hand, one is limited to overlapping content. Intrinsically linked to the above, the low-fidelity prototyping method presented in this paper is an excellent method for idea generation. In this way, the prototype has the function of a „filter‟, as proposed by Lim et al. [7]: it makes it possible „to examine an idea‟s qualities without building a copy of the final design‟. In the case presented in this paper, the low-fidelity prototype presented ideas about how to shape the interaction. The low-fidelity prototype inspired new concepts such as content layering and having all material immediately available, without „closing‟ functionality or using a recycle bin. This type of design ideas did not emerge during high-fidelity testing. LIMITATIONS AND FUTURE WORK

The study described above highlights some aspects of high and low-fidelity prototyping that are quite universal in multi-touch applications (e.g. the 2D vs. 3D distinction) and have been described in existing literature (e.g. the territoriality issues in [19]). However, future studies will have to confirm if the specific observations described in this paper can be generalised to multi-touch applications in general.

Figure 6: Similarities and dissimilarities between low-fidelity and high-fidelity prototypes.

However, the possibility to interact in a 3D space vs. on a 2D surface influenced the participant‟s interactions profoundly. While it was expected that the 3D tangible objects vs. 2D touch surface would influence more lowlevel interactions, the results show that higher-level interactions with the work surface and between participants are also influenced. This influence on higher-level interactions can be seen as a side effect of more obvious, low-level interactions, also affecting the usage of the prototype as a whole [7]. An example of this is the highlevel table layout being influenced by the (low-level interaction) possibility to stack items, and create „layers‟ of content. Although the influence of 2D vs. 3D on higherlevel interactions is often a more indirect influence, its effects cannot be ignored.

In addition, low-fidelity prototyping has some often-used, adequate methods such as paper prototyping for GUIs, using paper and post-its to create drop-down menus and other controls. For multi-touch tabletops, however, such often-used methods have not yet been established. It would be interesting to see in future work which innovative lowfidelity methods can be developed to make low-fidelity observations more predictive of high-fidelity results. CONCLUSION

We have reported on a study in which we compare highlevel interactions on a physical and digital prototype of a tabletop application. Although the low-fidelity prototype was designed to have a focus on global, high-level interaction, and not on detailed manipulation of objects, test results showed that differences in more detailed interaction and manipulation of objects also have a profound influence on higher-level interactions. In order to create testable lowfidelity multi-touch prototypes, we found that only very high-level user interactions can be generalised towards a high-fidelity prototype. In addition, the low-fidelity prototype approach presented proved to be very valuable to generate design ideas concerning both high and low-level user interactions.

We can conclude that when using low-fidelity, tangible prototyping as an evaluation method of high-level interactions, one should be cautious in interpreting the results. Although several low-fidelity observations were similar in the high-fidelity situation, the difference between 2D vs. 3D interaction also influences interaction with the surface and other users at a high level. Due to this influence, similarities between low-fidelity and highfidelity prototypes stop, and observations start to diverge. It is, however, difficult and unpredictable to pinpoint the exact point at which the observations diverge. As shown in figure 6, similarities between the low and high fidelity tests always lie on a high level of interaction. The dissimilarities found can mostly be related to the prototyping medium used and the specific constraints linked to that medium. For example, paper is a medium that enables three-dimensional

ACKNOWLEDGEMENTS

MuTable is a Belgian project funded by IBBT. Project partners in the project consortium are CUO (KULeuven), EDM (UHasselt), SMIT (VUB), Kunstencentrum Vooruit, Artec, STAM, Ravi, and University Library of Ghent.

156

Full Papers

Proceedings: NordiCHI 2010, October 16–20, 2010

Mine, Don‟t Touch!: interactions at a large multi-touch display in a city centre. In Proc. CHI 2008, ACM Press (2008), 1285-1294.

REFERENCES

1. Al Mahmud, A., Mubin, O., Shahid, S., and Martens, J. Designing and evaluating the tabletop game experience for senior citizens. In Proc. NordiCHI 2008, ACM Press (2008), 403-406. 2. Apted, T., Kay, J., and Quigley, A. Tabletop sharing of digital photographs for the elderly. In Proc. CHI 2006, ACM Press (2006), 781-790.

14. Piper A.M., Hollan J.D., Tabletop Displays for Small Group Study: Affordances of Paper and Digital Materials. In Proc. CHI 2009, ACM Press (2009), 12271236.

3. Arnowitz, J., Arent M., and Berger, N. Effective Prototyping for Software Makers. Morgan Kaufmann, San Francisco, CA, 2007.

15. Piper, A. M., O'Brien, E., Morris, M. R., and Winograd, T. SIDES: a cooperative tabletop computer game for social skills development. In Proc. CSCW '06, ACM Press (2006), 1-10.

4. Gutwin, C., and Greenberg, S. A Descriptive Framework of Workspace Awareness for Real-Time Groupware. Comput. Supported Coop. Work 11, 3 (Nov. 2002), 411-446.

16. Quigley, A., Subramanian, S, and Izadi, S. Special issue on interaction with coupled and public displays. Personal and Ubiquitous Computing, 13, 8, SpringerVerlag, November 2009.

5. Kristensson, P. O., Arnell, O., Björk, A., Dahlbäck, N., Pennerup, J., Prytz, E., Wikman, J., and Åström, N. InfoTouch: an explorative multi-touch visualization interface for tagged photo collections. In Proc. NordiCHI 2008. ACM Press (2008), 491-494.

17. Saffer, D. Designing Gestural Interfaces. O‟Reilly, Sebastopol, CA, 2009.

6. Lim, Y., Pangam, A., Periyasami, S., and Aneja, S. Comparative analysis of high- and low-fidelity prototypes for more valid usability evaluations of mobile devices. In Proc. NordiCHI 2006. ACM Press (2006), 291–300. 7. Lim, Y., Stolterman, E., Tenenberg, J. The anatomy of prototypes: Prototypes as filters, prototypes as manifestations of design ideas. ACM Transactions on Computer-Human Interaction. 15, 2 (2008), 1-27. 8. Marco, J., Cerezo, E., Baldassarri, S., Mazzone, E., and Read, J. C. Bringing tabletop technologies to kindergarten children. In Proc. British Computer Society conference on HCI 2009. British Computer Society (2009), 103-111. 9. Mazalek, A., Winegarden, C., Al-Haddad, T., Robinson, S. J., and Wu, C. Architales: physical/digital co-design of an interactive story table. In Proc.TEI 2009, ACM Press (2009), 241-248. 10. McCurdy, M., Connors, C., Pyrzak, G., Kanefsky, B., and Vera, A. Breaking the fidelity barrier: an examination of our current characterization of prototypes and an example of a mixed-fidelity success. In Proc. CHI 2006, ACM Press (2006), 1233-1242. 11. Microsoft Surface. www.surface.com. 12. North, C., Dwyer, T., Lee, B., Fisher, D., Isenberg, P., Robertson, G., and Inkpen, K. Understanding Multitouch Manipulation for Surface Computing. In Proc. IFIP TC 13 Conf. on Human-Computer interaction: Part II. Lecture Notes In Computer Science, vol. 5727. Springer-Verlag, Berlin, Heidelberg, 236-249. 13. Peltonen, P., E. Kurvinen, A. Solovaara, G. Jacucci, T. Ilmonen, J. Evans, A. Oulasvirta, and P. Saarikko. It‟s

18. Scott, S.D., Grant, K.D., Mandryk, R.L. System Guidelines for Co-located, Collaborative Work on a Tabletop Display. In Proc. ECSCW 2003, Kluwer Academic Publishers (2003), 159-178. 19. Scott, S.D., Carpendale, M.S.T. & Inkpen, K.M. Territoriality in Collaborative Tabletop Workspaces. In Proc. CSCW’04, ACM Press (2004), 294-303. 20. Sefelin, R., Tscheligi, M., and Giller, V. Paper prototyping - what is it good for? A comparison of paper- and computer-based low-fidelity prototyping. In Proc. CHI 2003,Extended Abstracts on Human Factors in Computing Systems, ACM Press (2003), 778-779. 21. Sharp, H., Rogers, Y., Preece, J. Interaction Design. Beyond Human-Computer Interaction. (2nd. edition) Wiley, West Sussex, England, 2007. 22. Shen, C., Lesh, N. B., Vernier, F., Forlines, C., and Frost, J. Sharing and building digital group histories. In Proc. CSCW 2002, ACM Press (2002), 324-333. 23. Snyder, C. Paper Prototyping. The Fast and Easy Way to Design and Refine User Interfaces. Morgan Kaufmann, San Francisco, CA, 2003. 24. Terrenghi, L., Kirk, D., Sellen, A., and Izadi, S. Affordances for manipulation of physical versus digital media on interactive surfaces. In Proc. CHI 2007. ACM Press (2007), 1157-1166. 25. Wobbrock, J. O., Morris, M. R., and Wilson, A. D. User-defined gestures for surface computing. In Proc. CHI 2009, ACM Press (2009), 1083-1092. 26. Yee, W. Potential Limitations of Multi-touch Gesture Vocabulary: Differentiation, Adoption, Fatigue. In Proc. HCI International 2009. J. A. Jacko, (Ed.) Lecture Notes In Computer Science, vol. 5611. Springer-Verlag, Berlin, Heidelberg, 291-300.

157

Full Papers

Proceedings: NordiCHI 2010, October 16–20, 2010

Drop-and-Drag: Easier Drag&Drop on Large Touchscreen Displays Sebastian Doeweling SAP Research Center Darmstadt Bleichstr. 8, 64283 Darmstadt, Germany [email protected]

Urs Glaubitt Michaelisstr. 16A, 64293 Darmstadt, Germany [email protected] [15]). In office settings, larger displays were found to generally improve performance and were preferred to averagesized displays by the majority of users [8, 6]. Furthermore, tasks that require spatial orientation were found to benefit from the use of large displays (e.g. [22]). This effect was found to be stronger when resolution is also increased [1].

ABSTRACT

Large displays have been found to offer a number of benefits over average-sized desktop displays: They increase productivity in office settings, improve performance on spatial tasks and offer increased user satisfaction in several contexts. However, their physical dimensions can complicate drag&drop interactions for users, especially when touch or pen input is used. Existing approaches (e.g. push-and-pop) have addressed this problem for simple drag&drop operations, but fall short when it comes to more complex ones (e.g. dropping a target onto a currently hidden node of a file tree or a specific location on a digital map).

Additionally, Hawkey et al. [12] report that the interaction setting defined by large touch-operated displays, i.e. users near to each other as well as near to the display, is most effective for collaborative tasks. This claim is backed by a number of field reports, e.g. [20] (office and research settings) or [13] (digital design studios).

To address this issue, we propose drop-and-drag, an interaction technique which introduces fully interactive proxy targets and which allows the interruption and resumption of drag&drop operations. The results of a controlled experiment show that drop-and-drag is significantly faster than traditional drag&drop for sufficiently distant targets. Additionally, the results report improved user satisfaction when drop-and-drag is used, especially for complex drop targets.

However, as reported by e.g. Robertson et al. [19], there are also challenges that accompany especially the use of large displays: Discontinuities in the displayed image caused by bezels for installations formed by multiple screens/projectors; configuration problems when screens are added to or removed from multiple monitor installations; cursor tracking problems; problems with window and task management; and problems related to accessing distal information. As display size increases, distal access gains particular importance: large touch- or pen-operated wall displays may well confront the user with the unattainable challenge of interacting with elements beyond his/her physical limits; in multi-user settings spatial interference, both physical[16] and virtual[23], may be an additional challenge.

Author Keywords

drop-and-drag, wall-size display, drag&drop, touch screen, interaction technique ACM Classification Keywords

H.5.2 Information Interfaces and Presentation: User Interfaces—Graphical user interfaces

Drag&drop is a very common interaction with desktop applications that suffers from these limitations. Especially for touch-operated devices, which require the user to maintain physical contact2 during the whole operation, drag&drop becomes more challenging: if contact is lost, an accidental drop is performed, canceling the operation or even leading to unintended system actions such as moving files to an incorrect folder. Consequently, drag&drop was found to be more error-prone with increasing distance on touch-screen displays [18].

INTRODUCTION

Over the last few years, large interactive display walls as well as larger tabletop systems have gained attention in both research communities and the general public (especially the presentation of Jeff Han [10] at the TED1 conference in 2006 or commercially available systems like Microsoft Surface 1

Technology, Entertainment, Design

Permission to to make make digital digital or or hard hardcopies copiesofofall allororpart partofofthis this work Permission work forfor personal or or classroom classroom use use is is granted granted without without fee fee provided provided that that copies copies are are personal not made made or or distributed distributed for not for profit profit or or commercial commercialadvantage advantageand andthat thatcopies copies bear this or bear this notice notice and and the thefull fullcitation citationon onthe thefirst firstpage. page.ToTocopy copyotherwise, otherwise, republish, to post on servers or toorredistribute to lists, requires prior prior specific or republish, to post on servers to redistribute to lists, requires permission and/or a and/or fee. a fee. specific permission CHI 2009, 2010, April 4October - 9, 2009, Boston, Massachusetts, USA. NordiCHI 16–20, 2010, Reykjavik, Iceland. Copyright 2009 2010 ACM 978-1-60558-246-7/09/04... ISBN: 978-1-60558-934-3...$5.00. $5.00.

158

While state-of-the art approaches such as drag-and-pop [3] or push-and-pop [7] already offer support for drag&drop interactions with simple targets, they fall short when taking more complex ones into account, i.e. drop targets that may require state modifications prior to the actual drop operation: 2 strictly speaking, for some camera-based systems very close proximity is sufficient

Full Papers

• Simplify drag&drop interactions with distant targets.

minimal hardware reqs.

minimal overhead

minimal interference

To address this problem, we designed drop-and-drag with three major goals in mind:

direct input techniques

Missile Mouse Tablecloth ScaleView Portals Push-and-throw Drag-and-pop Push-and-pop The Vacuum

complex targets

A common scenario involving complex drop targets is copying a file from one file tree to another using drag&drop. In order to complete the operation, one must frequently modify the state of the target file tree, i.e. collapse or expand branches, to access the desired drop location. This has to be done either prior to or during the actual drag&drop operation. While the former is time consuming for remote targets on large screens, the latter is often burdensome, requiring the user to hover above a tree node for a given time in order to expand it. Furthermore, there may even be components such as digital maps which do not offer the possibility to modify their state during a drag&drop operation, at all.

easier interaction

Proceedings: NordiCHI 2010, October 16–20, 2010

+ + + + + +

o + + o +

o + + + + +

n/r + + + + + -

+ + o + o

+ o o o o -

Table 1. Review of state of the art techniques for drag&drop on large displays, according to the six categories introduced at the beginning of the related work section (abbreviated for this table). A rating of ’+’ has been given where the technique fits the requirement very well; ’o’ indicates an average fit, e.g. there is no active support, but also no obstacle to the requirement; ’-’ is used, when the requirement is not met. ’n/r’ (not rated) has been used for Missile Mouse and its requirements on hardware because support for direct input is missing there.

• Support drop targets that require complex interactions prior to the actual drop operation. • Design for minimal interference in multi-user settings. The remainder of this paper is structured as follows: First, we present related approaches and state-of-the-art techniques for drag&drop on large displays. Then, we elaborate on the design of our approach, following a discussion of the setup and results of a controlled experiment, which we conducted to analyze the performance and user satisfaction of drop-and-drag. Finally, we argue about the shortcomings and limitations of our approach, present our conclusions and propose further enhancements to our interaction technique.

Techniques facilitating general access to remote objects

Missile Mouse, devised by Robertson et al. [19], is a technique which is targeted at very large displays - with, however, mouse input. Because its primary focus is accelerating the cursor with relative input devices, it does not lend itself very well to touch-operated devices. Tablecloth [19] is another approach by the same authors, which allows the user to scroll the entire screen content for access to distant areas. However, there are two major issues when this technique is used for drag&drop:

RELATED WORK

A number of techniques have been proposed to ease the problems, which accompany drag&drop on large displays. We separate these into techniques that are not geared towards drag&drop interactions specifically, but aim at allowing more efficient access to remote objects on large screens in general and those that focus on drag&drop. All techniques were reviewed according to the following criteria. Although some requirements may be more relevant to some scenarios than to others, an ideal interaction technique will meet all of them, allowing optimal flexibility:

• First, it is geared towards single-user multi-monitor setups and facilitates access to remote areas by scrolling the relevant part of the desktop to the central screen - this may cause conflicts in multi-user settings where different users may want to access different content. • Second, in its current form, the user may either drag the desktop to access remote content or execute a standard drag&drop operation, but not both a the same time. Thus the user is effectively required to use standard drag&drop.

• support for easier interaction with remote drop targets. • support for complex drop targets.

ScaleView portals is an approach presented by Bezerianos [5]. These user-created portals represent alternative views of any area of the screen. Not only the visuals of the original area are recreated, but all actions performed within the portal are also delegated to the remote area and vice versa. Objects can be passed to the remote area by dragging them into the portals. Thus, the portals reduce the distance in visual and motor space alike. However, as ScaleView portals are geared towards more persistent interaction with remote locations, the creation of these portals imposes significant overhead for ephemeral interactions like drag&drop. Thus, drop-and-drag takes up the idea of fully interactive portals to

• support for direct input techniques, i.e. pen- and touchinput. • minimal requirements on available hardware (ideally the technique should work on single-touch or single-pen systems with pressure or proximity detection). • minimal overhead to invoke the drag&drop operation. • minimal interference in multi-user settings. The results of the review can be found in Table 1.

159

Full Papers

Proceedings: NordiCHI 2010, October 16–20, 2010

remote locations, but it modifies this idea for the context of drag&drop interactions.

covers towards the origin of the arc, i.e. the current pen position - to fit into the influence area, targets are reduced in size accordingly. The Vacuum can be used for both selection and drag&drop, depending on whether the user invoked it on empty screen space or on a specific object. While it does allow for complex interactions with remote targets, drag&drop operations with such targets will require the user to trigger two interactions: The first on empty screen space to manipulate the target, the second to perform the actual drop operation - an overhead that may be especially challenging for users unfamiliar with pen-input systems. Furthermore, The Vacuum is designed to use proximity information, i.e. a hover state, which is not available for a larger number of multitouch systems (e.g. those based on frustrated total internal reflection). Also, its clearly highlighted arc of influence may cause interference in multi-user settings.

Techniques which specifically address the challenges of drag&drop interaction on large displays

Push-and-throw, an approach proposed by Hasco¨et and Collomb [11], enables users to throw the object instead of dragging it. To counter the imprecision of human motor skills, a technique called trajectory snapping is integrated into pushand-throw. Trajectory snapping allows for minor inaccuracies by snapping the throwing trajectory to the drop target, if the drop target is not located on the trajectory but rather in close proximity. This technique can be expected to work well for simple, free-standing drop targets. However, it is questionable whether it is still applicable when multiple drop targets are clustered in a small remote area or when preceding interactions like expanding branches within a file explorer are required.

THE DESIGN OF DROP-AND-DRAG

Based on the evaluation criteria introduced in the related work section, we derived the following design decisions for drop-and-drag (respective evaluation criterion in brackets):

Baudisch et al.[3] describe a drag&drop extension called drag-and-pop which takes the opposite approach to pushand-throw. Upon the invocation of a drag-and-drop operation, so-called tip icons, which represent the original drop target, are shown around the drag source. The connection to the original target is made via a virtual “rubber band”. In order to avoid confusion caused by displaying too many tip icons, the set of targets is reduced to those which accept the data of the drag source and which are located in the direction of the initial drag movement. However, as the authors note, “the most common problem with drag-and-pop was getting the right group of targets to pop up”. Furthermore, the iconified versions do not allow complex interactions or precise positioning on the respective drop target.

• Easier interaction with remote targets will be facilitated via proxy targets, i.e. visually and functionally identical copies, of all compatible drop targets projected into the area the user can immediately reach3 . To support quick orientation, an algorithm that preserves the relative spatial layout of the original drop targets will be used to place the proxy targets in the area of immediate reach. (support for easier interaction with remote drop targets). • Proxy targets will be fully interactive and equal in size to the original. To allow for a maximum number of simultaneously displayed proxy targets at the same time, these targets consist only of the user interface components on which a drop can actually take place (e.g. only the file tree from an explorer window). Furthermore, the technique introduces a suspend mode for drag&drop operations to facilitate interaction with complex drop targets without burdening the user to maintain contact with the screen or additional hardware requirements4 (support for complex drop targets).

Based on push-and-throw and drag-and-pop, Collomb et al. propose a drag-and-drop technique push-and-pop [7]. When starting a drag motion, a take-off area is created, surrounding the current cursor position. This take-off area is still a miniature version of the screen as it is in push-and-throw. However, it contains tip-icons of all targets accepting the dragged object, thus eliminating the risk of invoking the wrong set of targets. Additionally, the tip-icons are positioned within the take-off area relative to their locations on the screen. As a result the layout of the take-off area is always the same for a given drag source, independent of its location or the direction of the initial drag motion. This enables the user to perform drag-and-drop actions based on muscle memory, i.e. the user memorizes the particular movement routine due to frequent repetition. Similar to push-and-throw and dragand-pop, push-and-pop does not allow for complex interactions with the available drop targets. However, the idea of projecting the relevant drop targets into the vicinity of the drag source while preserving the proportions of the original screen layout, is taken up in our approach.

• No assumptions will be made on whether there is pen or touch input (support for direct input techniques, i.e. penand touch-input). • Only single taps (clicks) will be necessary to operate the technique (minimal requirements on available hardware). • The technique will be invoked automatically once the user drags an object (minimal overhead to invoke the drag&drop operation). 3 To identify possible drop targets, hard-wired solutions (i.e. mapping drop targets and drag sources at compile time) are possible (and in fact this approach has been used for the implemented prototype). However, there are also more flexible solutions based on the semantic annotation of user interface components [17] 4 (contrary to Kobayashi and Igarashi, who use a throwing metaphor [14], the pause is triggered via a simple drop on empty screen space in drop-and-drag)

Another approach, based on drag-and-pop, is The Vacuum, developed by Bezerianos and Balakrishnan [4] and geared towards pen-input. This technique addresses the problem of getting the right group of targets to pop up by introducing a flexible arc which draws all the objects in the area it

160

Full Papers

Proceedings: NordiCHI 2010, October 16–20, 2010

• No visual connections to the original targets are used (minimal interference in multi-user settings).

and the actual target, allowing the user to look for it in the original direction.

Thus, drop-and-drag enables the user to conveniently interact with distal targets, reducing the source-target distance in visual and motor space alike, while preserving the full set of possible interactions with these targets. Proxy Features

Created by cloning the visuals of the drop target, a proxy represents an alternative view of the drop target, similar to a ScaleView portal[5] with the drop target as its remote area. Additionally, all actions performed within the proxy are delegated to the actual drop target. If the visuals of the drop target change due to such a delegated action, the visuals of the proxy are updated as well. Thus, a proxy enables a user to access all functionality provided by the actual drop target. This includes common actions as clicking buttons, expanding and collapsing of trees or invoking menus, as well as more rare actions such as panning a map component to the desired location. Once the user starts to drag an object, his/her options to complete the drag&drop operation using the appearing proxies are twofold: • He/she can drop the dragged object on one of the proxies directly (given the target does not require any additional interaction), • or he/she can pause his/her interaction by temporarily dropping the dragged object next to the proxies. All proxies will remain visible and a resume button on each of them will then be enabled. Then, the user can perform the necessary changes to the state of the target object (without the burden of maintaining the drag&drop operation) and continue his/her drag&drop interaction afterwards. This is done via the resume button which causes the next click to be interpreted as a drop. As the user might want to nullify the paused operation instead of completing it, another button is added to allow the user to cancel the whole interaction, causing all proxies to disappear. An exemplary proxy target, using a file tree is shown in the steps 2 to 6 of the walkthrough in Figure 2. Proxy Placement

After a dragging motion is started and all drop targets accepting the currently dragged object are determined, the locations of the corresponding proxies need to be computed. The layout algorithm that we employ for this purpose was designed with three requirements in mind:

Figure 1. a) The initial version of the proxy positioning algorithm proxy targets are projected nearer to the user, regardless of the original target’s position. b) The refined version of the algorithm places proxy targets directly over the original targets if they are located within the reach of the user’s flexed arm. All other targets are projected into the area between the user’s flexed and stretched arm’s radius. This compensates for user’s moving the dragged objects too fast and overshooting close-by proxy targets.

We implemented these requirements through an algorithm that projects the location of a drop target into the screen space the user can currently reach, i.e. the screen space defined by a circle with the user’s arm length as its radius and the origin of the drag&drop operation as its center. Our first version of this algorithm handled all targets the same way: A proxy is positioned such that d/dmax = d /dmax holds true as illustrated in Figure 1(a). (rstretched denotes the distance defined by the user’s stretched arm; dmax is the maximum distance from the drag source location to the edge of the screen; dmax denotes the maximum projected length and is therefore equal to rstretched . d is the distance between the drag source and the original (potential) drop target P . d is the distance of the new proxy target position P  , calculated by projecting the ratio d/dmax onto dmax .) However, preliminary tests revealed issues with this implementation. Users often expected proxies to appear with a reasonable distance to the origin of the drag&drop operation. When dragging very fast, they overshot the proxy before it became visible and then failed to notice it immediately, as they focused on the original drop target. Furthermore, users frequently dragged over a short distance before dropping on empty space. This allowed them to take full advantage of drop-and-drag, i.e. they could interact with the proxy without being forced to keep on dragging the object. Nearby

1. A proxy should always be within the user’s immediate reach. 2. Spatial arrangement, alignment and proximity between the actual drop targets should be preserved. 3. In order to reduce the need for reorientation, the proxy should be placed on the line of sight between the source

161

Full Papers

Proceedings: NordiCHI 2010, October 16–20, 2010

proxies often interfered with this strategy by covering the relevant screen space. To avoid these issues, we modified the algorithm by distinguishing between two kind of drop targets: those within the reach of the flexed arm and those beyond this distance. If a target is located within the reach of the user’s flexed arm, its proxy is positioned right above it. All other drop targets are projected into the ring between the length of the user’s stretched arm (rstretched ) and the length of his/her flexed arm (rf lexed ), as illustrated in Figure 1(b). (dmax is the maximum distance from the drag source location to the edge of the screen minus rstretched , d is the distance between the drag source and the original drop target P minus rstretched , P  is the location of the projected drop target proxy. d is the distance of the new proxy target position P  , calculated by projecting the ratio d/dmax onto dmax , which denotes the difference between the user’s stretched and flexed arm’s radius.) This minimizes the amount of proxies located within very close proximity. As a result it is very unlikely that the user overshoots a proxy or that close-by proxies interfere with the intention to pause the drag&drop operation. At the same time it is still assured that every proxy is within immediate reach. Our current algorithm can be applied to an arbitrary number of drop targets simultaneously. For a high number of large potential targets, this may lead to occlusion problems. This and other possible limitations of drop-and-drag are discussed later in the next section. Exemplary drop-and-drag interaction

To illustrate the features offered by drop-and-drag, Figure 2 shows an exemplary sequence of actions, using drop-anddrag to move a folder from a newly attached portable flash drive to a folder currently not visible in the file explorer view of the user’s computer: 1. The file explorer showing the user’s computer is located in the upper left of the screen - the flash driver’s file explorer has just opened in the lower right corner. 2. The user initiates a dragging gesture on the folder to copy the file - a drop target proxy, which shows the file tree part of the “computer-file explorer”, is displayed close to the starting point of the dragging gesture. 3. As the the node/folder on which the user wants to drop the dragged folder, he/she pauses the drag&drop operation by dropping the dragged object on empty screen space. 4. Now he/she can conveniently interact with the file tree displayed by the proxy and expand the corresponding node. 5. By pressing the resume button at the lower left edge of the proxy, the user indicates that he/she wants to actually drop the dragged object. 6. Subsequently, he/she selects the now visible drop location. As a result, the dragged folder is copied to the folder

162

Figure 2. An exemplary drop-and-drag operation where pictures are copied from a flash drive to a sub-folder of “My Pictures”.

Full Papers

Proceedings: NordiCHI 2010, October 16–20, 2010

within the file tree and the desktop returns to its initial state. 7. However, the state of the original file tree branch is now expanded as the state changes were forwarded by the drop target proxy. EVALUATING DROP-AND-DRAG

To assess the general performance of drop-and-drag and to investigate possibilities for future improvements, we conducted a controlled experiment comparing drop-and-drag to traditional drag&drop (later, in the discussion section, we elaborate on the advantages and limitations of this choice). Participants of this study had to perform drag&drop operations using both techniques in two different scenarios involving complex and simple drop targets. We had five hypotheses:

Figure 3. If the finger is not approaching the display surface perpendicularly, an unintended cursor position may occur.

issues. They were advised to touch the display perpendicularly if possible. Participants

Twenty-eight researchers and students from our lab volunteered to participate in our experiment. Five of them were female, twenty-three were male. Their ages were between twenty-three and forty. All but three participants were right handed. However, two of the three left-handed participants stated that they use the mouse with their right hand and were therefore treated as right-handed during the course of the experiment. None of the participants had any prior experience using drop-and-drag. They rated their experience in using touchscreen devices from 2 to 4 on a 5 point scale (1 no experience at all, 2 below average, 3 average, 4 above average, 5 extensive experience) with a mean value of 2.6.

H1 With increasing distances, drop-and-drag becomes significantly faster than traditional drag&drop. H2 The effect of H1 is more distinct in case of complex drop targets. H3 The average error rate of drop-and-drag does not exceed the error rate of traditional drag&drop. An operation is considered erroneous, if it is aborted or if the dragged object is dropped on a target, that was not in the correct state. H4 For more distant targets drop-and-drag will be less errorprone than traditional drag&drop. H5 drop-and-drag will exhibit a better user acceptance on large touchscreen devices.

Design

A within subject design with repeated measures and three independent variables was used. These variables were technique (traditional drag&drop, drop-and-drag), target distance (very close, close, medium, far) and target complexity (simple, complex). The target distance was not defined by static values, but rather in relation to the participants’ arm length. ’Very close distance’ targets were positioned at a distance of approximately 0.5 times the participant’s arm length, ’close distance’ targets at 0.9 times the arm length, ’medium distance’ targets at 1.6 the arm length and ’far’ targets at 2.5 the arm length. In case of ’simple’ drop targets, the dragged object could be dropped without any prerequisites. ’Complex’ targets, however, consisted of a panel containing two tabs. In order to perform a successful drop, the user was required to switch to the second tab (cf. Figure 4). In case of the traditional drag&drop technique, this forces a participant to reach for the target and select the second tab prior to the start of the actual drag interaction. By contrast, drop-anddrag enables the user to start the drag, perform a temporary drop anywhere on the screen, switch the tab on the displayed proxy and complete the operation via the resume button and a subsequent click on the actual target panel. Simple drop targets were 225 pixels wide and 210 pixels high. Complex drop targets were 24 pixels higher to accommodate for the additional tabs necessary to provide the tabbed pane. Thus, the area relevant to the drag&drop operation, i.e. the actual drop target, retained its size. Consequently, the size relevant to the index of difficulty according to Fitts’ law remains constant throughout the experiment as well. In summary the ex-

Apparatus

The user study was performed on a Barco[2] display wall with a screen width of 300 cm and a height of 150 cm. The screen was placed 87 cm above the ground and was operated at a resolution of 4200 x 2100 pixels. Single touch input was provided by an array of infrared sensors, fabricated by SMART technologies [21]. Inherent to this technique for recognizing touch input two side effects affected our experiment: First, a “touch” can be registered without physical contact to the screen. This sometimes confuses users who are unfamiliar with the device, as unintentional clicks are performed. Second, the accuracy of the touch input depends on the angle between the screen surface and the user’s finger. The flatter the angle, the more likely other body parts, e.g. other fingers, the wrist or the elbow, are registered by the infrared sensors as well, resulting in a cursor position that does not correspond with the one intended by the user (cf. Figure 3). This problem most often occurs when acquiring distant targets, as the user tries to maximize his/her reach by aligning his/her finger with his/her arm. As a consequence, this behaviour frequently leads to very imprecise and sometimes unintentional touch input. In order to avoid errors that are a mere result of this touchrecognition technique, participants were made aware of these

163

Full Papers

Proceedings: NordiCHI 2010, October 16–20, 2010

periment consists of 1792 samples (28 participants x 2 techniques x 2 target complexities x 4 distances x 4 repetitions).

Figure 5. A schematic screen layout for a right handed person. Each set of targets is depicted by a different symbol: target 1-4 by a square, target 5-8 by a pentagon, target 9-12 by a star, and target 13-16 by a triangle. The reach of the participant’s flexed and stretched arm is depicted by the two dashed circles.

Figure 4. The two states of a complex drop target: In the left part of the figure, the target does not accept any drops. The right side shows the target, where the correct tab is selected and drops are accepted.

Test Procedure

To avoid the impact of exhaustion or learning effects on the results, the order of the test runs was permuted over the experiment. Additionally, the single operations during each of the test runs were performed in random order.

Before performing any drag-and-drop operations, the participants were required to configure some user specific parameters before proceeding to the actual testing phase. Those parameters were: • The height at which the participant could comfortably touch the screen while standing in front of it, • the preferred hand, and • the length of both the flexed as well as the stretched arm.

Results

To test our hypotheses, we analyzed task completion times, error rates and user ratings for the two experiment scenarios. Additionally we collected qualitative feedback during and after the test runs that gave hints to possible improvements of drop-and-drag. Task Completion Time

After completing the configuration phase, the screen layout was adapted accordingly. If the participant was right (left) handed, the source of all drag-and-drop operations was placed at the left (right) edge of the screen. This was done to generate sufficiently distant targets, but should not affect validity - for more realistic scenarios, the placement algorithm will adapt to the possibly range in the other direction. On the basis of the length of the participant’s flexed and stretched arm, four sets of four targets each were generated (cf. Figure 5), one set of targets for each distance (’very close distance’, ’close distance’, ’medium distance’ and ’far distance’). Consequently, only eight targets were within the user’s immediate reach. The remaining targets required the user to walk towards them, as they had a distance of 1.6 to 2.5 times the user’s stretched arm length to the drag source.

Two-way repeated measures ANOVA tests revealed that distance significantly affected task completion times independently of technique and in both scenarios (simple targets: F (3, 216) = 34.00, p  0.01; complex targets: F (3, 216) = 34, 01, p  0.01). While the technique alone did not significantly affect the task completion times (simple targets: F (3, 216) = 0.1, p = 0.76; complex targets: F (3, 216) = 3.02, p = 0.08), a strong technique × distance interaction (simple targets: F (3, 216) = 6.67, p < 0.01; complex targets: F (3, 216) = 17.25, p  0.01) indicates that dropand-drag and traditional drag&drop are affected differently by changes in distance. Distance very close

Users were asked to accustom themselves with the screen layout and the different set-ups for about five minutes in order to avoid learning effects during the experiment affecting the results. Subsequently, four test runs, i.e. every combination of both drag-and-drop techniques and complex or simple drop targets, were conducted for each participant. Every test run consisted of at least sixteen drag-and-drop operations, one for each target. In case of an error, i.e. the dragged object was not dropped on the drop target, the operation had to be repeated. For each step in each test run there was only a single valid drop target. Furthermore, the participants were asked to look for the target first, i.e. before they initiated the logged drag&drop operation by activating the source with a button click.

close medium far

Simple Targets F (1, 54) = 15.88 p < 0.01 F (1, 54) = 4.80 p = 0.03 F (1, 54) = 0.01 p = 0.91 F (1, 54) = 5.73 p = 0.02

Complex Targets F (1, 54) = 29.58 p  0.01 F (1, 54) = 15.98 p = 0.01 F (1, 54) = 0.07 p = 0.79 F (1, 54) = 13.94 p  0.01

Table 2. Results of one-way repeated measures ANOVA tests. The nominal factor was technique (traditional drop&drag, drop-and-drag). The results show significant differences between techniques for ’very close’, ’close’ and ’far’ distances.

The results of additional one-way repeated measures ANOVA tests presented in Table 2 support this: there was a significant

164

Full Papers

Proceedings: NordiCHI 2010, October 16–20, 2010

difference between techniques for targets at ’very close distance’, ’close distance’ and ’far distance’. Only for targets at ’medium distance’ no significant difference was found (this can be attributed to the fact that drop-and-drag starts to outperform traditional drag&drop around this distance). These findings are reinforced by an analysis of the task completion times of all participants. As illustrated in Figures 6 and 7, traditional drag&drop remains faster than drop-anddrag for targets in the user’s immediate reach; this is the case for targets at ’very close distance’ and ’close distance’.

Figure 8. Relative time savings of drop-and-drag versus target distances in relation to arm length.

to be observed more clearly when drop targets require additional interaction to successfully complete the operation. Although drop-and-drag begins to be faster at approximately the same distance, regardless of target complexity, Figure 8 shows that the time savings when using drop-and-drag with complex targets exceed those for simple targets. We therefore consider our hypothesis H2 confirmed. Figure 6. Task completion times of test runs with simple targets versus target distances in relation to arm length.

Error Rates

Although error rates were expected to rise with increasing distance, no such correlation was observed. Overall error rates were relatively low. For simple targets, drop-and-drag performed better than traditional drag&drop; for complex targets, traditional drag&drop was slightly less error-prone (cf. Table 3). However, paired t-tests showed that both differences are not significant, due to a very high variance within our data coupled with a relative low overall error rate. The most frequent error when using drop-and-drag with complex targets was related to resuming a paused drag&drop operation. It seemed that despite the time the participants were given to familiarize themselves with drop-and-drag, some of them did not acquaint themselves with the concept of pausing and resuming an operation before completing some of the timed tasks. Summing up, H4 is rejected by our results, H3 is confirmed for the overall error rate, but the results for complex targets indicate room for improvement.

Figure 7. Task completion times of test runs with complex targets versus target distances in relation to arm length.

At a distance of approximately 1.6 times the user’s arm length, i.e. the distance of the ’medium distance’ targets, both techniques perform equally fast. When exceeding a distance of about 2 times the arm’s length, which was the case for the ’far distance’ targets, drop-and-drag clearly outperforms traditional drag&drop. For these targets drop-and-drag was on average 32% faster. Overall, these findings support the hypothesis that drop-and-drag becomes significantly faster than traditional drag&drop with increasing distances (H1). Furthermore, we hypothesized that this effect is supposed

165

Targets Simple Complex Overall

Drag&drop 8.57% 10.62% 9.60%

drop-and-drag 3.58% 12.71% 8.15%

Table 3. Mean error rates

Subjective Satisfaction

After completing all four test runs, the participants answered a short questionnaire in order to capture their subjective satisfaction concerning traditional drag&drop and drop-and-drag. Participants were asked whether the task was easy to fulfill using the different techniques and whether the respective

Full Papers

Proceedings: NordiCHI 2010, October 16–20, 2010

techniques provided reasonable support - their ratings were captured on a five point Likert scale (“1 being “full disagreement” and 5 “full agreement”).

drop targets in push-and-pop[7], we plan to re-implement this technique for future evaluations as, besides this, it meets most of our other requirements for ideal drag&drop techniques for large touchscreens very well.

While both techniques were perceived equally difficult in case of simple drop targets (the average rating was 3.4 for traditional drag&drop and 3.8 for drop-and-drag), drop-anddrag outperformed traditional drag&drop when operating on complex drop targets (average rating of 3.5 versus 2.4). Furthermore, only four of the 28 participants would still prefer traditional drag&drop when interacting with a very large touch-screen. This supports our expectation that drop-anddrag exhibits a better user acceptance due to the benefits it offers (H5). Further evaluation will however be necessary to test whether this hypothesis also holds against other supporting techniques for drag&drop (see next section).

CONCLUSION & OUTLOOK

When designing drop-and-drag, we aimed for a technique which makes drag&drop operations on large touchscreen displays both faster and easier, especially when more complex state manipulations are required. The findings of the controlled experiment show that this goal was met. Although traditional drag&drop exhibits faster performance when for targets within immediate reach, the ability of dropand-drag to efficiently bridge large distances, even when additional interaction complicates the drag&drop operation, is apparent (time-savings of up to 32% compared to traditional drag&drop). Furthermore, user satisfaction was improved notably: During their sessions multiple participants stated they felt more comfortable with drag&drop operations that use drop-and-drag rather than traditional drag&drop.

DISCUSSION

While the the results of our experiment show that drop-anddrag performs well, especially for targets in medium (1.5 × user’s arm length) and far (2.5 × user’s arm length) distance, there is also a number of limitations in both the evaluation and our current design itself.

Yet, there is room for improvement: Qualitative feedback from some study participants indicated a number of refinements of our technique. Two participants suggested that employing animated transitions when creating the drop target proxies could improve the usefulness of drop-and-drag. Adding a rubber band as in push-and-pop would also be an option, but was not proposed by participants. Further on, additional research will be necessary to determine whether these visual cues cause interference in multi-user settings.

Proxy size and placement

As noted when elaborating on our design in design section, our current design will work for an arbitrary number of potential drop targets, where each target results in a proxy in equal size. For a reasonably large number of targets this may lead to occlusion problems: Important information could be hidden from the user and he/she might be unable to access essential interaction elements such as buttons or menus. However, an approach that scales down the proxy targets size (like The Vacuum [4]) may also be challenging, especially when touch input is used and thus precision is limited by the human finger. More refined techniques may include fisheye view [9], but additional investigation should clarify the typical number of targets involved in drag&drop operations, first. Also, an analysis of the context users need to identify the proxy target (e.g. the surrounding explorer window for a file tree) could help to determine the necessary size.

More controversial was the way the drag operation is resumed. While one participant explicitly stated that he liked the possibility to complete the operation with two clicks, two others stated that they would prefer to resume the operation by another drag interaction, which would originate from a scaled-down version of the original source, placed at the current position of the resume button. The latter suggestions might also offer an improvement to the unexpectedly long time it took for some participants to get accustomed to pausing and resuming drag&drop operations.

Visual connections and interference

We deliberately decided not to include any visual connections between proxy targets and original targets, as we expected them to cause visual interference in multi-user settings. However, we noticed during our evaluation that some participants would reorient during the drag&drop operation, even though proxy targets were placed on the line of sight between drag source and original target. Thus further investigation on the interference caused by visual connections is necessary for multi-user settings. It may be reasonable to compromise and introduce lightweight connection visuals like the rubber-bands of push-and-pop[7].

Furthermore, as pointed out during the discussion of the limitations of drop-and-drag, future work will need to include refinements to the the current positioning algorithm which may have occlusion problems for a larger number of targets. Finally, while drop-and-drag did outperform the traditional technique, a more extensive comparison will also need to include state-of-the-art approaches like push-and-pop[7], at least for the case of simple drop targets. ACKNOWLEDGMENTS

We would like to thank all those participating in our experiments for their time and effort. Also we would like to thank both our colleagues and the reviewers for giving valuable advice on how to improve this paper and the presented interaction technique.

Evaluation against standard drag&drop

While the evaluation of our technique against standard drag&drop allowed a basic assessment of its performance, it allows comparison to state of the art techniques only to a very limited degree. Even though there is no support for complex

166

Full Papers

Proceedings: NordiCHI 2010, October 16–20, 2010

The work presented in this paper has been partly funded by the German Federal Ministry of Education and Research.

12. K. Hawkey, M. Kellar, D. F. Reilly, T. Whalen, and K. M. Inkpen. The proximity factor: impact of distance on co-located collaboration. In GROUP ’05: Proceedings of the SIGGROUP conference on Supporting group work, pages 31–40. ACM, 2005.

REFERENCES

1. R. Ball and C. North. Effects of tiled high-resolution display on basic visualization and navigation tasks. In CHI ’05: Extended Abstracts of the SIGCHI conference on Human factors in computing systems, pages 1196–1199. ACM, 2005.

13. A. Kahn, J. Matejka, G. Fitzmaurice, G. Kurtenbach, N. Burtnyk, and B. Buxton. Toward the digital design studio: Large display explorations. Human-Computer Interaction, Volume 24(1 & 2):9–47, 2009.

2. BARCO. Barco control rooms. http://www.barco.com/controlrooms/. last accessed: 2010-05-11.

14. M. Kobayashi and T. Igarashi. Boomerang: suspendable drag-and-drop interactions based on a throw-and-catch metaphor. In UIST ’07: Proceedings of the ACM symposium on User interface software and technology, pages 187–190. ACM, 2007.

3. P. Baudisch, E. Cutrell, D. Robbins, M. Czerwinski, P. Tandler, P. T, B. Bederson, and A. Zierlinger. Drag-and-pop and drag-and-pick: techniques for accessing remote screen content on touch- and pen-operated systems. In INTERACT ’03: Proceedings of the IFIP TC13 Conference on Human-Computer Interaction, pages 57–64. Springer, 2003.

15. Microsoft. Microsoft surface product information. http://www.microsoft.com/surface/. last accessed: 2010-05-10. 16. C. M¨uller-Tomfelde and C. Schremmer. Touchers and mousers: commonalities and differences in co-located collaboration with multiple input devices. In CHI ’08: Proceedings of the SIGCHI conference on Human factors in computing systems, pages 1149–1152. ACM, 2008.

4. A. Bezerianos and R. Balakrishnan. The vacuum: facilitating the manipulation of distant objects. In CHI ’05: Proceedings of the SIGCHI conference on Human factors in computing systems, pages 361–370. ACM, 2005. 5. A. Bezerianos and R. Balakrishnan. View and space management on large displays. IEEE Computer Graphics and Applications, 25(4):34–43, 2005.

17. H. Paulheim. Ontology-based Modularization of User Interfaces. In EICS ’09: Proceedings of the SIGCHI Symposium on Engineering interactive computing systems, pages 23–28. ACM, 2009.

6. X. Bi and R. Balakrishnan. Comparing usage of a large high-resolution display to single or dual desktop displays for daily work. In CHI ’09: Proceedings of the SIGCHI conference on Human factors in computing systems, pages 1005–1014. ACM, 2009.

18. J. Rekimoto. Pick-and-drop: a direct manipulation technique for multiple computer environments. In UIST ’97: Proceedings of the ACM symposium on User interface software and technology, pages 31–39. ACM, 1997.

7. M. Collomb, M. Hasco¨et, P. Baudisch, and B. Lee. Improving drag-and-drop on wall-size displays. In GI ’05: Proceedings of the Graphics Interface conference, pages 25–32. Canadian Human-Computer Communications Society, 2005.

19. G. Robertson, M. Czerwinski, P. Baudisch, B. Meyers, D. Robbins, G. Smith, and D. Tan. The large-display user experience. IEEE Computer Graphics and Applications, 25(4):44–51, 2005. 20. D. M. Russell, J. P. Trimble, and A. Dieberger. The use patterns of large, interactive display surfaces: Case studies of media design and use for blueboard and merboard. In HICSS ’04: Proceedings of the Hawaii International Conference on System Sciences, 2004.

8. M. Czerwinski, G. Smith, T. Regan, B. Meyers, G. G. Robertson, and G. Starkweather. Toward characterizing the productivity benefits of very large displays. In INTERACT ’03: Proceedings of the IFIP TC13 Conference on Human-Computer Interaction, pages 9–16. Springer, 2003.

21. SMART Technologies. Smart technologies. http://www.smarttech.com/. last accessed: 2010-05-10.

9. G. W. Furnas. Generalized fisheye views. In CHI ’86: Proceedings of the SIGCHI conference on Human factors in computing systems, pages 16–23. ACM, 1986.

22. D. S. Tan, D. Gergle, P. Scupelli, and R. Pausch. With similar visual angles, larger displays improve spatial performance. In CHI ’03: Proceedings of the SIGCHI conference on Human factors in computing systems, pages 217–224. ACM, 2003.

10. J. Y. Han. Low-cost multi-touch sensing through frustrated total internal reflection. In UIST ’05: Proceedings of the ACM symposium on User interface software and technology, pages 115–118. ACM, 2005.

23. T. Tsandilas and R. Balakrishnan. An evaluation of techniques for reducing spatial interference in single display groupware. In ECSCW’05: Proceedings of the European Conference on Computer Supported Cooperative Work, pages 225–245. Springer, 2005.

11. M. Hasco¨et. Throwing models for large displays. In HCI ’03: Proceedings of the British HCI Group Annual Conference, pages 73–77, 2003.

167

Full Papers

Proceedings: NordiCHI 2010, October 16–20, 2010

Extending Boundaries with Meta-Design and Cultures of Participation Gerhard Fischer Center for LifeLong Learning and Design (L3D) University of Colorado Boulder, CO 80309-0430 USA [email protected] ABSTRACT

INTRODUCTION

Human-computer interaction (HCI) has refocused many research efforts within computer science from a technology-centered view to a human-centered view. The developments so far, however, have seen humans mostly as users and consumers rather than as active contributors and designers.

In the last 20 years, research and development in humancomputer interaction (HCI) have made major contributions to support the productive and creative autonomy of individuals. Historically, however, these possibilities often have been of interest and accessible only to a small number of “high-tech scribes.” Meta-design [21] is focused on the challenge of allowing users of software systems who are not primarily interested in software per se to modify, extend, evolve, and create systems that fit their needs.

This paper provides a conceptual framework based on meta-design and cultures of participation that democratize design and allows all stakeholders to evolve systems to fit their needs. It establishes a new extended discourse for HCI research in which information technologies are interwoven with human lives in all aspects of our existence (at home, working, teaching, learning, and being a citizen). Specific socio-technical environments instantiating the framework in different application domains are described, including: (1) environments for people with cognitive disabilities, (2) table-top computing systems for framing and solving complex urban planning problems, (3) modeling the buildings of the world in 3D, and (4) using Smart Grids to support energy sustainability. These examples show how metadesign and cultures of participation are design approaches that allow researchers and practitioners to extend boundaries by allowing all stakeholders to have more control over their artifacts and by providing opportunities to encourage and to support contributions by many people in personally meaningful activities.

What the personal computer has done for the individual, the Internet has done for groups and communities. The first decade of Internet use was dominated by broadcast models and contributed little to change the existing strong separation of designers and users imposed by existing media. Cultures of participation [19], supported by meta-design and the participatory web [31], represent an evolving framework to exploit computational media in support of collaboration and communication. Providing all citizens with the means to become co-creators of new ideas, knowledge, and products in personally meaningful activities presents one of the most exciting innovations and transformations of digital media, with profound implications to extend the boundaries of HCI research. This paper identifies boundaries, defines conceptual frameworks (centered on meta-design and cultures of participation), and describes socio-technical environments [27] (in four different application domains) grounded in these frameworks. It discusses the implications of these research activities to extend the boundaries of HCI.

Keywords

meta-design, cultures of participation, systemic problems, boundaries, distances, motivation, control, socio-technical environments

BOUNDARIES

ACM Classification Keywords

A basic challenge insufficiently addressed by prior HCI research is that almost all of the significant problems of tomorrow will be systemic problems, which cannot be addressed by any one specialty. These problems require multi-disciplinary, multi-sector, and international collaborations, providing opportunities for knowledge workers to work in teams, communities, and organizations that encompass multiple ways of knowing and collaborating. Our research has focused specifically on complex, systemic design problems requiring cultural and epistemological pluralism to make all voices heard. As stakeholders who

H5.m. Information interfaces and presentation (e.g., HCI): Miscellaneous.

Permission to make digital or hard hard copies copies of of all all or or part part of ofthis thiswork workfor for personal or classroom classroom use use is is granted granted without withoutfee feeprovided providedthat thatcopies copiesare are not made or commercial advantage and and that copies madeorordistributed distributedforforprofit profit or commercial advantage that bear thisbear notice the full the first To copy copies thisand notice and citation the full on citation on page. the first page.otherwise, To copy or republish,ortorepublish, post on servers or on to redistribute requirestoprior otherwise, to post servers or to lists, redistribute lists, specific and/or a fee. and/or a fee. requires permission prior specific permission NordiCHI 2010, October 16–20, 16–20, 2010, 2010,Reykjavik, Reykjavik,Iceland. Iceland. Copyright 2010 ACM ISBN: ISBN: 978-1-60558 978-1-60558-934-3...$5.00.

168

Full Papers

Proceedings: NordiCHI 2010, October 16–20, 2010

are separated by spatial, temporal, conceptual, and technological distances [18] are brought together, boundaries can be overcome based on these distances. These distances will be briefly characterized as representing boundaries that need to be understood, respected, and extended.

These structures and skills often do not exist at the beginning but evolve incrementally and collaboratively. CoIs provide an example of the importance of combining voices from different communities.

Boundaries Based on Spatial Distance

The preceding subsections emphasized computer-mediated collaboration among humans to reduce the gaps created by spatial, temporal, and conceptual distances. In HCI systems, substantial information is embedded in computational artifacts.

Boundaries Based on Technological Distances

Bringing spatially distributed people together with the support of computer-mediated communication allows the prominent defining feature of a group of people interacting with each other to become shared concerns rather than a shared location. By allowing more people to be included, local knowledge can be exploited. These opportunities have been successfully employed by the open-source communities [34] as well as by social networks of people who have a shared concern (such as a family member with a disability [9]). Transcending the barrier of spatial distribution is of particular importance in locally sparse populations (e.g., as they exist among people with disabilities).

Unlike passive design materials, such as pen and paper, computational artifacts are able to interpret the work of designers and actively “talk back” to them [36]. However, barriers occur when the back talk is presented in a form that users cannot comprehend or when the back talk created by the design situation itself is insufficient, and additional mechanisms and information are needed. To increase the back talk of the situation, we have developed a series of critiquing systems [22] that monitor the actions of users as they work to achieve a common goal and inform the users of potential problems during this process.

Boundaries Based on Temporal Distance

Design processes often take place over many years, with initial design followed by extended periods of evolution and redesign. In this sense, design artifacts are not designed once and for all, but instead gradually change over long periods of time. For example, within most computer networks, when a new device or technology emerges, the infrastructure is incrementally enhanced and updated rather than redesigned completely from scratch. To be able to do this requires that new designers “collaborate” with the original designers. Long-term collaboration is crucial for the success of evolutionary development. This is hampered when individual designers are not informed about how the decisions they make interact with decisions that were made in the past from designers whose voices have been lost.

EXTENDING BOUNDARIES Supporting Users as Active Contributors with Meta-Design

In a world that is not predictable, improvisation, evolution, and innovation are more than a luxury—they are a necessity. The challenge of design is not a matter of avoiding the emergent, but rather of including it and making it an opportunity for more creative and more adequate solutions to problems. Meta-design [21] explores objectives, techniques, and processes to enable users to act as designers and active contributors, allowing them to create new knowledge rather than restricting them to the consumption of existing knowledge. The need for meta-design is founded on the observation that design requires open systems that users can modify and evolve [24]. Because problems cannot be completely anticipated at design time when the system is developed, users at use time will encounter mismatches between their problems and the support that a system provides. These mismatches will lead to breakdowns [22] that serve as potential sources for new insights, new knowledge, and new understanding. Meta-design advocates a shift in focus from finished products or complete solutions to conditions for users to resolve mismatches and repair breakdowns when they are discovered during use.

Boundaries Based on Conceptual Distances

To analyze the contribution of voices from different communities, two types of communities can be differentiated: communities of practice (CoPs) and communities of interest (CoIs). CoPs [46] consist of practitioners who work as a community in a certain domain undertaking similar work. Examples of CoPs are architects, urban planners, research groups, software developers, and end-users. CoPs gain their strength from shared knowledge and experience. However, they face the boundaries of domain-specific ontologies and tools, empowering the insiders but often creating barriers for outsiders and newcomers. CoIs [15,17,29] can be thought of as “communities-ofcommunities” brought together to solve a problem of common concern. Examples of CoIs are (1) a team of software designers, marketing specialists, psychologists, and programmers interested in software development; or (2) a group of citizens and experts interested in urban planning. Fundamental challenges facing CoIs are found in building a shared understanding, learning to communicate with and learning from others, and establishing a common ground.

Meta-design engages diverse contributors (individuals and communities) in designing and building their own tools and systems by democratizing design. Our future world will be substantially shaped by stakeholders who will design, build, and evolve their own devices—our goal is to inspire, shape, support, foster, and analyze these communities. Metadesign allows creative and unplanned opportunism by addressing one of the fundamental challenges of a knowledge

169

Full Papers

Proceedings: NordiCHI 2010, October 16–20, 2010

society: to create socio-technical environments in which all participants in collaborative design processes can express themselves and engage in personally meaningful activities.

Web 2.0 architectures) have emerged to support social computing. These developments are the foundations for a fundamental shift from consumer cultures (specialized in producing finished goods) to cultures of participation (in which all people can participate actively in personally meaningful activities). Cultures of participation provide foundations for this fundamental transformation by exploring and supporting new approaches for the design, adoption, appropriation, adaptation, evolution, and sharing of artifacts by all participating stakeholders. These cultures are not dictated by technology alone; they are the result of incremental shifts in human behavior and social organizations [6].

By allowing users to become co-designers, HCI needs to develop architectures, seeds, and features that provide all stakeholders with opportunities, tools, and social reward structures to extend and evolve systems to fit their needs. In current design practices, teams of developers function as experts observing use and designing solutions to fit existing work practices, typically with input and involvement from users at one or more iterative stages. There is one “official” future toward which design efforts are oriented and around which design decisions are made. There is an intentional focus on the immediate needs of the current users; the end result must be a complete artifact, leading to rigid design decisions made at design time. These techniques yield closed systems based on decisions made at design time, and stakeholders are incapable of modifying and evolving the system when new and unanticipated requirements arise during use.

A major objective of cultures of participation is to attract large number of contributors. A number of notable success models exist, including open source software, Wikipedia, Second Life, YouTube, and 3D Warehouse (our research is less focused on social networks that primarily serve communication purposes such as Twitter and Facebook). By breaking down the boundaries between producers and consumers, cultures of participation have created new opportunities and challenges.

Design practices based on meta-design are focused on systems facing unpredictable future developments and changes. Participatory design includes users in the initial design process, but these participants can represent only the immediate users and immediate needs. Meta-design applies design techniques that enhance the further development of systems at use time. Design power is shifted towards the users, allowing them to ultimately act as both designers and consumers of the system and allowing the system to be shaped through real-time use. Table 1 summarizes the different role distributions.

Breaking Down Strict Role Separations

New concepts and models establishing middle ground between consumers and producers are emerging, including the following: • Prosumers [44] are techno-sophisticated and comfortable with the technologies with which they grew up. They have little fear of modifying and evolving artifacts to their own requirements. They do not wait for someone else to anticipate their needs, and they can decide what is important for them. They participate in learning and discovery and engage in experimenting, exploring, building, tinkering, framing, solving, and reflecting.

Transcending Consumer Cultures with Cultures of Participation

The first decade of the World Wide Web predominantly enforced a clear separation between designers and consumers. New technological developments (such as participatory

Design Approach

Design Time

Use Time

professionally dominated design

users have no voice

users have to live with artifacts designed by others

participatory design

users are active participants; systems are designed as complete artifacts at design time

systems are designed with users’ input, but they cannot be evolved to serve unforeseen needs

users are active participants; systems are designed as seeds; design is focused on use and participation

users can act as designers and evolve the artifact to fit new needs

meta-design

• Professional amateurs [26] are innovative, committed, and networked amateurs working up to professional standards. They are new social hybrids, and their activities are not adequately captured by the traditional dichotomous definitions of work and leisure, professional and amateur, and consumption and production. These new classes of contributors form the foundation for social production and mass collaboration [6] by relying on the following facts: • A tiny percentage of a very large base is still a substantial number of people; • Beyond the large quantitative numbers of contributors exists a great diversity of interests and passions among users; • Although human beings often act for material rewards, they can also be motivated by social capital, reputation, connectedness, and the enjoyment derived from giving away things of value.

Table 1. The role of stakeholders in different design approaches

170

Full Papers

Proceedings: NordiCHI 2010, October 16–20, 2010

Intrinsically motivated contributors can be found in the Long Tail [3], where people from around the world engage in topics and activities about which they feel passionate.

participation provide conceptual frameworks for the design of socio-technical environments because they give users design power to modify and evolve the systems according to their needs.

Open Design Spaces and User-Generated Content

This section briefly describes four specific examples that extend boundaries with meta-design and cultures of participation:

Cultures of participation emphasize the “unfinished” and take into account that design problems have no stopping rule, need to remain open and fluid to accommodate ongoing change, and can be characterized as having “continuous beta” as a desirable attribute rather than a to-be-avoided detriment. Cultures of participation can move from guidelines, rules, and procedures to exceptions, negotiations, and work-arounds to complement and integrate existing accredited and expert knowledge with informal, practice-based, and situated knowledge [42,47].

• the Memory Aiding Prompting System (MAPS), which fosters communities and creates new support tools for people with cognitive abilities and their caregivers; • the Envisionment and Discovery Collaboratory (EDC), which is a table-top computing environment supporting stakeholders from diverse backgrounds in face-to-face meetings;

User-generated content includes: (a) creating artifacts with existing tools (e.g., writing a document with a word processor) or (b) changing the tools (e.g., writing macros to extend the word processor as a tool). In specific environments, such as open-source software, the content is subject to the additional requirement of being computationally interpretable. Different activities for content generation (e.g., adaptation, generalization, improvement requests, specialization, and tailoring [2]) need to be supported.

• the three-dimensional (3D) modeling environment, which allows people from around the world to create and share 3D models; and • Smart-Grid developments, which involve consumers actively as decision makers to support new approaches toward energy sustainability. Cognitive Lever (CLever): Helping People Help Themselves

The CLever project has been a large-scale research project at the University of Colorado [9]. The project extended boundaries by complementing human minds through media and technologies. As an important development within CLever, the Memory Aiding Prompting System [8] is a meta-design environment in which caregivers can design task-support tools for people with limited memory and executive functions. Individuals with cognitive disabilities are often unable to live independently due to their inability to perform activities of daily living, such as cooking, housework, or shopping. By being provided with socio-technical environments to extend their abilities and thereby their independence, these individuals can lead lives less dependent on others. MAPS has explored meta-design by supporting mobile device customization, personalization, and configuration by caregivers and effective use by clients [10].

Rich Ecologies of Participation

Complex socio-technical environments cannot be understood as simple aggregations of the behavior of some nonexistent average user [37]. Stakeholders need to take on different tasks and responsibilities as they progress toward the demanding levels of participation. Most stakeholders will start as consumers, but only a small percentage will eventually become active contributors, curators, and metadesigners [33]. Creating innovative support mechanisms (e.g., learning environments, scaffolding, boundary objects) and social reward structures (e.g., intrinsic motivation, reputation economies, gift cultures) presents challenges for HCI research to reduce the funnel effect causing a limited migration toward more demanding roles [32]. SOCIO-TECHNICAL ENVIRONMENTS EXTENDING BOUNDARIES IN SPECIFIC APPLICATION DOMAINS

Meta-design is of critical importance for people with cognitive disabilities because they represent a “universe of one”: a solution for one person will rarely work for another. The success of MAPS is based on the empirical findings that (a) unexpected islands of abilities exist (clients can have unexpected skills and abilities that can be leveraged to ensure a better possibility of task accomplishment); and (b) unexpected deficits of abilities exist. Accessing and addressing these variations in skills and needs, particularly with respect to creating task support, requires an intimate knowledge of the client that only caregivers can provide. Currently, a substantial portion of all assistive technology is abandoned, resulting in the very population that could most benefit from technology paying for expensive devices that end up in the back of closets after a short time.

In the last decade, we have developed socio-technical environments to extend boundaries. In all of these application domains, socio-technical environments are needed because the deep and enduring changes of our ages are not technological in their core substance, but social and cultural. Changes in complex environments are not primarily determined by technology, but are the result of incremental shifts in human behavior and social organization. Sociotechnical environments are composed both of computers, networks, and software, and of people, processes, policies, laws, and institutions, thereby creating a complex web of socio-cultural concerns and requiring the co-design of social and technical systems. Meta-design and cultures of

171

Full Papers

Proceedings: NordiCHI 2010, October 16–20, 2010

framing and resolving complex urban planning by bringing together participants from various backgrounds in face-toface meetings. In this platform, the knowledge to understand, frame, and solve such problems does not already exist [16], but is constructed and evolves during the solution process—an ideal environment to study meta-design and cultures of participation. The EDC represents a socio-technical environment that incorporates a number of innovative HCI technologies and approaches, including (a) table-top computing, which fosters and invites participation by maximizing the richness of communication among stakeholders in face-to-face interaction; (b) the integration of physical and computational components, which supports new interaction techniques [14]; and (c) an open architecture, which supports metadesign activities.

Figure 1 Meta-design in MAPS: Empowering caregivers to act as designers

A unique challenge of meta-design in the domain of cognitive disabilities is that the clients themselves cannot act as designers, but the caregivers must accept this role (see Figure 1). Caregivers, who have the most intimate knowledge of the client, need to be empowered to become the end-user designers. The scripts needed to effectively support users are specific for particular tasks, creating the requirement that the people who know about the clients and the tasks (i.e., the local caregivers rather than a technologist far removed from the action) must be able to develop scripts.

Figure 2 shows three different but integrated components of the EDC: • the action space (bottom pane), in which stakeholders engage in participatory problem solving and decision making related to urban planning issues that are of concern to all participants; • the reflection space (top left pane), in which task-relevant information is displayed for the design created in the action space;

Caregivers generally have no specific professional technology training, nor are they interested in becoming computer programmers. Based on extensive end-user support, MAPS allows caregivers to design complex multimodal prompting sequences (including sound, pictures, and video to be assembled by using a film-strip-based scripting metaphor).

• embedded visualizations using Google Earth (top right pane), in which the impact of new buildings are shown. The vision of the EDC is to provide contextualized support for reflection-in-action [36] within collaborative design activities. In our research with the EDC during the last decade, we have extended the following boundaries:

The design of MAPS involves three different groups of participants: assistive technology professionals and special education teachers, parents of clients, and professional caregivers. MAPS was tested with representatives of several different groups, resulting in the identification of the following requirements for meta-design: (a) discover and learn about the client’s and caregiver’s world and their interactions; (b) observe and analyze how tasks and learning of tasks were currently conducted; (c) understand and explicate the process of creating and updating scripts; (d) comprehend and analyze the process of using the scripts with a real task; and (e) gain an understanding of the role of meta-design in the dynamics of MAPS adoption and use.

• Transcending limited engagement with cultures of participation. Participants are more readily engaged if they perceive the design activities as personally meaningful by associating a purpose with their involvement [7,35].

By designing the MAPS environment to enable script redesign and reuse, caregivers were able to create an environment that matched the unique needs of individuals with cognitive disabilities. MAPS represents an example for extending boundaries by supporting meta-design, embedding new technologies into socio-technical environments, and helping people with cognitive disabilities and their caregivers have more interesting and more rewarding lives. Supporting Collaborative Design with the Envisionment and Discovery Collaboratory

• Figure 2. Different components of the EDC: action space, reflection space, and visualization support

The Envisionment and Discovery Collaboratory (EDC) [4] is a long-term research platform that explores conceptual frameworks for extending boundaries in the context of

172

Full Papers

Proceedings: NordiCHI 2010, October 16–20, 2010

Participants must be able to naturally express what they want to say [28], and the interaction mechanisms offered must have a “low threshold” for easy participation and a “high ceiling” for expressing sophisticated ideas [38].

models. In addition, the environment supports tagging, ratings, and reviews by the participating community. Interested users can utilize 3D Warehouse for creative collaborations by sharing, downloading, modifying, extending, and reusing existing models. Google Earth has the capability to show objects contained in 3D Warehouse. shows Downtown Denver, Colorado, in 3D as an example.

• Transcending group think by involving stakeholders with different interests and different knowledge. More creative solutions to problems can emerge from the collective interactions with the environment by heterogeneous communities (such as communities of interest [17], which are more diverse than communities of practice [25,46]). Boundary objects are needed [41] to establish common ground and establish shared understanding for communities of interest.

In an ongoing collaboration with our partners from the Google Boulder office in Colorado, we are extending boundaries by pursuing the following research issues. • Allowing users to act as active contributors to 3D Warehouse requires extensive learning support to achieve sufficient mastery of the tools and environments provided.

Obstacles to further investigate the above objectives rest with the difficulties of democratizing the design of the EDC [45] by providing more control to the participants. Currently, EDC participants have to customize the system at the source-code level to reflect the specific characteristics of the city and its urban planning problem. Support for end-user developments is important for the EDC, because each urban planning problem is unique: it has to take into consideration the geography, culture, and population of specific cities. In most cases, EDC developers (the metadesigners) do not have sufficient knowledge of the problem and the social context; they do not know which issues are of greatest concern to the city planners and citizens and which conflicts need to be resolved through the EDC system.

• Assessing the effectiveness of different reward structures (recognition by the community and featuring the best models in 3D Warehouse and Google Earth) for motivating users to participate in the collaborative effort to model the whole world. • Supporting richer ecologies of participation, including roles such as creators, raters, curators, power users, and local developers, while attending to the diversity and independence of participants. Smart Grids and Energy Sustainability

Emerging changes in renewable and sustainable energy include rethinking how electricity is produced, transmitted, distributed, and consumed. The Smart-Grid vision [12] combines electrical and intelligence infrastructures. In March 2008, Boulder, Colorado, was selected to serve as the first Smart-Grid City in the United States. The vision and objective behind Smart-Grid Cities concept is focused on a number of infrastructure upgrades and customer offerings, including:

Modeling the World in 3D: SketchUp, Building Maker, 3D Warehouse, and Google Earth

Having the whole world modeled in 3D and allowing users to explore this virtual world on their computers is the objective behind Google’s effort to integrate SketchUp, Building Maker, 3D Warehouse, and Google Earth. The amount of work and local knowledge needed to model billions of buildings is beyond the scope and capability of any locally operating development team. It requires the contributions of a large user base, and as such represents a unique, large-scale example for assessing the conceptual framework underlying meta-design and cultures of participation.

• transformation of the existing metering infrastructure to a robust, dynamic electric system and a communication network for two-way communication throughout the distribution grid;

SketchUp and Building Maker are 3D-modeling environments (http://sketchup.google.com/). SketchUp allows users to develop sophisticated and highly creative models, but it requires a substantial learning effort. Building Maker is a tool for creating simpler models from aerial images with less effort. Powerful learning mechanisms for these systems are critical to allow everyone who wishes to contribute to learn how to do so. The 3D Warehouse is an information repository for the collection of models created by all users who are willing to share their models (see http://sketchup.google.com/3dwarehouse/). It contains tens of thousands of models from different domains, including buildings, houses, bridges, and so forth, and it uses collections to organize

Figure 3. Downtown Denver, Colorado, in 3D

173

Full Papers

Proceedings: NordiCHI 2010, October 16–20, 2010

• conversion of substations to “smart” substations capable of remote monitoring, near real-time data, and optimized performance; and

complemented by designing socio-technical environments focused on reflective communities. There is an urgent need for understanding, stimulating, fostering, and supporting creative co-production with meta-design and cultures of participation beyond support for solitary activity.

• installation of programmable in-home control devices (at the customer’s invitation) and the necessary systems for more intelligent home energy use.

As cultures evolve, specialized knowledge becomes favored over generalized knowledge [39]. Taking advantage of the distances (spatial, temporal, conceptual, and technological) discussed earlier in this paper represents promising developments for making as many voices as possible heard, exploiting the symmetry of ignorance between different communities of practice with communities of interests [20], and creating shared understanding among stakeholders from different disciplines [40]. Specifically, the EDC is an attempt to extend boundaries by addressing these challenges.

The Smart-Grid vision represents a challenging application domain for meta-design and cultures of participation. Our research [13] is grounded in the basic assumption that to harvest the benefits of Smart Grids, the underlying technologies are necessary but they are not sufficient. To take advantage of Smart Grids, humans must be engaged as active decision makers, and not only as passive consumers. The socio-technical environments must identify and offer the right balance between “automating” and “informating” [30,48]. Taking advantage of the potential of the new emerging technologies will require new mindsets that lead to changes in the way users (as individuals, communities, organizations, and governments) think and learn about energy sustainability.

Understanding the Impact of Personally Meaningful Activities

Actively contributing in cultures of participation, even when supported by powerful meta-design environments, requires a substantial amount of learning and engagement. All humans have a limited amount of time and attention, so they have to carefully choose the activities in which they decide to become active contributors. Socio-technical environments can fail in two directions:

DISCUSSION AND REFLECTIONS

A promising methodology to extend boundaries in HCI research is to exploit the synergistic efforts by applying and testing the conceptual frameworks (meta-design and cultures of participation) in the socio-technical environments discussed in the previous section and in return modifying and expanding the conceptual frameworks by the demands for specific applications. provides a summary how the four different application domains extend boundaries. The following subsections describe a number of themes that originated from these synergistic efforts.

• By forcing participants to act as designers when they prefer to be consumers; this is the case in personally irrelevant activities (depending on the individual, examples of such environments might be having to check one’s own baggage at airports or check out one’s own groceries at supermarkets); • By limiting participants to consumer roles when they

Transcending the Unaided Individual Human Mind

want to be active contributors and designers; this is the case in personally relevant and meaningful activities (examples of such activities for different individuals and communities are provided by the four application domains discussed in the previous section—providing more independence for a family member with a cognitive disability by using the MAPS system represents a personally meaningful problem for the caregiver).

The primary HCI objective of the past to support and educate individual reflective practitioners [36] needs to be Application CLever EDC

3D Modeling

Smart Grids

Extending Boundaries universal design; caregivers are empowered to be active developers support for small groups in face-to-face meetings with table-top computing, boundary objects, visualizations, and simulations; individual artifacts (urban planning environments) are complex and wicked problems decentralized mass collaboration, rich ecology of participation; the complexity is primarily in the aggregate (hundred thousands of models exist in the 3D Warehouse) environments to support, inform, and motivate users to act as active decision makers in the use of energy

Motivation, Control, Ownership, Creativity, and Quality of Artifacts

Meta-design and cultures of participation create a new understanding of motivation, creativity, control, ownership, and quality [6]—topics that have not been considered of great importance for HCI in the past, but which enter center stage in extending the boundaries for the next generation of HCI systems. Motivation

Human beings are diversely motivated beings. We act not only for material gain, but also for psychological wellbeing, for social integration and connectedness, for social capital, for recognition, and for improving our standing in a reputation economy. The motivation for going the extra

Table 2: Summary of how the different applications extend boundaries

174

Full Papers

Proceedings: NordiCHI 2010, October 16–20, 2010

step to engage in active participation was articulated thus [35]: “The experience of having participated in a problem makes a difference to those who are affected by the solution. People are more likely to like a solution if they have been involved in its generation; even though it might not make sense otherwise.” There is widespread evidence that people attach more value to things they create themselves (the so-called “Ikea effect” [5]). Meta-design relies on intrinsic motivation for participation; it offers a new platform for human connection by bringing together otherwise unconnected individuals and replacing common background or geographic proximity with a sense of well-defined and shared purpose. Motivation is critical in the framework and systems discussed in this paper because participation cannot be enforced, only encouraged, fostered, and supported (as indicated by the proverb: “one can lead a horse to water but one cannot make it drink”). Research in motivation transcends usability concerns (what people can and cannot do) by focusing on what people want to do [11].

of those around them) [43], (c) decentralization (participants are able to specialize and draw on local knowledge) [3], and (d) aggregation (mechanisms exist for turning individual contributions into collections). By focusing on open systems and on the unfinished, meta-design and cultures of participation create architectures and seeds that can be evolved on an ongoing basis.

Control

CONCLUSIONS

The importance of the distribution of control has been emphasized as important for architecture [1]: “I believe passionately in the idea that people should design buildings for themselves. In other words, not only that they should be involved in the buildings that are for them but that they should actually help design them.” Meta-design and cultures of participation distribute control among all stakeholders in the design process and are based on the observation that shared control will lead to more innovation [45]: “Users that innovate can develop exactly what they want, rather than relying on manufacturers to act as their (often very imperfect) agents.”

Democratization of the development and evolution of personally meaningful socio-technical environments is more than letting users customize their applications; it represents a fundamental new challenge allowing and supporting the co-creation of situational applications. Empowering and motivating users to become active contributors and decision makers is a grand challenge problem for HCI research. In doing so, we should not ignore that boundaries are in many contexts important means for distinguishing communities and strengthening their identity. The challenge is not ignoring or eliminating boundaries, but respecting them and making them traversable as needed to explore new approaches and new frontiers.

Quality of Artifacts

Quality assurance in systems created by cultures of participation is often grounded in the following concern: “How are we to know that the content produced by widely dispersed and qualified individuals is not of substandard quality?” An interesting case study to reflect upon this concern was done in the context of Wikipedia. The journal Nature has compared the quality of articles found in the Encyclopedia Britannica with those in Wikipedia [23] and has come to the conclusion that Wikipedia comes close to Britannica in terms of the accuracy of its science entries.

Ownership ACKNOWLEDGMENTS

Our experiences gathered in the context of the design, development, and assessment of our systems indicate that meta-design methodologies are less successful when users are brought into the process late (thereby denying them ownership) and when they are “misused” to fix problems and to address weaknesses of systems that the developers did not fix themselves. Meta-design works when users are part of the participatory design effort in establishing a meta-design framework, including support for intrinsic and extrinsic motivation, user toolkits for reducing the effort to make contributions, and the seeding of use communities in which individuals can share their contributions.

I thank the members of the Center for LifeLong Learning & Design (L3D) at the University of Colorado, who have made major contributions to the ideas described in this paper. I specifically would like to thank: Elisa Giaccardi, who is the co-developer of our meta-design framework; Stefan Carmien, who created and evaluated the MAPS environment; Hal Eden, who is the major architect and developer of the EDC; John Bacus from Google Boulder, who provided unique insights into the design of the 3D modeling effort; and Holger Dick and Hal Eden, who are developing the Smart Grid environment. The research was supported in part by (1) grants from the National Science Foundation, including: (a) IIS-0613638 “A Meta-Design Framework for Participative Software Systems,” (b) IIS-0709304 “A New Generation Wiki for Supporting a Research Community in ‘Creativity and IT’,” and (c) IIS-0843720 “Increasing Participation and Sustaining a Research Community in ‘Creativity and IT’”; (2) a Google research award “Motivating and Empowering Users to Become Active Contributors: Supporting the Learning of High-Functionality Environments”; and (3) a

Creativity

Where do new ideas come from in meta-design environments and cultures of participation? Their creativity potential is grounded in user-driven innovations, taking advantage of breakdowns as sources for creativity, and exploiting the symmetry of ignorance [20]. To increase social creativity requires: (a) diversity (each participant should have some unique information or perspective), (b) independence (participants’ opinions are not determined by the opinions

175

Full Papers

Proceedings: NordiCHI 2010, October 16–20, 2010

major grant “CLever: Cognitive Levers: Helping People Help Themselves” from the Coleman Institute for Cognitive Disabilities at the University of Colorado.

14. Eden, H. Getting in on the (inter)action: Exploring affordances for collaborative learning in a context of informed participation. In G. Stahl (Ed.), Proc. CSCL '2002 Conference, Boulder, Colorado (2002), 399-407.

REFERENCES

15. Ehn, P. Manifesto for a digital bauhaus. Digital Creativity 9, 4 (1998), 207-216.

1.

2.

Alexander, C. The state of the art in design methods. In N. Cross (Ed.), Developments in Design Methodology, John Wiley & Sons, New York, 1984, 309-316.

16. Engeström, Y. Expansive learning at work: Toward an activity theoretical reconceptualization. Journal of Education and Work 14, 1 (2001), 133-156.

Andersen, R. and Mørch, A. Mutual development: A case study in customer-initiated software product development. In V. Pipek, M.B. Rossen, B. deRuyter, and V. Wulf (Eds.), End-User Development, Springer, Heidelberg, 2009, 31-49.

3.

Anderson, C. The Long Tail: Why the Future of Business Is Selling Less of More. Hyperion, New York, 2006.

4.

Arias, E.G., Eden, H., Fischer, G., Gorman, A., and Scharff, E. Transcending the individual human mind— Creating shared understanding through collaborative design. ACM Transactions on Computer HumanInteraction, 7, 1 (2000), 84-113.

5.

Ariely, D. The Upside of Irrationality – the Unexpected Benefits of Defying Logic at Work and at Home..HarperCollins, New York, 2010.

6.

Benkler, Y. The Wealth of Networks: How Social Production Transforms Markets and Freedom. Yale University Press, New Haven, CT, USA, 2006.

7.

Brown, J.S., Duguid, P., and Haviland, S. Toward informed participation: six scenarios in search of democracy in the Information Age. The Aspen Institute Quarterly 6, 4 (1994), 49-73.

8.

Carmien, S. Socio-Technical Environments Supporting Distributed Cognition for Persons with Cognitive Disabilities. Ph.D. dissertation, University of Colorado at Boulder. http://l3d.cs.colorado.edu/~carmien/.

9.

Carmien, S., Dawe, M., Fischer, G., Gorman, A., Kintsch, A., and Sullivan, J.F. Socio-technical environments supporting people with cognitive disabilities using public transportation. Transactions on HumanComputer Interaction (ToCHI) 12, 2 (2005), 233-262.

17. Fischer, G. Communities of interest: Learning through the interaction of multiple knowledge systems, In Proc. IRIS'24, Ulvik, Norway (2001), 1-14. 18. Fischer, G. Social creativity: Turning barriers into opportunities for collaborative design. In F. deCindio and D. Schuler (Eds.), Proc. PDC’04, CPSR, University of Toronto, Canada (July 2004), 152-161. 19. Fischer, G. End-user development and meta-design: Foundations for cultures of participation. Journal of Organizational and End User Computing 22, 1 (2010), 52-82. 20. Fischer, G., Ehn, P., Engeström, Y., and Virkkunen, J. Symmetry of ignorance and informed participation. In Proc. PDC’02, Malmö University, Sweden (2002), 426-428. 21. Fischer, G., and Giaccardi, E. Meta-design: A framework for the future of end user development. In H. Lieberman, F. Paternò, and V. Wulf (Eds.), End User Development. Kluwer Academic Publishers, Dordrecht, The Netherlands, 2006, 427-457. 22. Fischer, G., Nakakoji, K., Ostwald, J., Stahl, G., and Sumner, T. Embedding critics in design environments. In M.T. Maybury and W. Wahlster (Eds.), Readings in Intelligent User Interfaces. Morgan Kaufmann, San Francisco, 1998, 537-559. 23. Giles, J. Internet encyclopaedias go head to head. Nature 438 (2005), 900-901. 24. Henderson, A., and Kyng, M. There's no place like home: Continuing design in use. In J. Greenbaum and M. Kyng (Eds.), Design at Work: Cooperative Design of Computer Systems, Lawrence Erlbaum Associates, Inc., Hillsdale, NJ, USA, 1991, 219-240.

10. Carmien, S.P., and Fischer, G. Design, adoption, and assessment of a socio-technical environment supporting independence for persons with cognitive disabilities. In Proc. CHI 2008, ACM Press (2008), 597-607.

25. Janis, I. Victims of Groupthink, Houghton Mifflin, Boston, MA, USA, 1972. 26. Leadbeater, C., and Miller, P. The Pro-Am Revolution – How Enthusiasts Are Changing Our Economy and Society, 2008. http://www.demos.co.uk/files/proamrevolutionfinal.pdf

11. Csikszentmihalyi, M. Flow: The Psychology of Optimal Experience, HarperCollins Publishers, New York, 1990. 12. Department of Energy, Smart Grid. http://www.oe.energy.gov/smartgrid.htm.

27. Mumford, E. A socio-technical approach to systems design. Requirements Engineering, 2000, 59-77.

13. Dick, H., Eden, H., and Fischer, G. Energy sustainability and Smart Grids: Fostering and supporting cultures of participation in the energy landscape of the future. (submitted for publication, 2010).

28. Myers, B.A., Ko, A.J., and Burnett, M.M. Invited research overview: End-user programming. In Proc. Human Factors in Computing Systems, CHI'2006, Montreal (2006), 75-80.

176

Full Papers

Proceedings: NordiCHI 2010, October 16–20, 2010

29. National Research Council. Beyond Productivity: Information Technology, Innovation, and Creativity, National Academy Press, Washington, DC, 2003.

39. Simon, H. A. The Sciences of the Artificial, 3rd ed. The MIT Press, Cambridge, MA, USA, 1996.. 40. Snow, C.P. The Two Cultures, Cambridge University Press, Cambridge, UK, 1993.

30. Norman, D.A. Things That Make Us Smart AddisonWesley, Reading, MA, USA, 1993. 31. O'Reilly, T. What Is Web 2.0 – Design Patterns and Business Models for the Next Generation of Software. http://www.oreillynet.com/pub/a/oreilly/tim/news/ 2005/09/30/what-is-web-20.html.

41. Star, S.L. The structure of ill-structured solutions: Boundary objects and heterogeneous distributed problem solving. In L. Gasser and M.N. Huhns (Eds.), Distributed Artificial Intelligence, Volume II, Morgan Kaufmann, San Mateo, CA, USA, 1989, 37-54.

32. Porter, J. Designing for the Social Web. New Riders, Berkeley, CA, USA, 2008.

42. Suchman, L.A. Plans and Situated Actions. Cambridge University Press, Cambridge, UK, 1987.

33. Preece, J., and Shneiderman, B. (2009) The reader-toleader framework: Motivating technology-mediated social participation. AIS Transactions on HumanComputer Interaction 1, 1 (2008), 13-32.

43. Surowiecki, J. The Wisdom of Crowds. Anchor Books, New York, 2005. 44. Tapscott, D., and Williams, A.D. Wikinomics: How Mass Collaboration Changes Everything, Portofolio, Penguin Group, New York, 2006.

34. Raymond, E.S., and Young, B. The Cathedral and the Bazaar: Musings on Linux and Open Source by an Accidental Revolutionary. O'Reilly & Associates, Sebastopol, CA, USA, 2001.

45. von Hippel, E. Democratizing Innovation, MIT Press, Cambridge, MA, 2005. 46. Wenger, E. Communities of Practice – Learning, Meaning, and Identity. Cambridge University Press, Cambridge, UK, 1998.

35. Rittel, H. Second-generation design methods. In N. Cross (Ed.), Developments in Design Methodology, John Wiley & Sons, New York, 1984, 317-327.

47. Winograd, T., and Flores, F. Understanding Computers and Cognition: A New Foundation for Design Ablex Publishing Corporation, Norwood, NJ, USA, 1986.

36. Schön, D.A. The Reflective Practitioner: How Professionals Think in Action, Basic Books, New York, 1983.

48. Zuboff, S. In the Age of the Smart Machine, Basic Books, Inc., New York, 1988.

37. Shirky, C. Here Comes Everybody: The Power of Organizing without Organization. Penguin Books, New York, 2008. 38. Shneiderman, B. Creativity support tools: Accelerating discovery and innovation. Communications of the ACM 50, 12 (2007), 20-32.

177

Full Papers

Proceedings: NordiCHI 2010, October 16–20, 2010

Wattsup?: Motivating reductions in domestic energy consumption using social networks 1

Derek Foster1, Shaun Lawson1, Mark Blythe2 and Paul Cairns2 2 Lincoln Social Computing Research Centre Department of Computer Science University of Lincoln University of York Brayford Pool, Lincoln Heslington, York {defoster, slawson}@lincoln.ac.uk {mblythe, pcairns}@cs.york.ac.uk

ABSTRACT

energy at home and fly too often. And yet they do next to nothing to change this. Indeed a recent article in the Ecologist on unnecessary travel was written on board a transatlantic Boeing 747 flight [30]. At the moment we want to cut carbon emissions without changing our lifestyles; we do not want sustainable technologies but perhaps we want to want them. How then can technologies designed to change behaviour become compelling, desirable and enjoyable?

This paper reports on the design, deployment and evaluation of “Wattsup”, an innovative application which displays live autonomously logged data from the Wattson energy monitor, allowing users to compare domestic energy consumption on Facebook. Discussions and sketches from a workshop with Facebook users were used to develop a final design implemented using the Facebook API. Wattson energy monitors and the Wattsup app were deployed and trialled in eight homes over an eighteen day period in two conditions. In the first condition participants could only access their personal energy data, whilst in the second they could access each others‟ data to make comparisons. A significant reduction in energy was observed in the socially enabled condition. Comments on discussion boards and semi-structured interviews with the participants indicated that the element of competition helped motivate energy savings. The paper argues that socially-mediated banter and competition made for a more enjoyable user experience.

It is increasingly recognised that interaction design must address issues of sustainability, e.g. [4, 24, 33]. There has been much work in HCI in the past decade on persuasive technology, e.g. [9, 18, 23]. However Fogg recently pointed out that persuasive technologies very often fail and urged practitioners to think small [19]. Monitoring technologies alone (e.g. pedometers) are often not enough to make meaningful changes in behaviour. This paper draws on previous work on persuasive technologies, e.g. [18, 19] in order to address environmental concerns over domestic energy consumption. Households are responsible for 30% of the UK‟s total energy use [12]. Since 1970, household energy demands have grown by 32% [29] and still continue to grow. Energy demands have grown because of increases in home temperature and the proliferation of consumer electronics [14]. Ulrick Beck identifies increasingly individualized forms of living such as: “living alone, single parenthood, non-marital cohabitation, childless marriage, serial marriage” and “living apart together”, where partners live in separate dwellings” [2]. The impacts of these trends are social but also environmental in that one person living alone uses more energy than two in the same household [3]. Rising energy consumption currently means increased CO2 emissions so domestic energy consumption is very much a world problem e.g. [20, 46].

Author Keywords

Sustainability, Persuasive Technology, Social Networking, Competitive Energy Saving, User Experience ACM Classification Keywords

H5.m. Information interfaces and presentation (e.g., HCI): Miscellaneous. INTRODUCTION

It is generally acknowledged amongst scientists and, increasingly, politicians and corporations that current levels of energy consumption are not sustainable [25]. Many people already know very well that they consume too much Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. NordiCHI 2010, October 16–20, 2010, Reykjavik, Iceland. Copyright 2010 ACM ISBN: 978-1-60558-934-3...$5.00.

This paper reports on the design, deployment and evaluation of a Facebook application which aimed to encourage energy saving by using live and historical energy feedback in a social-normative context.

178

Full Papers

Proceedings: NordiCHI 2010, October 16–20, 2010

BACKGROUND

numerical display that can show units like watts and pounds sterling.

Numerous studies have demonstrated that energy usage falls when people know it is being monitored [46]. Studies in Environmental Psychology have shown that feedback on energy consumption can achieve behavioural change though it is not necessarily sustained without timely reminders [11, 16]. The goal of this project therefore, was not just to effect behavioural change but to demonstrate larger reductions in energy consumption through the addition of a social aspect in monitoring energy usage. The desire to belong and willingness to adapt behaviour to follow what others are doing has been seen as a fundamental motivator [1]. Social norms such as peer pressure have also been seen as a means of changing behaviour to align with the ideas or beliefs of groups [41]. It has been argued that computers now operate as social actors designed to influence our behaviour, filling the roles of teachers, sales people and health agents and [17, 18]. Computers have, for example, been shown to help people overcome their fear of public speaking [43]. Recent work on persuasive technology has argued that reciprocal interaction through instant messaging and “pokes” can be effective strategies for persuasion [47]. Whilst there is insufficient space to give a full account of the psychological theories of social motivation, effecting behavioural change through computer mediated social networks seems promising.

Figure 1: The Wattson energy monitor from DIY Kyoto

Smart monitors such as the Wattson are increasingly common and The UK government has committed to replacing all current electricity metres with smart metres by 2020. This is no guarantee that it will happen of course, but it indicates that the availability of such devices is likely to increase. Facebook

The social networking site Facebook now has over 250 million active users [15]. If the site were a country its population would be greater than that of Russia. Studies of Facebook have demonstrated that users read other people‟s postings, play games, upload comments on photographs and add to their own „profile‟ many times daily [27]. These sites provide a powerful means of delivering small, asynchronous applications to peer groups of likeminded real-world friends in a manageable and pleasant way. There may then be potential in leveraging the engaging power of small applications, offering rich social interactive features to change behaviour.

Energy Monitors

In many homes, electricity meters are often difficult to access (located in cupboards or corners) and usually difficult to read. Energy measurements in kilowatts are difficult to make sense of either in terms of finance or ecological impact [8]. Dynamically updating kilowatt readings are however much more understandable [48]. Studies of paper electricity bills have also shown that awareness and understanding can be increased through graphical representations of financial and normative comparisons [10]. However bills do not provide frequent feedback and a recent study found that continuous feedback over a long period is the best means of changing patterns of energy consumption [16]. The importance of raising awareness about patterns of consumption can be illustrated with reference to still widely held but mistaken beliefs about domestic appliances. One study of mistaken folk theories of energy consumption found that there was still credence given to such myths as - turning a thermostat up higher than necessary heats a room up quicker, and leaving computers and lights on consumes less energy than turning them on and off frequently [28].

Feedback Studies

In recent years there have been many studies of innovative ways of displaying energy consumption. The “power aware cord” offers per-appliance feedback in the form of a glowing power cord but does not indicate energy used or cost [21]. The use of portable and stationary, minimalist direct displays for the home was investigated in another recent study [48]. This found that occupants would move around the house turning appliances on and off to monitor the change of feedback on the portable display they carried and enjoyed this playful approach (ibid). The Phillips iCat study used a robot with happy or sad facial expressions corresponding to low or high energy use scenarios in a lab [37]. Although the study was limited to the lab and involved the use of a very expensive robot it indicated that participants responded more positively to the playful expression of the cat rather than a bar chart [22].

The Wattson home energy monitor is a standalone device which is designed to raise awareness of domestic energy consumption. It is an off the shelf technology which takes readings from electricity metres and displays the information as real-time energy usage data. It has a light system that provides ambient feedback as well as a

The two previously discussed studies did not employ the use of any normative feedback, social or otherwise. But some work has been carried out in using social platforms to motivate people to reduce their ecological impact. Mankoff

179

Full Papers

Proceedings: NordiCHI 2010, October 16–20, 2010

[34] proposed the use of web widgets like “badges” showing carbon footprints which could be incorporated on platforms like MySpace but the proposed implementation relied on users self-reporting their energy information. A recent study used a large situated display to compare energy consumption between dormitories of a university campus showing expenditure by both halls and individuals. The display was only updated bi-weekly yet resulted in massive reductions in electricity and water [36]. A Facebook group was created for the study but did not have a high uptake. A Facebook group network can only be fed updates in the form of self-reported textual or static image posts with little guided or sophisticated interaction. Facebook applications provide a more personalised experience through access to the users profile, profile actions and news feed. A Facebook application displaying personal student energy usage tied into each student‟s unique Facebook account profile may have been more effective.

Figure 2: WattsOn and Nabaztag YouTube Video

Participants commented on the comic potential of the device and the possibility of fun, playful interactions. This generated general discussions around the concept design of a socially enabled home energy monitor and what that might look like.

This study aimed to address a gap in current work on leveraging social platforms by embedding live, continuous energy data into a fully interactive socially-enabled energy application. Using the Facebook Developers Kit (FDK) API, Wattson devices were linked to Facebook allowing us to investigate whether sharing such information between friends might make for further reductions in energy consumption.

Pencils, coloured pens, scissors and other craft materials were also provided allowing participants to create their own interface elements.

DESIGN PROCESS

Focus groups were conducted with a convenience sample [40] of four Facebook users aged between twenty three and thirty eight. There were three males and one female and all were responsible for paying the energy bills in their homes. Discussions took place in a home lab on campus at a university and helped the participants focus on the home as a design space. It should be noted that as the participants were students on a Masters course in HCI they were more sensitive to issues of interface design than other groups might be.

Figure 3: Initial Sketches developed in focus group

A large number of ideas were generated and discussed. Various graphical metaphors were suggested such as a mouse running in a cage turning faster or slower depending on energy consumption, balloons with user‟s faces on them were pictured being inflated larger or smaller and floating higher or lower to indicate ranked consumption rates (see figure 3. There were a number of interesting suggestions. For instance: “a digital photo frame, if you aren’t using much energy then you see pictures of beautiful scenery or if you’re using a lot some stark scenery, almost like a piece of art that changes over time”. Smiley and sad faces were also suggested as simple but very easily understood graphical elements (see Figure 5).

Participants were shown the Wattson monitor and a selection of YouTube videos related to energy consumption. One video featured a Wattson monitor over a period of a few minutes displaying live home energy readings, from low to high when a cooker is switched on; simultaneously, a “Nabaztag” outputs a vocal message that a lot of energy is being used at that moment and flashes its lights.

The participants were then given a paper materials pack containing various paper interface elements such as user avatar icons, energy icons and CO2 icons to help create prototype interfaces. Plain interface template cards were added to the pack enabling the participants to place the interface elements on the blank interface templates.

180

Full Papers

Proceedings: NordiCHI 2010, October 16–20, 2010

Figure 4: Interface elements given to participants

Much of the discussion revolved around the difficulty of relating to the kilowatt as a unit of energy measurement. “Kilowatts, watts, I don’t want to see any of that, money yes.” There were interesting debates about whether financial representations were more powerful than ecological ones “I’m thinking a kind of erm some form of visual presentation....you’ll see more trees over time in an image if you use less energy, see trees knocked down if you use more” There were also interesting general discussions of issues such as privacy when sharing data raising concerns such as: “The risk of failure in front of your friends.” However it was generally agreed that introducing a competitive element between friends who were free to opt in or out of the group might help drive a reduction in consumption.

Figure 5: My Energy: Workshop sketches and final design

Sketches developed in the workshop (see the left hand side of figure 5) were developed into final designs (see the right hand side of figure 5). Workshop comments also informed the design e.g.: “What the dials can do is give you a comparison, although they are very abstract, I can still tell if I have used loads of energy today.” The final design then related quite directly to initial discussions. The Friends screen would display personal energy consumption against selected friends.

IMPLEMENTATION

Moving on from the conceptual design stage was made relatively straight forward due to the high quality of the user generated designs. Following discussions in the focus group it was decided that the main interface attributes for displaying energy would be expressed in Watts and pounds sterling (£) as well as CO2 emissions measured by weight. In addition to numerical representations, a graphical representation was selected to display alongside both numerical values for energy and Co2 emissions in the form of the happy/sad face theme as previously discussed. Three core interfaces were developed to provide an engaging user experience: My Energy, Friends and Rankings. The My Energy screen would show energy consumption with a dial visualisation and a seven day history bar chart.

Figure 6: “Friends” screen final design

Again this final design built directly on ideas and comments from the workshop “I can see it working amongst a group of friends, but finding a way to notify users of the group of whats happening’”.

181

Full Papers

Proceedings: NordiCHI 2010, October 16–20, 2010

The Rankings screen would show a table of highest and lowest energy users of the application.

Figure 7; Rankings Screen, Initial workshops sketches and final design.

Figure 8: Technical Design Overview

Again there was a very direct link between the final design and workshop suggestions such as “„I thought about a leagues table based on points, slightly competitive but not with pressure and a bit where people can discuss it’” and rough sketches such as those on the left hand side of Figure 7. The ranked pyramid of the heaviest consumers suggested in the workshop (see the bottom left hand side of Figure7) was simplified to a ranking table similar to that imagined in another workshop sketch (top left hand side of Figure 7).

The diagram illustrates the Wattson sensor at the householders fuse box sending the current energy reading by radio signal to the Wattson energy monitor which stores the data in its 30 day dedicated memory. The Wattson is physically attached to a PC via a USB cable which transports the energy data from the Wattson to the PC via the desktop “Powometer” application where it is stored in a local SQL mobile database. At configured intervals Powometer sends the stored energy data to the myenergyusage.org service via the internet where it is stored in multiple MySQL databases for redundancy. Once the energy data is stored on the myenergyusage.org databases it is then presentable to authorised applications to make use of it. The Wattsup technical design and implementation tasks were approached using Evolutionary Prototyping [38].

Another important interface element was the integration of a comments board. The rankings table would allow users to visualise what their standing was against others but it would not facilitate friends commenting on personal or others‟ energy consumption. This feature was added following discussions of the importance of interacting with friends through such an application. A cloud tag feature was also added where users could note the devices they thought were high energy users.

EXPERIMENTAL METHOD Aim

As with any prototype design, choices were limited by time and material constraints. A number of interesting concepts were generated in the workshop which were not feasible within the scope of the project. As is well documented in the participatory design literature, the sketching process was invaluable not only in generating the final designs but also in recognizing a wider design space e.g. [6].

The aim of the study was to see if energy savings could be increased by the addition of a social element to monitoring. As it is already known that monitoring can reduce energy consumption the focus was on the social element and we did not consider households without any energy monitors of any kind. To this end, we made a socially enabled version of the Wattson energy monitor via Facebook. The hypothesis was that less energy would be used whilst the Wattson was socially enabled than when it was not socially enabled.

DEPLOYMENT

The application was developed using ASP.NET, Microsoft C#, HTML/CSS, XML, MySQL, Facebook Markup Language, and the FDK API. When completed it was deployed to the Facebook application platform.

Participants

Eight households were recruited to trial the prototype with each participant being the person responsible for paying the electricity bill. The number of participating households was limited by the number of devices available to the researchers. Selection of the participants followed a purposive sampling method [40]. The criteria for

To afford the functionality displaying live and recent energy data within a Facebook application, an elaborate application framework was designed and implemented as seen in Figure 8.

182

Full Papers

Proceedings: NordiCHI 2010, October 16–20, 2010

recruitment were that the lead participant in each household must be responsible for paying their household electricity bill and be a daily user of the Facebook website. None of the households had owned any type of energy monitor prior to the study. Participant

Profession

Age

No. Household Members

Diane Alice

Nurse Nurse Office Admin Programmer School Teacher Student Student Writer

26 23

2 2

49

2

32

2

41

2

31 20 40

2 4 4

Rachael Christopher Robert David Richard Shirley

with the required desktop software one week before the experiment officially began. This was done to assist in reducing any effects on participants‟ energy usage by initially using the Wattson on its own, as it was likely to receive attention as a new gadget in the house (and, as previously noted, decrease energy consumption). Each participant gave their informed consent and carried out the experiment in their own home. The experiment took place over a period of 18 days with 9 days in each condition. Half of the participants started in condition A with the other half starting in condition B, after 9 days the participants were sent an email through Facebook informing them of the changeover of conditions. The applications were then reprogrammed to perform in the alternative conditions with the relevant participants. Data was collected in the Wattson device itself, a MS SQL mobile database using the installed desktop software and the myenergyusage.org web service via a MySQL database. The Google analytics service was also used to record the number of Facebook application page views for each interface.

Table 1: Wattsup participant demographics

The participants were also recruited in 4 pairs who resembled one another in circumstances as much as possible. For example if a participant belonged to a family of four then another participant belonging to a family of four was recruited. In total the participants belonged to households with 6 couples and 2 families of four, so twenty people in all were involved in this study. The lead participants had all been regular users of Facebook for at least one year and were all friends who were on one another‟s Facebook friends list. Additional demographical information on the participants is detailed in Table 1 with pseudonyms used in place of the participant‟s real names.

RESULTS

The energy usage, in kWH, in both conditions for each household is summarised in Figure 9.

Design

The experiment followed a within subjects design [7] with each participant taking part in two conditions or social modes. In condition A the Wattsup application was socially enabled, i.e. users could see their friends‟ data as well as their own, in condition B the Wattsup application was manipulated so that there were no social features i.e. users could only see their own energy usage. The households were divided into matched groups and the conditions were counterbalanced between the groups to avoid ordering effects [7]. Group 1 started in condition A, group 2 in condition B and the groups switched conditions halfway through.

Figure 9: Wattsup participant energy usage in each condition

A Wilcoxon test, for comparing repeated measures of nonparametric data, showed that energy consumption was significantly lower when using the socially enabled application (Z= -2.1, N=8, p=0.036).

The independent variable was therefore the Facebook application‟s social mode, either enabled or disabled. The dependent variable was the energy used in Kilowatt Hour units with a total measurement being taken in each condition for each household.

A total of 130Kw units of energy saved by the participants in condition A as opposed to condition B. This amount of energy would be expended by leaving a 60W bulb on for 9 days and result in Co2 emissions similar to those produced by driving an average-sized UK petrol car for 399Km.

Procedure

The experiment required hardware in the form of a Wattson energy monitor and a Windows based PC running the Powometer desktop software available from www.myenergyusage.org to collect energy data from the Wattson monitor. The Wattson monitor was installed along

Additional data collected from Google Analytics highlighted the differences in user interaction activity between both conditions. In terms of the number of times

183

Full Papers

Proceedings: NordiCHI 2010, October 16–20, 2010

Semi-Structured Interviews

the participants visited Wattsup, condition A was significantly more popular than condition B with a total of 263 versus 51 page views respectively. The number of visits to the Facebook application then showed a fivefold increase in the social condition. Participants spent most time on the rankings interface viewing and commenting on the rankings table. It can be assumed that participants enjoyed this feature the most due to the collective amount of page views for rankings as well as the average time spent viewing it

Participants took part in semi structured interviews about their experience with the set uy when the Wattson device was collected from their homes at the end of the study. They were asked: which condition they preferred and why, whether they had any problems using Wattsup and whether they would use it or something similar over a longer period of time. All of the participants preferred the socially enabled condition “I preferred the second one (socially) because I am quite competitive, it gave me further incentive. I think putting a bit of fun in it is quite important.”. All of the participants said they would be very interested in using the system over a longer period of time. With regard to problems using the device some suggested a more visual representation of Co2 emissions.

However with such a low number of participants it is worth exploring the result further by analysing the qualitative data collected. QUALITATIVE DATA

In addition to the descriptive and inferential statistics qualitative data analysis was undertaken on the comments board and on semi structured interviews following the trial.

As with other energy reduction systems participants found monitoring itself quite enjoyable: “Well, this morning we unplugged EVERYTHING one by one, room by room. The reading on the wattson went down by around 450 to 970 watts. Ha, we actually enjoyed investigating this though ;)”

Comments board

The comments board proved to be a popular feature with participants and comments were analysed using a small scale grounded theory approach [44]. Three main categories of comment emerged: banter, engagement and competition. Banter included teasing such as “how come you are at the top, cut down drastically on your cups of tea?” And gloating “good to see I'm higher in the table then you rob ha ha” as well as provocations such as ““energy vampire...you clearly are!!””. This kind of banter extended to joking exchanges such as:

Although financial and ecological concerns may also be important factors this and other studies clearly indicate that fun should be taken seriously, eg [5]. DISCUSSION

One UK study claimed that sustained behavioural change with domestic energy consumption was unlikely to alter until more than 3 months had elapsed [11]. However, the energy feedback in that study was not delivered within a socially enabled context; therefore it is possible that the claim of 3 months minimum for energy usage behaviour change may not hold when a social platform is used to deliver the feedback. Due to time constraints and resources available this project could not address the experiment duration issue for sustained behaviour change. These findings then may be viewed as a pilot study potentially leading onto a larger and longer term study.

Diane: “I’ve turned my washing machine settings down as this uses loads...no pun intended” Alice: “I once turned down a washing machine. Wasn't pretty. It went into a cycle of depression and self-loathing before finally giving me my socks back” Engagement included disclosure of information such as “Left my main PC on the last two nights. Made a massive difference to my scores.” And disbelief about energy usage “gone down in the rankings? I´m in Spain :S” Engagement could indicate enjoyment “Woohooo, looking good today!” but sometimes also mixed feelings of guilt and disinclination to change “I NEED TO STOP PLAYING PUTER! (idontwanttoidontwanttoidontwantto)”.

But the approach may be criticised more generally. It could be argued, for instance, that computers are part of the problem and not the solution. Part of the reason for the massive increases in domestic energy consumption is the proliferation of computers and other energy intensive devices. In 1995 29% of the UK population owned a PC, in 2006 this had more than doubled to 65% [35]. The power supply of today‟s computers is around 300w and growing as technology progresses, significantly more than computers of the 1990s [39]. Increasingly sophisticated components such as graphics cards and processors are using more energy with each subsequent generation.

Other comments indicated pleasure in the competitive aspect of the rankings “Take’s the top spot: D” this seemed enjoyable even to those who were not necessarily winning “I've been usurped”. This competitive element occasionally led to questions “hey [name omitted] what’s your secret? your energy rating is pretty good...”. Comments were made when participants moved both up and down the ranking table indicating engagement in the process throughout.

It is difficult to find reliable figures estimating the carbon footprint of an entity like Facebook. Some estimates place

184

Full Papers

Proceedings: NordiCHI 2010, October 16–20, 2010

CONCLUSION

the number of servers necessary as 10,000 with an additional 300,000 user PCs connected at any one time [32, 42]. While all carbon footprint estimates are disputable the footprint of Facebook and its users is clearly large, perhaps, as some claim, the equivalent of a major city. The number of Facebook users is often compared to the populations of nations. If Facebook is a country you need a computer to live in it. Using Facebook to reduce energy consumption then may be rather like eating more pies in order to lose weight.

The paper has described the design, deployment and evaluation of a Facebook application designed to allow friends to compare their domestic energy consumption. The quantitative and qualitative analysis of the data collected from participants in this study suggests that social networking sites may be able to play a role in reducing energy consumption in the home by making monitoring more enjoyable. This was a small scale study and only a larger investigation could conclusively determine how effective such applications may be. However, these results are encouraging. Competitive carbon counting appears to be both more enjoyable and more effective than individual monitoring.

However there may be a value to endeavours such as this beyond an immediate and measurable net reduction of personal energy consumption. Such studies cannot help but raise awareness of energy consumption if only because of the Hawthorne effect. For ecological change to take place there must also be ideological change. This study generated a considerable amount of press interest [e.g. 45] which indicates that sustainability is now a very real concern of users. That said, the focus on individuals and personal energy consumption must not detract from larger political interventions. As Paul Dourish has argued we must raise awareness not just of the ecological consequences of leaving the lights on but the consequences of our decisions at the ballot box [13].

The limits of how much any individual can achieve by changing their own lifestyles are often pointed out. But collective behaviour change even on a small scale is increasingly recognized as a key to tackling global warming. The International Energy Agency, for instance, estimate that devices on standby cause a full one percent of world greenhouse emissions, this is nearly equal to that of the entire aviation industry[46].

It has been argued that in an age of potential ecological catastrophe user centred design is no longer appropriate [e.g. 4]. The user‟s needs and preferences should not take precedent over the impact of their technologies on the environment. However this is to configure the user solely as a consumer. Users may also be considered as citizens. Although some scientists continue to doubt that global warming can be attributed to human activity it is clear that a great many users / citizens are now entirely persuaded and want to do something about it. The impact of our technologies is a real concern at the level of user experience. User experience is already broadly conceived in terms of social and psychological perspectives, it is becoming clear that it must be conceived in still wider terms to address the ecological challenges of the coming years.

Social networking sites like Facebook and Twitter are increasingly being appropriated by users for political and social ends. Facebook is of course primarily for fun but it may be that the enjoyable aspects of the service make for effective platforms for persuasive technologies.

Slavoj Zizek has argued that enjoyment is a political factor in any social structure, even, or especially if it is extremely repressive. The jokes and private satires against the party in communist states were no threat to the regime, indeed they were entirely necessary for its smooth running [49]. Lovelock has argued that if we were to take the ecological challenges that we face seriously then we would adopt systems of rationing much harsher than those of the second world war [31]. Even if the cataclysmic predictions of climate change are wrong it is clear that the way we consume energy must change. If harsh measures are to be endured then perhaps we must find ways to enjoy them.

REFERENCES

ACKNOWLEDGEMENTS

The authors wish to express their sincere appreciation to the project participants and their families for their invaluable assistance in the research and development of this project‟s. Particular thanks also go to Crispin Proctor for giving permission and assistance in accessing energy data online from the myenergyusage.org web service. Thanks also goes to the staff of the Lincoln Social Computing Research centre for offering their valuable opinions on various aspect of the project‟s work. 1. Baumeister, R.F., and Leary, M.R.: The need to belong: Desire for interpersonal attachments as a fundamental human motivation. Psychological Bulletin 117 (1995), 497–529. 2. Beck U. Individualisation: institutionalized individualism and its social and political consequences. Sage London UK, 2002 3. BERR, Energy consumption in the United Kingdom – BERR. http://www.berr.gov.uk/files/file11250.pdf 4. Blevis E. Sustainable Interaction Design: Invention and Disposal, Renewal and Reuse. In Proc. CHI 2007, ACM Press (2007), 502-512

185

Full Papers

Proceedings: NordiCHI 2010, October 16–20, 2010

5. Blythe, M.A., Overbeeke, K., Monk, A.F. and Wright, P.C. Funology: from usability to enjoyment. Kluwer Academic Publishers, Dordrecht, Germany, 2003

greenhouse gases. Office for National Statistics, London, UK 2004 21. Gustafsson, A. and Gyllenswärd, M. The power-aware cord: energy awareness through ambient information display. Ext. Abstracts CHI 2005. ACM Press (2005), 1423-1426.

6. Buxton B. Sketching the User Experience: Getting the Design Right and the Right Design. Morgan Kaufman. Amsterdam, Holland, 2007 7. Cairns, P., and Cox, A.L Research Methods for HumanComputer Interaction. Cambridge University Press, Cambridge, UK, 2008.

22. Ham, J. and Midden, C. A robot that says bad!: using negative and positive social feedback from a robotic agent to save energy. In Proc. HRI 2009. ACM Press (2009), 265-266. 23. Ham, J., Midden, C., and Beute, F. Can ambient persuasive technology persuade unconsciously?: using subliminal feedback to influence energy consumption ratings of household appliances. In Proc. Persuasive 2009, vol. 350. ACM Press (2009)

8. Chetty, M., Tran, D., and Grinter, R. E. Getting to green: understanding resource consumption in the home. In Proc. UbiComp 2008, vol. 344. ACM Press (2008), 242-251. 9. Consolvo, S., Everitt, K., Smith, I., and Landay, J. A. Design requirements for technologies that encourage physical activity. In Proc CHI 2006, ACM Press (2006), 457-466.

24. Huang E., & Truong K. Breaking the Disposable Technology Paradigm: Opportunities for Sustainable Interaction Design for Mobile Phones. In Proc. CHI 2008, ACM Press (2008), 323-332

10. CSE, Consumer Preferences for Improving Energy Consumption Feedback. http://www.cse.org.uk/pdf/pub1033.pdf

25. IPCC, IPCC Fourth Assessment Report: Working Group I Report “The Physical Science Basis” http://www.ipcc.ch/ipccreports/ar4-syr.htm

11. Darby, S. The effectiveness of feedback on energy consumption: A review for DEFRA of the literature on metering, billing and direct displays, Environmental Change Institute, University of Oxford, Oxford, 2006.

26. James, N., & Desai, P. A Study into the Development of Sustainability Rating for Homes. Advisory Committee on Consumer Products and the Environment BioRegional Development Group. http://www.defra.gov.uk/environment/consumerprod/ac cpe/research/pdf/accpe_final030711.pdf

12. Defra, Department for Environment, Food and Rural Affairs: Autumn Performance Report 2006. http://www.defra.gov.uk/corporate/apr/apr2006.pdf 13. Dourish P., Print This Paper, Kill A Tree: Environmental Sustainability as a Research Topic for Human – Computer Interaction. Working paper from the Laboratory for Ubiquitous Computing and Interaction at UC Irvine, n.d.

27. Joinson, A. N. Looking at, looking up or keeping up with people?: motives and use of Facebook. In Proc. CHI 2008. ACM Press (2008), 1027-1036. 28. Kempton, W. Two Theories of Home Heat Control, Cognitive Science 10 (1986), 75-90

14. EST, Energy Saving Trust‟s Response to the DTI Energy Review Consultation. http://www.berr.gov.uk/files/file31091.pdf

29. Kinver, M. Saving Energy on the Home Front (BBC News – 23/10/06) http://news.bbc.co.uk/1/hi/sci/tech/6056398.stm

15. Facebook, Facebook | Statistics http://www.Facebook.com/press/info.php?statistics

30. Lee M. How Do You Define Unnecessary Travel. The Ecologist. June 2009, 71-72

16. Fischer, C. Feedback on household electricity consumption: a tool for saving energy? Energy Efficiency 1, 1 (2008), 79-104

31. Lovelock, J. The Revenge of Gaia. Basic Books, 2006 32. Malik O. Facebook‟s Insatiatible Hunger for Hardware. Gigacom. http://gigaom.com/2008/04/25/Facebooks-insatiablehunger-for-hardware/

17. Fogg, B. J. and Nass, C. Silicon sycophants: the effects of computers that flatter. Int. J. Hum.-Comput. Stud. 46 (1997), 551-561.

33. Mankoff, J., Blevis, E., Borning, A., Friedman, B., Fussell, S., Hasbrouck, J., Woodruff , A., and Sengers, P. Environmental Sustainability and Interaction. In Proc. CHI 2007, ACM Press (2007), 2121-2124.

18. Fogg, B.J. Persuasive Technology – Using Computers to Change What We Think and Do. Morgan Kaufmann, San Francisco, 2003. 19. Fogg, B. Three possible futures for persuasive technology. In Proc. Persuasive 2009, ACM Press (2009)

34. Mankoff, J., Matthews, D., Fussell, S. R., and Johnson, M. Leveraging Social Networks To Motivate Individuals to Reduce their Ecological Footprints. In Proc. HICSS, 2007, IEEE, London, 87a

20. Francis, P. The impact of UK households on the environment through direct and indirect generation of

35. National Statistics, National Statistics Online http://www.statistics.gov.uk/cci/nugget.asp?id=1710

186

Full Papers

Proceedings: NordiCHI 2010, October 16–20, 2010

36. Odom, W., Pierce, J., and Roedl, D. Social Incentive & Eco-Visualization Displays: Toward Persuading Greater Change in Dormitory Communities. In Proc. of Public and Situated Displays to Support Communities. Workshop at OZCHI 2008.

43. Slater, M., Pertaub, D-P., and Steed, A. An Experiment on Public Speaking Anxiety in Response to Three Different Types of Virtual Audience. Presence 11, 1 (2002), 68-78

37. Philips, iCat http://www.hitech-projects.com/icat/

44. Strauss A. and Corbin J. Basics of Qualitative Research: techniques and procedures for developing grounded theory. Sage Publications, London, 1998.

38. Pressman, R.S Software Engineering: A Practitioner’s Approach, 6th edn. McGraw Hill, New York, 2005.

45. Thompson, R (2009) Facebook App Measures Your Carbon Footprint. Computer Weekly, August, 2009.

39. Przybyla, D. and Pegah, M. Dealing with the veiled devil: eco-responsible computing strategy. In Prco. SIGUCCS, ACM Press (2007), 296-301.

46. Walker G., and King D. The Hot Topic: How to Tackle Global Warming and Still Keep the Lights on. Bloomsbury, London, 2008.

40. Robson, C. Real World Research, 2nd edn. Blackwell Publishing, Malden MA, 1993.

47. Weiksner, M. and Fogg, B.J. Liu: Six Patterns for Persuasion in Online Social Networks. In Proc. Persuasive 2008, pp151-163

41. Schultz, P.W., Nolan, J.M., Cialdini, R.B., Goldsteing, N.J., and Griskevicius, V. The Constructive, Destructive and Reconstructive Power of Social Norms. Psychological Science, 18, 5 (2007), 429-434.

48. Yun, T. Investigating the impact of a minimalist inhome energy consumption display. In Ext. Abstracts CHI 2009. ACM Press (2009), 4417-4422.

42. Sen A. 10 Billion Facebook Photos – Is That Ecologically Sustainable? Trendspotting Blog. http://www.trendsspotting.com/blog/?p=487

49. Zizek S. (2008) For They Know Not What They Do: Enjoyment as a Political Factor. Verso, London, 2008

187

Full Papers

Proceedings: NordiCHI 2010, October 16–20, 2010

Empirical Investigation of Web Design Attributes Affecting Brand Perception Franca Garzotto, Fabio Sorce Politecnico di Milano Departments of Electronics and Information Via Ponzio 34/A 20133 Milano, Italy {Garzotto, sorce}@elet.polimi.it

Davide Bolchini, Tao Yang Indiana University School of Informatics - Indianapolis 535 W Michigan St. 46202 Indianapolis IN U.S.A. {dbolchin,taoyang}@iupui.edu the corporate branding strategy but also to identify the “value” resulting from a web user experience in forming or sustaining a positive “brand image”. The work presented in this paper contributes to the understanding of this issue, focusing on the role of web design in e-branding.

ABSTRACT

The web has become a mainstream communication tool used by companies, institutions, celebrities, and politicians to establish, reinforce, or repurpose their brand, almost bypassing more conventional branding media. In this scenario, understanding the effects of the different design qualities of a website on users’ attitude towards the brand is of major importance. This paper contributes to the exploration of this issue by presenting a wide empirical study that investigates the degree to which users’ perception of a brand is affected by different design aspects of a website, namely usability, aesthetics, and communicability. The results of this multidimensional analysis have implications on HCI research and practice, as they provide empirically founded guidelines to prioritize design choices in relationship to branding goals.

The practice of marketing and industrial design generally acknowledges that the user experience (UX) with any well design reification of a brand – being it a product, a service, a communication or advertisement artefact - translates to a positive attitude towards the brand itself [47]. An implication of this general principle is that the quality of the UX with a website affects the gamut of feelings, beliefs, or judgments that users associate to the brand. Recent HCI studies pinpoint that people’s opinion on a web user experience is modulated by a number of factors of different nature: subjective (related to the individual predisposition of the user [26]), objective (related to the design characteristics of the website), and contextual (how the experience is situated in place and unfolds in time). The objective (i.e., design-oriented) features that are usually considered to affect the perceived quality of the UX include both pragmatic attributes, which focus on the fulfillment of user’s “operational” goals, and hedonic qualities, which affect the psychological well-being of the individuals [28]. The former are typically associated to the general concept of usability. The latter are related to a broad range of design attributes, such as aesthetics, attractiveness, engagement, joy, and fun [4, 18, 25, 28, 37].

Author Keywords

Brand, Web, User Experience, Design, Value, Usability, Aesthetics, Communicability, User Attitude ACM Classification Keywords

H5.m. Information interfaces and presentation (e.g., HCI): Miscellaneous. INTRODUCTION

The promotion of a brand through the web — a process known as e-branding — is reaching in importance, if not overcoming, traditional paradigms of branding that support more passive, “one-way” forms of communication (e.g., conventional advertising) and are less engaging for users [9, 23, 30, 47, 48]. In this arena, it becomes important for many stakeholders - managers, designers, and marketers – not only to align the design of a web communication artefact to Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. NordiCHI 2010, October 16–20, 2010, Reykjavik, Iceland. Copyright 2010 ACM ISBN: 978-1-60558-934-3...$5.00.

188

The purpose of our research is to investigate the aggregated and individual effects of different objective, design oriented characteristics of a web UX on brand perception. We focus on a pragmatic design attribute – usability, and a hedonic attribute – aesthetics. In addition, we consider a third attribute of the design of websites – communicability – which, to our knowledge, has never been taken into account in existing studies in the design/e-branding domain. Specializing a concept of traditional semiotic theories [16, 39], we define communicability as the capability of the sole information content of a website, abstracting from the “channel” that wraps up it (interface and the interaction capabilities), to convey meaning. The “meaning” we are interested in is the set of messages that the brand underlying the web application intends to convey to its target.

Full Papers

Proceedings: NordiCHI 2010, October 16–20, 2010

The study involved overall 140 participants and was designed as a set of seven experiments. Each experiment considered a different website for which we evaluated the above design attributes and measured the effects of web exposure on users’ brand perception, which was operationalized using value-based metrics. We then analyzed the resulting data in order to identify the existence of significant correlations between each design attribute and the modification in brand perception, comparing the different findings.

brand value can be a moral, ethical, social, or cultural belief, benefit, or lifestyle which an entity is committed to thorough its overall behavior, its products and services, and is worthwhile for a given target group, either at individual or collective level. A brand value represents therefore those traits of an entity’s personality that the entity holds as salient to leverage on and to communicate via its products, interactive artifacts, services, or marketing actions. Syntactically, a brand value takes the form of a single statement or attribute, defining a trait of the brand, as in the following example of brand values for one of the applications considered in our study (Table 1).

To our knowledge, this is the first work that systematically investigates the effects of web design for e-branding purposes from such a multidimensional perspective, and provides some empirical evidence of the individual role played by different design qualities in the complex trajectory from a web user experience to the creation or modification of the brand image. At the same time, our findings raise a number of questions that highlight the complexity of this domain and the need for future research.

BV1

BV2

Magic/Fairytale Spectacular

BV3

BV4

BV5

Exciting

Amusing

Adrenalinic

Table 1. A subset of Brand Values (BVs) of Gardaland (the largest entertainment park in Italy - www.gardaland.com)

Consistently with this line of reasoning, we define Brand Perception in terms of Brand Values Perception, i.e., the users’ subjective opinion that an entity, its products and services, actually fulfill the promise of the values declared by their brand. Given a brand, we measure Brand Perception as the strength of the association, in the users’ mind, between the brand values and the entity, products, or services associated to the brand.

THE EMPIRICAL STUDY Research Goal

The research goal of our empirical study is to discover and isolate the individual effects of three web design attributes – usability, aesthetics, and communicability - on brand perception, identifying whether and at which degree these attributes are good indicators of the effectiveness of a website to convey the brand image, and discovering if there is any dominant factor in this process. We operazionalize and measure these three independent variables in seven different websites with the purpose of discovering their correlation with the change of brand perception (before vs. after using the site) as dependent variable.

Performance-based and Heuristics-based Usability

In our study we are interested in the most “classical”, functional view of web usability, which considers the operational ease of use and the degree of effective and efficient support to users’ information and operational goals. Among all the possible metrics that can be used to assess usability of a website, we adopt two complementary measures: standard performance-based usability — the degree to which users can successfully perform a set of tasks — and heuristics-based usability. The latter denotes the results of heuristics inspection of usability as performed by a team of expert evaluators examining the interface and judging its compliance with recognized usability principles. Rather than adopting Nielsen’s heuristics, we needed to consider finer grained and more structured set of usability principles that could specifically address the pragmatic qualities of the web design suggested by many existing web and hypermedia design models [41]. To this end, we adopted the set of “domain independent” heuristics1 provided by MILE+ method for web usability inspection [7]. MILE+ distinguishes between:

Study Variables Brand Perception

The modern concept of brand is extremely wide and deserves multiple interpretations that go beyond the simple, original notion of brand as a “sign” — word expression or symbol — which identifies the goods or services of a vendor and differentiates them from those of the competitors. A more recent approach to brand definition is more abstract, and encompasses the perceived qualities of both the good (i.e., the products or the service) and the “entity” that stand behind the good: a company, an institution, an organization, a person or, at a wider level, a community or a country. [17] proposes that the brand represents a promise of recognizable values that are unique to an entity and its goods, and they can keep to all their stakeholders - customers, trades, stockholders, employees, fans, or supporters. These are not necessarily functional, utilitarian qualities, but can be anything that gives rise to positive, emotional, or affective effects.

•

1

Content heuristics (8): they address pragmatic qualities of content like accuracy, consistency, linguistic

MILE+ also provides a set of heuristics for assessing usability features in specific web application domains, which are not relevant for the purpose of this study.

Embracing this approach, and in line with value-based design thinking in HCI [22] [10] [11], we model the concept of brand in terms of Brand Values [5, 6, 8]. A

189

Full Papers

correctness, readibility of the communicated to the users by a website;

Proceedings: NordiCHI 2010, October 16–20, 2010

information

•

Navigation heuristics (36): they consider the consistency and effectiveness of information architecture and link structures;

•

Presentation heuristics (31): they address the consistency and effectiveness of the presentation of the different interaction elements and of the overall page lay-out;

•

process that involves multiple subjective and contextual factors and elaborates both explicit and implicit (indirect, evocative, symbolic, or allusive) references to such “objects”. As a consequence, it is difficult to identify objective measures of communicability. It is more feasible to consider the perceived communicability of a website, as the results from users’ subjective rating of the evidence of brand values messages in contents elements only. Scoping Our Research

Technology heuristics (7): they focus on technologydriven features of the application, e.g., system messages or page loading time, or effects caused by implementation defects.

To restrict the boundaries of our investigation and better scope our research, we focused our study on a “category” of e-branding artifacts: brand-oriented content-intensive websites2. These are characterized by the goals that inform their design: reinforcing, establishing or repurposing a brand, and satisfying substantial information and knowledge needs of users. In other words, brand-oriented content-intensive websites are designed both to trigger positive feelings and attitudes towards a given brand and to support the effective use of a large quantity of articulated content.

Usability assessment carried on using MILE+ allows us to measure web usability (although subjectively as any inspection method) along multiple perspectives, which in turn enables us to explore the effects of pragmatic design qualities on brand perception of web users in a much finer grained way. MILE+ offers all the known advantages of heuristic evaluation and has been empirically proved [7] to be easy to learn and more effective (in terms of quantity of discovered usability defects vs. time employed in the inspection) than conventional Nielsen’s method, since it provides a systematic guidance to perform the inspection process.

It is important to characterize the investigated websites in terms of both the above design goals (brand and information purposes) because these criteria are appropriate for the design qualities we set out to investigate in relationship to brand perception. We consider, in fact, hedonic factors like aesthetics, attributes that are more pragmatic (usability), and features related to content (communicability and content accuracy, consistency, linguistic correctness, readibility). “Brand-oriented only” websites tend to privilege the hedonic aspects of the UX and the creation of emotional engagement with a brand. As such, they would offer less significant case studies than websites that are also content-intensive, because we aim at comparing how design attributes of diverse nature affect brand perception and also at understanding the branding role of web UX also along its functional, utilitarian, and more rational dimension.

Aesthetics

The model of perceived aesthetics used in our study follows the one proposed by Lavie and Tracktinsky [33], which differentiates between classical aesthetics and expressive aesthetics. Classical aesthetics refers to “traditional” aesthetic notions emphasizing orderly, balanced, and clear design. It includes attributes such as pleasant, clear, clean, symmetrical, and aesthetic. Expressive aesthetics is modeled by qualities that capture the design’s creativity and originality. Relevant items in this dimension are creative, fascinating, original, sophisticated design, and use of special effects.

Instruments

The study was organized in two sub-studies, the first one addressing the evaluation of usability, aesthetics, and brand values perception, and the second one addressing communicability. Each sub-study involved different subjects and, in part, required different instruments, as discussed below.

Communicability

As mentioned in the Introduction section, communicability denotes the “communication power” of the sole information content of a web system. More precisely, it expresses the degree to which text, image, video, sound, or animation units in a website manifest the brand values of the entity underlying the application and are capable to convey them to the users.

“Original” Websites

We selected seven brands – all related to consolidated companies or institutions in different business sectors and the corresponding websites (see Table 2), meeting the criteria of being brand-oriented and content-intensive, as discussed above.

Our concept of communicability does not imply that we consider explicit brand messages only, as determined, for examples, by the entity’s mission statements, brand mottos, tag line, or logo. As Eco teaches us [16], the meaning of content signs is not necessarily determined by whether they explicitly refer to actual “objects” – brand values in our case. Indeed, the meaning of a message that is formed in a person’s mind is a result of a complex interpretation

2

The term “content-oriented” website is commonly used in web engineering, in most cases as synonymous of “data intensive”, to indicate websites that provide large amount of information items.

190

Full Papers

Proceedings: NordiCHI 2010, October 16–20, 2010

Website #1 Apple #2 Bang & Olufsen #3 Fender #4 Gardaland #5 Mirabilandia #6 Egyptian Museum Turin #7 Nintendo

url www.apple.com/it www.bang-olufsen.it www.fender.it www.gardaland.it www.mirabilandia.it www.museoegizio.it

Business Sector High-Tech High-Tech Musical Instruments Entertainment Park Entertainment Park Cultural Heritage

www.nintendo.it

Digital Entertainment

to re-implement each entire application in this form, we used the following approach. For each activity scenario of a given website, we defined, an appropriate “path” on the website, i.e., a sequence of pages that a “typical” user would traverse to complete the tasks in the scenario. The path was identified on the basis of the results of the first study of usability and aesthetics, identifying the most frequent steps followed by the users to execute the assigned scenarios. For each sequence of original pages we created a sequence of content-only html or flash pages that we then submitted to the participants involved in the communicability assessment study. A content-only page provides all text, sound, video, animation, and image units extracted from its corresponding page in the real website but it does not include any interaction and navigation capability, and have minimal lay-out (e.g., no background or special fonts - see Figure 1).

Table 2. The websites considered in our study. Brand Values and Activity Scenarios

For each website, we performed a detailed analysis to determine how the entity behind the application wants to be perceived by its target audience and to elicit the entity’s brand values. We followed the process guidelines provided by the value-driven requirements method illustrated in [5, 8], which structures brand values elicitation into a combination of business analysis, techno-organizational analysis, and user analysis. In short, we carried out brand value extraction from explicitly declared brand sources (institutional mission statements, press releases, and other public domain information such as investor relations material), interviews (via phone or face-to-face) to both institutional stakeholders (e.g., marketing managers or CEO’s) and brand “customers”, and direct, contextual experience of the product or services (e.g., visits to the entertainment parks or brand shops). The qualitative and quantitative data collected with these instruments were distilled in a list of (ten, on average) brand values for each website, which were further validated with institutional stakeholders.

Participants and Settings

A total of 140 users were involved in the overall study, which was carried on from February to September 2009. For each website, two subject groups were recruited and randomly assigned to either the study on usability, aesthetics, and brand perception, or to the study on communicability. For each website, we assigned ten participants to the first study and ten different participants to the second study. One hundred and twenty students of two master classes in HCI at Politecnico di Milano carried out the recruitment of participants (involving relatives and friends) as part of the course homework. All subjects were screened from a larger set, using a pre-test questionnaire that elicited demographic data and consistency of their profile with the brand target of each website. It is important to notice that we considered only persons who never used the website under study, to prevent pre-conceptions and biases derived from previous uses of the application. The sample was composed of males (56%) and females (44%) of different ages (65% aged 1835, 20% aged 35-50), balancing the age distribution according to the profile of the targets of the different brands. Reward for participation was the gift of an interactive CD (either a multimedia guide of a local museum or an interactive game, both developed by the research lab of one of the authors).

During this preliminary phase, we also identified an articulated profile of the main target users for each website, including their information and operational needs and the motivations for using the application. We distilled this knowledge into the definition of a set of activity scenarios that helped us screen the participants involved in the study and define the tasks to be assigned to the users during the exposure to each website. A set of tasks, specific for each application, operationalized high-priority goals for the intended users (e.g., “planning a family visit” on a museum website). The nature and sequence of the tasks were designed to frame a plausible and realistic experience with the website as it would arise in a spontaneous situation of use.

Observers and Usability Inspectors (13 overall) were members of our research lab in Milan, all with a significant experience on UX evaluation both in academic and industrial contexts. To preserve ecological validity, the first study (assessment usability, aesthetics, and brand perception) was performed in the natural usage environment of the participants (at home, work office, or university). The second, more controlled study, given the technical skills required to prepare and administer the communicability test, was carried out at the laboratory of one of the authors.

Content-Only Web “Sites”

Measuring communicability requires distinguishing the effects of users’ exposure to implicit and explicit messages contained in the pure content units from the semiotic effects of the interactive experience of contents as mediated by the interface. For each website, we therefore created a “content-only version”, where we removed all lay-out and interaction or navigation features. Since it would have been too expensive

191

Full Papers

Proceedings: NordiCHI 2010, October 16–20, 2010

Figure 1. Left: Home Page of an original website (www.gardaland.it); Right: content-only version (non-interactive experience).

Procedure

scenarios, the questionnaire used prior to the website explosure was administrated to the users, to collect their actual perception of the brand values after the experience with the website.

Measurements of Brand Perception and Performance

To assess Brand Values Perception, we used a questionnaire. Some questions aimed at identifying whether and at which degree users would associate an “entity”, or its products or services, to a set of brand values, inviting users to indicate, for each brand value, the strength of this association on a 5-points Likert scale (0 = Not at all; 4= Strongly). Other questions asked participants to express their level of agreement with a brand’s fulfillment of brand values statements, measured using a 5-points Likert scale (0 = Strongly Disagree; 4= Strongly Agree).

Inspecting Heuristic Usability

Each website was assigned to a team of 3 inspectors, who individually performed a systematic heuristics evaluation using the MILE+ method and its reporting instruments (structured data entry forms). Each inspector rated each applicable MILE+ heuristic with a 5 points Likert scale denoting compliance level (0=Very Low; 4 = Very High) and reported design problems according to shared guidelines. Then, as in conventional inspection methods [40, 46] the 3 inspectors came together to discuss the problems found and their severity, motivate their judgments, consolidate the results and finally converge their scores to an agreed level.

One important aspect of our procedure is the need to isolate the effects on brand perception resulting from the exposure to the website during the experiment from those resulting from previous experience with other brand reifications, e.g., traditional marketing actions, word of mouth, use of brand products or services or the website itself. To this end, we measured users’ brand perception before and after the use of the website, and considered the actual increment or decrement of these measures only. As mentioned in the previous section, study participants had never used the website before. This removed the potentially confounding factor that a previous exposure to e-branding actions could have introduced in our study.

Evaluating Aesthetics

Each website was evaluated by the users on each of the 10 adjectives (defined by the model in [33]) that denote classic and expressive aesthetics using a 5-points scale (0 = Strongly Disagree; 4= Strongly Agree). We administered a questionnaire after the exposure to a website and after compiling the brand perception post-usage questionnaire.

The brand perception questionnaire was administrated to the users before using the website. Then each subject was asked to use the website by following 3 assigned activity scenarios. The average overall duration of a session of use was approximately 40 minutes. During users’ execution of the assigned scenarios, observers collected data on performance, in terms of level of task success (task completion rate) and time-on-task. At the end of all

Evaluating Communicability

Each subject in the “content-only” group was exposed to a scenario-based experience of the content-only version of the assigned website. We removed any form of interactivity from the user experience by giving the facilitator the responsibility of opening, for each scenario, the corresponding “content-only” pages, one after another, in the proper sequence, at a pace defined by each user.

192

Full Papers

Proceedings: NordiCHI 2010, October 16–20, 2010

Correlations

Dynamic elements (sound, video, animation) were also activated by the facilitator on user’s request.

To check whether the four predictors are measuring different characteristics of the considered applications, we looked for the presence of high correlations among the predictors. A high correlation may increase or mitigate the effect of a single predictor, which would make the situation much more complex to interpret. The VIF (Variance Inflation Factor), which detects strong linear relationships between predictors, shows that there are no strong correlations (1.196). The highest correlation is between classic aesthetics and expressive aesthetics, but it is not high in absolute terms (r=.478). Therefore, we can confidently conclude that the four predictors actually measured meaningfully different constructs of the user experience with the website.

A questionnaire was submitted to the participants after the overall content-only experience to assess communicability. Questions were articulated around brand messages, asking users whether and to what degree, in their opinion, each brand value “emerged” from the experienced content, on a 5 points Likert scale (0=Not communicated at all; 4=Strongly Communicated). RESULTS

We analyzed the data with standard effect and correlation analysis methods to investigate the relationships between the independent variables (performance-based usability, heuristics-based usability, aesthetics, and communicability) and the dependent variable (brand values perception). The main results are discussed in the rest of this section.

Overall, these preliminary results indicate that the selected websites are a “good” sample to represent content-intensive and brand-intensive web applications. They are normally distributed in terms of communication impact (Mean=.27, SD=.39), and they significantly differ in each of the four predictors.

Checking for Variability and Sample Quality Usability Performance

The level of usability performance on the seven websites is significantly different (one-way ANOVA, F(7,69) = 9.95, p < .01). On average, Turin Egyptian Museum has the highest score (mean = 88.65, SD = 5.36), while Gardaland the lowest (mean = 68.57, SD = 7.62). These results indicate that the sample of websites selected account for substantial variability in terms of usability performance, and this contributes to strengthen the overall reliability of the data.

Multi-Dimensional Effect on Brand Perception Factors Affecting Brand Perception Change

We run a multiple regression to discover relationships between performance-based usability, aesthetics, and content communicability as predictors, and brand perception change as the outcome variable (Figure 2). Based on the results of the multiple regression analysis (see Table 3), we used the standardized regression coefficient (β) as an indicator of the effect of a predictor on the communication impact. We were then able to quantitatively model how brand values perception is affected by usability performance, aesthetics, and content communicability. The model shows how much a given predictor affects the outcome variable (brand values perception), if the effects of the remaining predictors are held constant.

Classic and Expressive Aesthetics

The scales used to measure classic and expressive aesthetics are both reliable (Cronbach’s Alpha > .7). The websites are significantly different among each other both on classic (F(7, 69) = 5.29, p < .01) and expressive aesthetics (F(7, 69) = 5.19, p < .01). Apple has the highest score on class aesthetics (mean = 4.06, SD = .31), while Gardaland the lowest (mean = 2.88, SD = .78). For expressive aesthetics, Mirabilandia has the highest score (mean = 3.62, SD = .65), while Fender the lowest (mean = 2.38, SD = .86).

Constant Performance-based Usability Classic Aesthetics Expressive Aesthetics Content Communicability

Content Communicability

Like the other two factors, the values of content communicability of these websites are also significantly different (F(7, 69) = 6.87, p < .01). The content of Nintendo website has the highest communicability (mean = .44, SD = .34), however, the Fender website has the lowest and even a negative communicability (mean = -.31, SD = .41).

B -.740

SE B .388

β

.004

.004

.097

.206 -.024 .423

.069 .057 .106

.342* -.047 .416*

Table 3. Multiple Regression for Brand Values Perception: R2 = .364 (p < .001). * p < .01

Brand Values Perception

The communicability of the site is the predictor that most significantly accounts for the variability of the brand values perception (β =.416). In other words, the brand values perceived by the users when exposed to the multimedia content of the site only (in isolation from the interactive and graphic elements of the design) account for most of the overall brand perception in the fully interactive experience.

Overall, users’ perceptions of the brand values for each website are significantly different (F(7, 69) = 4.68, p < .01). The brand values for Nintendo website are best perceived by the users (mean = .64, SD = .46), whereas the brand values for Gardaland website are the worst perceived (mean = -.03, SD = .30).

193

Full Papers

Proceedings: NordiCHI 2010, October 16–20, 2010

Classic aesthetics (β =.342) is the second most important factor that significantly affects brand values perception. Expressive aesthetics shows a negative, but not significant effect (β =-.47).

The fact that the considered design attributes can only explain 36.4% of the variation in brand values perception (R2=.364) is an indicator that there are a number of variables — of subjective and objective nature — we could not fully control for in our study, but have an impact on brand perception. Hedonic and pragmatic design factors other than usability, aesthetics, and communicability may play a strong role in forming or modifying the image of a brand in the user mind and are worth being investigated. Furthermore, the heterogeneity of the business sectors considered may have introduced a confounding variable. The selection criterion for the websites in our study was only to be brand-oriented and content-intensive, as this class of applications has design goals that are particularly relevant for the design attributes considered. As we wanted to study both hedonic and pragmatic factors in relationship to brand perception we needed to focus on websites whose design is informed by both branding goals and functional goals (such as information goals). Still, abstracting from the business sector brought in a broad variability of branding strategies and approaches. This in turn may have introduced a strong variability in the initial brand perception of the study participants who had familiarity with the brand; it may have acted as a strong mediating effect on the user’s judgment during the experiment [12] in spite of our attempt to control for this aspect (by isolating what impacted on user’s brand perception at the beginning through pre-use brand perception evaluation).

We also checked for a potential interaction effect between content communicability and classic aesthetics. In fact, we have seen that better communicability yields better brand values perception, but the effects of communicability could have been influenced by classic aesthetics (present in the stimuli of the communicability study). However, we verified that this effect is not significant (p > .05). Interestingly, usability performance does not significantly affect brand values perception (β =0.97). This suggests that, in our considered case studies, users could still strongly and positively perceive the brand values even when task performance is low but other design dimensions are well designed. Factors Correlating with Brand Perception Change

The assessed values of heuristics-based usability as a whole significantly correlate with the level of brand values perception (r=.817). This suggests that the level of usability of the design characteristics as inspected by the evaluators follows closely the variation in users’ brand perception. Still, when looking at the specific design dimensions considered during inspection, presentation design has the only significant correlation with brand perception (r=.775). The measures of the other pragmatic factors – usability of content, navigation, and technology aspects - do not have significant correlations.

On the other hand, focusing on brands in a specific business sector could reduce this variability factor but might introduce further complexity. Using the business sector as the only selection criterion for our case studies would have meant not to discriminate between brand-oriented only websites and websites that are also content-intensive, thus removing a confounding variable but adding another one.

In summary, we point out three key findings: 1.

Communicability is the most important predictor of brand perception impact, followed by classic aesthetics.

2.

As to the intrinsic characteristics of the application evaluated by inspectors’ ratings, presentation design has the strongest positive correlation with brand perception, while the other pragmatic factors seem to have a marginal effect.

3.

Overall, communicability, classic and expressive aesthetics, and usability account for little more than one third (36.4%) of the variation in brand values perception (R2=.364).

In spite of these limitations, we think that the results of our study have some interesting implications that are worth discussing and can be influential on the practice and research in user experience design and evaluation. Implications Content as Key Driver for the Brand Experience

Our findings clearly indicate that the brand values communicated by the content are the ones that mostly determine the overall brand perception. This goes much beyond the simplistic slogan that “content is king” and points to the fact that properly designed content is the most critical vehicle to effectively convey brand values and create a positive brand image in users’ mind. Note that content here does not indicate just “information”, but the organic, value-driven shaping of the messages (conveyed by various media) that the website intends to get across to its target users. We can summarize the key implication of our research as follows: Value-driven content is the primary concern when designing a website for branding purposes.

DISCUSSION Limitations of our study

Our research has its shortcomings which may have affected its validity, most notably the relatively limited sample size used for each website. Still, the fact that we have replicated the study in seven web applications using the same methodological framework partially compensates this drawback and strengthens the external validity of our results.

194

Full Papers

Proceedings: NordiCHI 2010, October 16–20, 2010

Designing value-charged content plays an often underestimated role in UX design. A deep exploration of design concerns pertaining content requirements, conception, articulation, deployment and evaluation, and a seamless integration of this understanding into existing HCI methods and tools opens important perspectives for the future development of the field.

significantly affect brand value perception. This means, for example, that even strong (positive or negative) variations in user performance do not change significantly the user’s opinion on a brand. On the other hand, the overall usability of content, navigation and presentation design (as measured through usability heuristics expert rating) has a significant correlation with brand values perception. This suggests that the brand values communication effect originated by these pragmatic qualities is generally stronger than threats to the brand coming from breakdowns in user performance.

The Role of Aesthetics

The significant effect of classic aesthetics on brand perception demonstrates that implicit brand messages are effectively conveyed through design attributes that are very much related to the most conventional concept of “beauty”, such as clean, clear, pleasant, and symmetrical. This finding is somehow consistent with the high correlation found between presentation heuristics score and brand perception. In fact, the heuristics used by experts during the inspection of the presentation-related aspects of the website are conceptually akin to characteristics of classic aesthetics, since they mainly focus on the consistency of the visual characteristics of page elements and on their lay-out balance and regularity. In contrast, the attributes of expressive aesthetics, which concern more the emotional sphere, do not seem to significantly modify brand perception. This is quite surprising, since it seems to contradict, at least in part, the principles of emotional design [37] which suggests that emotions have a crucial role in the human ability of sense making. A possible explanation of this result is that users are more familiar with classic aesthetics attributes, which are more related to our conventional concept of beauty, and therefore can identify more immediately, transparently, and rationally the associations with intended brand values [46] Rating expressive characteristics, instead, require users to reach a deeper level of cognitive reflection, interpretation, and judgment, which in turn makes the association to brand values more indirect and distant.

RELATED WORK AND CONCLUSIONS

Recent studies pinpoint the complex role that different intertwined dimensions of the web user experience play in users’ judgment of a website and their attitudes towards its. Still, few of them specifically investigate these aspects in relationship to e-branding and brand perception. [12] illustrates that a subjective factor like brand attitude does influence the users’ evaluation of aesthetics, pleasure, and usability of websites, while [29] highlights the influence of brand awareness on the user evaluation of search engine results. [45] highlights the correlation between users’ perception of interface aesthetics and their judgment on system’s usability. [27] introduces a framework for users’ design quality judgments based on Active Decision Making Theory. [3] manipulates usability and aesthetics of an interactive system to assess the relative impact of these factors in users’ perceived “value” of the application. [32] considers a number of key visual design factors and identifies their quantitative relationships with 13 classes of emotions that people usually feel when viewing diverse homepages. In the field of persuasive computing [19], a number of studies have examined the design factors that affect consumer’s trust in e-commerce websites [21, 24, 44], influence people’s perception of the website credibility [20] and ultimately have an impact on consumers' product preference. [14] considers in particular the relationship between beauty and usability and the general impact of beauty on product choice and use.

It is also important to notice that the fact that classical aesthetic qualities of the design are related to the overall brand values perception across the sites analyzed, does not yet reveal which specific brand values are more influenced by this aesthetics dimension. Thus we cannot exclude that our results is influenced by the nature of the brand values of the websites considered in our study. A finer grained analysis of the relationship of each aesthetic dimension with specific values may reveal that the perception of values that have a more emotional, affective nature is indeed affected by expressive aesthetics. This analysis was outside the scope of our study, but the exploration of this and similar issues points to interesting future research directions. Based on the results of our study, we can suggest that after quality of value-charged content, designing for classic aesthetics is the most effective way to positively affect the brand image.

A number of authors claim that brand image - intended as the set of “brand associations held in consumer memory” [31] – is influenced by hedonic aspects of an interactive product [37] as well its pragmatic characteristics (e.g., usability [43]). [6] provides some empirical evidence to the correlation between usability and brand perception, and also offers a reusable, methodological support to replicate the reported study and to investigate additional design characteristics of a website in relationship to brand. [6] provides a useful operationalization of the notion of brand perception, which we have also used in the study reported in this paper, adopting a value-oriented perspective to model brand perception, and is centered around the concept of “value”. This notion is widely investigated in (online) marketing, also in relationship to measures of the perception of brand personality and the economic benefits derived from an improved brand image [1, 2, 13, 34, 35, 36, 38, 40, 42]. We instead embrace a notion of value that is

The Role of Usability

It is important to note the tension between two emerging results. On the one hand, usability performance does not

195

Full Papers

Proceedings: NordiCHI 2010, October 16–20, 2010

content – communicability, followed by classic aesthetics, are the most important factors to consider to effectively generating brand values perception. Still, we have assessed that the considered design factors can only explain little more than one third of the variation in brand values perception, at least in our case studies, that focused on a specific class of applications. If confirmed by further studies, involving a larger size sample of testers and a wider number of websites, these results mean that, beyond usability, classic aesthetics, and communicability, there are other, subjective and objective factors in the web User Experience that may account for the remaining variation of brand values perception. Further research is needed to discover them and indentify their role for e-branding. Finally, other genres of branded websites are challenging domains for investigating the applicability of similar user studies. We think for example of brand-oriented applications that are mainly intended to provide services, and therefore stress the functional qualities of the interface and the interaction, or those that focus almost exclusively on branding goals and thus heavily rely on special advertising-like multimedia effects. These classes of branded web systems may require similar approaches as ours, but different constructs and investigation methods, more relevant to the nature of the brand communication strategy at stake.

closer to what has been investigated in HCI design frameworks, such as Values-Sensitive Design [22] and Value-Centered Design [10]. More specifically, we take inspiration from Worth Centered Design (WCD) [11] This paper substantially extends the results of [6] for a number of reasons. [6] only considered how web usability as a whole affects brand image, without discriminating the role played by the different design dimensions that modulate usability, as we do in this paper. The work reported here also expands the analysis to a spectrum of brand affecting design dimensions beyond usability that include [classic and expressive] aesthetics and communicability. This is the first work that empirically investigates the effects of web design for e-branding purposes from such a multidimensional perspective, and provides evidence of the role played by the different design characteristics in the creation or modification of the “brand image”. Among the considered design factors, communicability is novel. A branch of HCI named Semiotic Engineering uses this term with a different meaning, as “the degree to which an interactive system communicates efficiently and effectively to users the underlying design intent and interactive principles” [15]. Instead, we embrace the concept of communicability from a different perspective, more related to traditional semiotic theories existing before the advent of interactive multimedia [16, 39]. Based on these approaches, we consider communicability as the fundamental semiotic property of the sole content elements of a website to more or less transparently communicate meaning. In particular, we evaluate the degree to which “pure” multimedia content, isolated from all the interactive elements through which it is delivered, conveys the intended brand image, and define an original procedure to measure it.

ACKNOWLEDGMENTS

A special thank to all master students of the HCI classes at Politecnico di Milano who contributed to user recruitment in their families and work places, and to all the participants who volunteered for the study. REFERENCES

1. Aaker D.A. The financial information content of perceived quality. Journal of Marketing Research, 31, 1996, 191-201

Our work provides contributions both from a research and practical perspective. In terms of contributions to the body of knowledge on user experience research and e-branding, we have: (a) analytically modeled and isolated some hedonic and pragmatic factors of web user experience design influencing brand perception, in a class of web application that are both brand-oriented and contentintensive; (b) for this category of websites, empirically demonstrated and discussed the relative importance and role of communicability, aesthetics, performance-based usability and heuristics-based usability to influence the user’s brand values perception of a website.

2. Aaker D.A. Dimensions of Web Personality. Journal of Marketing Research, 34, 1997, 347-356 3. Ben-Bassat, T., Meyer, J., Tractinsky, N., Economic and subjective measures of the perceived value of aesthetics and usability. ACM TOCHI, 13, 2, ACM Press (2006) 210-234, 4. Blythe, M.A., Overbeeke, K., Monk, A.F., Wright, P.C. (eds.): Funology: From Usability to Enjoyment. Kluwer (2005) 5. Bolchini D. Garzotto F., Value-Driven Design for “Infosuasive” Web Applications. Proc. 17th Int. Conf. on the World Wide Web (WWW’08). ACM Press (2008), 745-754 6. Bolchini D. Garzotto F. & Sorce F. Does Branding Need Web Usability? A Value-Oriented Empirical Study. Proc. INTERACT 2009, LNCS 5727, Springer (2009), 652-665

Besides, our work can have implications for the practice of UX design and e-branding. A question that seems to be crucial in the industrial world is “How do you prioritize interaction design when boundaries are set by time, money and resources?” (as raised at the Industrial Day UX session at Interact 2009) Our work sheds a preliminary light on this issue, as it provides implications for the consideration and prioritization of the design dimensions to inform effective user experience designs. Specifically, we have provided some empirical evidence that a semantic property of media

7. Bolchini, D., Garzotto, F. Quality and Potential for Adoption of Web Usability Evaluation Methods: An Empirical Study on MILE+. J. Web Engineering, 7 (4) Rinton Press (2008)299-317, 8. Bolchini, D., Garzotto, F., Paolini, P., Branding and Communication Goals for Content-Intensive Interactive Applications. Proc. 15th IEEE International Conference on Requirements Engineering, IEEE Press (2007) 173 - 182 173182.

196

Full Papers

Proceedings: NordiCHI 2010, October 16–20, 2010

9. Chaffey, D., Ellis-Chadwick, F., Johnston, K., Mayer, R. Internet Marketing - Strategy, Implementation and Practice. Prentice Hall (2006)

30. Kapferer, J. N. The New Strategic Brand Management: Creating and Sustaining Brand Equity Long Term. Kogan Page, London, 2004.

10. Cockton G. A Development Framework for Value-Centered Design. Proc. CHI 2005. ACM Press (2005) 1292-1295

31. Keller, K. L. Strategic Brand Management; Building, Measuring, and Managing Brand Equity. Prentice-Hall, 1998

11. Cockton, G. Designing Worth is Worth Design. Proc. NordiCHI 2006. ACM Press (2006) 165-174.

32. Kim J., Lee J., Choi D. Designing emotionally evocative homepages: an empirical study on the quantitative relationships between design factors and emotional dimensions. International Journal of Human-Computer Studies, Elsevier (2003) 899-940

12. De Angeli A., Hartmann J, Sutcliffe A. The Effect of Brand on the Evaluation of Websites. Proc. INTERACT 2009, LNCS 5727, Springer (2009) 638–651 13. Delgado, E., Munuera, J. L., Yagüe, M. J. Development and Validation of a Brand Trust Scale. International Journal of Market Research. 45. 1, 35-54. 2003 14. Diefenbach S., Haasenzahl M. The Beauty Dilemma: Beauty is Valued but Discounted in Product Choice. Proc. CHI 2009, ACM Press (2009), 1419-1426 15. De Souza C.S, Leitão C. F. Semiotic Engineering Methods For Scientific Research in HCI. Morgan and Claypull (2009) 16. Eco U. A Theory of Semiotic, Macmillan (1977) 17. Fisher-Buttinger C., Vallaster C. Connective Branding. Wiley (2008) 18. Fishwick, P. (ed.) Aesthetic Computing. MIT Press (2005) 405422 19. Fogg, B.J. Persuasive technology. Morgan Kaufmann (2003) 20. Fogg BJ, et al., , What Makes Websites Credible? A Report on a Large Quantitative Study. Proc. CHI’01, ACM Press (2001), 61-68 21. Friedman, B., Kahn, P. H., Howe, D. C. Trust Online. Comm. of the ACM. 43.12, ACM Press (2000) 34-40. 22. Friedman, B., Kahn, P. H. Human Values, Ethics, and Design. In J. A. Jacko & A. Sears (Eds.). The HCI Handbook. Lawrence Erlbaum Associates (2003). 1177-1201. 23. Gilbert A. C., Iacobucci D. Marketing Research: Methodological Foundations. Thomson South-Western Publishers (2005) 24. Ha H., Perks, H. Effects of Consumer Perceptions of Brand Experience on the Web: Brand Familiarity, Satisfaction and Brand Trust. Journal of Consumer Behavior. 4. 6, 2005. 438452. 25. Hartmann, J., Sutcliffe, A., Angeli, A. D. Investigating Attractiveness in Web User Interfaces. Proc. CHI 2007 ACM Press (2007). 387-396. 26. Hartmann, J., De Angeli, A., Sutcliffe, A.: Framing the User Experience: Information Biases on Website Quality Judgement. Proc. CHI 2008. ACM Press (2008), 855-864 27. Hartmann, J., Sutcliffe, A., De Angeli, A. Towards a theory of user judgment of aesthetics and user interface quality. ACM TOCHI 15, 4. ACM Press (2008), 1-30 28. Hassenzahl, M.: The Interplay of Beauty, Goodness, and Usability in Interactive Products. Human-Computer Interaction 19 (2004) 319-349 29. Jansen, B., Zhang, M., Zhang, Y. The Effect of Brand Awareness on the Evaluation of Search Engine Results. Proc. CHI 2007. ACM Press (2007) 2471-2476.

197

33. Lavie, T., Tractinsky, N. Assessing dimensions of perceived visual aesthetics of websites. International Journal of HumanComputer Studies 60, 2004, 269-298 34. Low G.S., Lamb C.W. The measurement and dimensionality of brand associations. Journal of Product and Brand Management, 9 (6), 2000 35. Macias, W. A Preliminary Structural Equation Model of Comprehension and Persuasion of Interactive Advertising Brand Websites. Journal of Interactive Advertising. 3. 2, 2003. 36. Müller, B., Chandon, J. The Impact of Visiting a Brand Website on Brand Personality. Electronic Markets. 13. 3, 2003. 210 – 221 37. Norman, D. A. Emotional Design. Basic Books (2004) 38. Page, C., Lepkowska-White, E. Web Equity: A Framework for Building Consumer Value in Online Companies. Journal of Consumer Marketing. 19. 3, 2002, 231-248 39. Pierce, C.S. Collected Writings. Indiana University Press (1982) 40. Romaniuk, J., Nicholls, E. Evaluating Advertising Effects on Brand Perceptions: Incorporating Prior Knowledge. International Journal of Market Research. 48. 2, 2006.179-192. 41. Rossi, G.; Pastor, O.; Schwabe, D.; Olsina, L. (Eds.), Web Engineering: Modelling and Implementing Web Applications, Springer (2008) 42. Slotegraaf, R. J., Pauwels, K.. The Impact of Brand Equity and Innovation on the Long-term Effectiveness of Promotions. International Journal of Marketing Research. 45. 3, 2008. 293306. 43. Spool, J.M. Branding and http://www.uie.com/articles/branding_usability/

usability.

44. Tsygankov, V. A. Evaluation of Website Trustworthiness from Customer Perspective, a Framework. Proc. 6th International Conference on Electronic Commerce. ACM Press (2004). 265271 45. Tractinsky N., Katz A.S., Ikar D., What is beautiful is usable. Interacting with Computers 13 (2000), 127-145 46. Tractinsky N. and Zmiri D. Evaluating the consistency of immediate aesthetics perception of web-pages. Int. J. of HumanComputer Studies 64 (2006) 10071-1083 47. Yoo B. Donthu N., Lee S. An Examination of Selected Marketing Mix Elements and Brand Equity. Journal of the Academy of Marketing Science, 28, 2, 2000. 1995-211 48. Ward, M. R., Lee, M. J. Internet shopping, consumer search and product branding. Journal of Product & Brand Management 9, 1 MC UP (2000) 6-29

Full Papers

Proceedings: NordiCHI 2010, October 16–20, 2010

“Luckily, I don’t need it”: Elderly and the Use of Artifacts for Time Management Leonardo Giusti FBK-irst Trento, Italy [email protected]

Eleonora Mencarini University of Siena Siena, Italy

ABSTRACT

Massimo Zancanaro FBK-irst Trento, Italy [email protected]

In this paper we adopt the ecological perspective proposed by Forlizzi and colleagues [2] as a way to make sense of the experience of aging. This framework takes into consideration the complex interactions between, and the experiences that result. New technology should be conceived of as a part of an interdependent system encompassing people, products, and activities to support elderly values and practices. In particular, in this paper we address the specific ecology of practices and values that mediate elderly people use of time-scheduling artifacts.

In this paper, we describe the artifacts and the practices that a group of elderly people use to plan future events and remembering information about future event at the proper time. The role of such artifacts in the construction of a narrative and emotional account of elderly life is discussed. A particular attention is given to the description of the values that mediate the use of time-scheduling artifacts. By taking into consideration the whole complexity of time management ecology, we finally suggest a number of design opportunities.

We present the results of several semi-structured interviews and some contextual interviews in the homes of a group of 19 elderly people with no specific health problems and with a rich social life.

Author Keywords

Elderly, Time-management,. ACM Classification Keywords

In the followings sections, we will describe in details the artifacts and the practices that elderly people use to plan future events and remembering information about future events at the proper time. Furthermore, the role of such artifacts in the construction of a narrative and emotional account of elderly life will be discussed. A particular attention will be given to the description of the values that mediate the use of time-scheduling artifacts.

H5.m. Information interfaces and presentation (e.g., HCI): Miscellaneous. INTRODUCTION

In this paper, we describe a research work aimed at understanding how older adults in Northern Italy use lowtech artifact for managing their time. The ultimate goal being to inform the design of time-scheduling technologies to better fit this class of users.

By taking into consideration the whole complexity of time management ecology, we finally suggest a number of design opportunities. In particular, we feel that any new technology aimed at supporting time-management should be considered not only from a functional point of view: the role that such artifacts have in supporting elderly in making sense of time should be considered as well.

The adoption or rejection of a new technology by elderly people is known to be strictly dependent how they find it intrusive, complex and revealing their limitations or disrupting their home environment [1]. It is therefore extremely important to take into consideration existing practices and values when designing new technologies for elderly people.

RELATED WORK

In recent years there have been many studies on the design of time management tools. They mostly concentrate within two main areas: the environment in which time management takes place, either working place [3] or the domestic environment [4, 5] and the use of calendars and notes [6, 7] as management tools.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. NordiCHI 2010, October 16–20, 2010, Reykjavik, Iceland. Copyright 2010 ACM ISBN: 978-1-60558-934-3...$5.00.

Notes have three main functions: immediate use, temporary storage and perspective memory aid [6]. They focus on

198

Full Papers

Proceedings: NordiCHI 2010, October 16–20, 2010

present information and its future use as passive reminders; they have to be seen at the proper time by the user [6]. On the contrary, a calendar is a way to navigate the future (both personally and socially), it is an information management tool, a communication device (it provides awareness thanks to its visibility), it has a mnemonic function (reminder) and it provides a rationalized representation of time [4]. The calendar can be used to plan future events and to make decisions on how to organize the daily life.

retirement they find activities to fill their days, to socialize and to feel useful for the community. Activities and commitments make their life still meaningful while technology is perceived as threatening it [10, 2]. Literature on elderly and technology usually deals with home assistive technologies that aim to counterbalance physical impairments due to the age. To our knowledge, the only study involving elderly in the use of time management artifacts is Plaisant et al. [11]. This study focuses on the coordinating aspect of time managing, aimed at the whole family members and older adults were treated as special users. A noteworthy aspect that they reported is that maintenance of family calendars information in the majority of cases is up to more than one person, but when just a single person takes care of it, usually it is a woman.

In general, people still prefer paper to technological devices because of some of its availability (it is almost everywhere); portability (it can be brought everywhere) [8]; reliability (it does not run out of power); accessibility (anyone can use it) [8]; user-friendliness (it makes it easy to take notes when speaking on the phone) [5]; lack of structure (anyone can sketch on it in a way that suits his needs) [5]. Yet, paper can fail us by becoming misplaced, unintelligible, or ignored [6]. Indeed, the biggest contribution that digital technology has brought to time management until now is notification functionality. This function transforms a passive note into an active reminder [6]. On the other hand, digital calendars some disadvantages: in particular they must be powered up to display the content of the reminder; they are bulky and require too much time to enter information [6].

Forlizzi et al. [2] give important design guidelines based on elderly scale of values. Their research shows that elderly are constantly on the lookout for sense of self, independence and dignity; this affects how they use products. In the same way, Hirsch et al. [12] found that elders’ first aim is the maintenance of a good quality of life. Two main factors affect quality of life: independence and engagement. Between these two factors there is a special trade off to keep a stable equilibrium: as functional ability (and relative independency) decreases, connections to other people and engagement become exceedingly important in determining the individual wellness. A special matter in this area is users’ perception of their own abilities. Often perception is out of step with their actual capabilities. This can include an elderly operating at a level below his capacities, causing him to be fearful of attempting relatively safe tasks. Conversely, the elderly may overestimate her own ability, causing him to undertake risky tasks. This highlights that old people’s attitudes must be taken into consideration in design, in order to maximize acceptability.

Time management is often a social task. Organizing life in the immediate future may require information exchange with other people. Shared organizational systems turned out to be good for coordinating with other people in both domestic and work context because they share responsibility among people who use them [9, 5]. Indeed, the functionality of shared use relies on members’ compliance with coordination protocols [4]. Taylor and Swan [5] have shown how paper lists can help a mother to organize household activities and tasks as well as her children to grow in responsibility and to test the family order. On the contrary, Brzozowski et al. [9] have found that by using GCS (Group Calendaring Systems) in the working field the visibility of group members’ availability exerts a form of social pressure to encourage compromise and honesty but it can also raise problems relative to privacy and peer judgment about time allocation [3].

USER STUDY

The involvement of elderly people in a user research study greatly challenges the applicability of traditional methods such as in-situ interviews, shadowing or ethnography because of the peculiar psychological and ethical issues that enter into play [13, 14, 15]. Research in these contexts is often regarded as not merely difficult but often inappropriate and intrusive [4]: several researchers have noted the difficulty of obtaining the consent for in-home observations [16].

Other studies highlighted the emotional, reflective and narrative dimensions of time management tools [5, 8]. Sometimes people annotate events after they have happened in order to keep memory of them and to see them in a past perspective, often addressing to the emotional sphere [8].

For these reasons, we have adopted a comprehensive approach to motivate the participation of older people in the design team and to establish a long-term relationship with the other elders of the group. This approach is based on the work carried out by [13] and encompasses: (i) the endorsement by, and the direct involvement in the project of, local authorities and associations of older people with the role of mediators and guarantors; (ii) the establishment and nurturing of the sense of belonging to a pioneering

In digital calendars, personal signs and contextual marks for emphasizing certain entries reflect the emotional relation between entries and users. Yet, the sense of emotion embodied in material artifacts, such as paper, which is often missed by the technological tools [5]. Literature on the management of time by elderly people is too huge to be summarized here. An important aspect to mention is that elderly like having a busy life, after 2

199

Full Papers

Proceedings: NordiCHI 2010, October 16–20, 2010

group, with emphasis on the value of both individual and group contributions.

daily routines and were easily integrated in the everyday activities of the center.

Following these principles, we have involved in the study a day center for senior citizen located in the city of Trento in Northern Italy. The staff of the center has established during the years a well-consolidated relationship with the attendants and has greatly contributed to the creation of a trusted atmosphere between participants and researchers.

While a structured interview has a pre-defined, limited set of questions, a semi-structured interview is flexible, allowing new questions to be brought up during the interview as a result of what the interviewee says [17]. The interviewer has a framework of themes to explore. This framework is generally a grouping of topics and questions that the interviewer can ask in different ways to different participants.

The study was articulated in four phases: (i) Preliminary group meeting, (ii) Individual Semi-structured Interviews, (iii) Contextual Interviews and (iv) Final group meeting.

An important aspect of the semi-structured interview is that questions are asked when the interviewer feels it is appropriate to ask them. Elderly people could consider some questions as inappropriate or intrusive; therefore, it is important to adapt the interview structure and the questions to the different subjects’ personality and sensibility in order to do not create conflicts. Indeed, interview sessions were also an occasion to establish and consolidate a trusted relationship with participants.

Participants

The participants were 19 older people (16 women and 3 men; age range 60-87; mean 75) living in the urban area. All of them are fairly healthy and autonomous people; 14 live alone (at the national level: 77.9% of older adults live alone), one live with his son and the 4 are married. The gender asymmetry reflects the gender distribution in the center.

In our case, the framework was focused on the artifacts and the practices that people adopt to make use of time, with a particular attention to the emerging cultural and emotional values. In particular, it was articulated in four relevant dimensions of use of artifacts: planning, remembering, coordinating and reflecting.

People attending the day center are involved both in day center activities (for example, the creation of hand-made toys for charity sales, volunteer work, support to the staff in the organization of events and so on) and personal activities related to their family (such as taking care of grandchildren). In general, these subjects look for a social and active life and strive for having lot of commitments during the day.

The dimensions of planning referred to how people use artifacts to plan future events and the strategies to scheduling them. The dimension of remembering had to do with what artifacts and practices people adopt to remember things to do (i.e. call a friend, payments), appointments and other kinds of up-coming events (i.e. travels). The dimension of coordination investigates the social dimensions of the artifacts that elderly people use to manage their life. Finally, the dimension of reflection investigates the use of artifacts and the practices by which people look retrospectively to their life.

Preliminary Group Meeting

In the first encounter with the researchers both the staff of the day center and several of the older adults that regularly attend the center were present: by creating a safe and protected social context, reducing the individual involvement and mitigating the fear to be exposed to a novel situation, the participation of people was favored. The staff of the center presented the main objectives of the study and stressed the importance of belonging to a group of people that could play a critical role in the development of new technology aimed to improve the elderly people’s quality of life. Then, the researchers explained the main phases of the study, detailing what kind of participation was required in each different phase. At the end of the meeting, the staff and the researcher had a very short conversation with the people interested in continuing the study and the appointments for the second phase of the study were scheduled.

Beyond the understanding of the functional role of artifacts in mediating the activity of planning, remembering, coordinating and reflecting we also wanted to investigate the emotional and the cultural values tied to the practices of use. Contextual Interviews

A contextual interview [18] is an interview conducted in the same environment which is the focus on the interested for the research. In our case, the main goal of the contextual interviews was to observe the role of the artifacts, in the natural context of use. Contextual interviews may help revealing reveal latent and tacit cultural values that are difficult to verbalize and report during an standard interview because artifacts can be shown rather than simply described and practiced can be enacted rather than told.

Semi-Structured Interviews

The second phase of the study consisted in individual semistructured interviews conducted in the daily center. The reason to conduct the interviews in the center rather than in the homes is because that the participants know the center very well and perceive it as familiar and comfortable place; furthermore, the interview sessions became part of their

The contextual interviews were mainly used to refine the results of the data collected during in the previous phase.

200

Full Papers

Proceedings: NordiCHI 2010, October 16–20, 2010

The results of the semi-structured interviews have been used as a lens to conduct the contextual interviews. We asked people to show off the artifacts they use to organize their life and to describe related practices. By directly observing people using artifacts, we had the opportunity to observe the context where artifacts were located, how people mark the different events in a calendar or the different post-it (colors, size, materials) they use to jotting down a reminder. In this way, it was possible to elicit the cultural, emotional and personal values related to the practices of use of the artifacts.

daily life of elderly people. It is not common that a routine activity is forgotten. Furthermore, elderly people routines are very flexible: failing in performing a routine activity does not have any major consequence. Time management becomes critical when they have to remember future extraordinary events, such as an appointment with the hairdresser or esthetician, physical examinations, meals or trips with other people. These events are deviations from habits and could be easy forgotten. Moreover, failing in attending such events could result in serious consequences: in general, these events are appointments with professionals or public officers and have a very rigid social and temporal structure.

Final meeting

In the final meeting, the results of the study was presented to the older people that participated in the study and the staff of the center.

Finally, deadlines, activities that have to be completed at a given time, are the most peremptory events. However, they are not felt as a problem because our subjects’ deadline are usually bills payments almost all of them have their bills automatically paid. A special case of deadlines are medication prescriptions that are usually dealt with ad-hoc strategies.

Data Collection and Analysis

We conducted 19 semi-structured interviews and 3 contextual interviews in the homes; each one lasted in average 18 minutes for the interviews in the daily center and 30 for the interviews at home. Each interview was audio-recorded and transcribed. During the contextual interviews, we took several pictures to document the use of artifact..

As we will discuss in the next session, routines do not require specific time management activities but what elderly find really difficult to deal with is a sudden change in their habits. For this reason, extraordinary events like appointments and deadlines require a particular attention. In the next session, we will describe several practices that elderly people have spontaneously elaborated to deal with the limits of their perspective memory.

The semi-structured interviews were analyzed by a grounded theory approach. Three researchers independently codified the transcriptions and identified the main emerging categories. The three sets of findings were then reconciled and consolidated as a list of strategies and practices use by the older adults in our group to manage their time, and in particular to plan future events, to remember upcoming events, to reflect on the past experiences and to coordinate with others. Furthermore, a specific attention has been paid to the role and the proprieties of the artifacts involved in these practices and to cultural and emotional values relate to their use. Some less clear aspects were further analyzed with the contextual interviews by focusing the interviews on those specific aspects.

Practices on planning

Planning is mainly related to the scheduling of extraordinary events, such as physical examination, personal care (i.e. haircuts, pedicure) or visiting friends and relatives. As many interviewees pointed out, it is not a critical issue in time management: “I don’t need to plan. I’m not busy. I mean…I really know when I’m available. I’m available every afternoon but Thursday.”

DISCUSSION OF FINDINGS

Planning is not perceived as a critical issue for several reasons. Extraordinary events are not so frequent and daily routines ,which occupy most of the time in elderly life, are very well-consolidated in memory, elderly perfectly know when they are available and will easily avoid any future conflict with extraordinary events. Also elderly routines are very flexible events that that can be changed or modified without any major complication.

How Elderly use time

We have identified three main kinds of events that need to be accommodated in the life of our subjects: routines, extraordinary events and deadlines. Our subjects daily life is mostly characterized by the presence of well-consolidated and regular routines. In general, the first hours of the morning are dedicated to personal care and to house care, in some case also to worshipping. The rest of the morning and the early afternoon are usually dedicated to more social and public activities as shopping, meeting friends or volunteering or recreational activities at the day center (i.e. ceramics, painting courses, etc) and so on Usually routines do not present particular problems in terms of time management. Once they have been defined, they create a rhythm of in the

Even when the elderly people life is full of arrangements, the emergence of future conflicts is unlikely: “Yes, I have many arrangements but I’m not obliged to attend them. I’m old and I like to be free and relaxed.” In general, our subjects do not use external support for planning (only two ladies with a very busy life mentioned the use of calendars and agendas). 4

201

Full Papers

Proceedings: NordiCHI 2010, October 16–20, 2010

Practices on remembering

activity. Furthermore, the information recorded in the notes is continuously presented to individuals and their memory is implicitly reinforced day by day.

Differently from planning, remembering up-coming extraordinary events constitutes a critical time management task for older adults: if a considerable amount of time passes from when the appointment is fixed to when it should be attended, it could be very difficult to retain the plan in the prospective memory.

Embedding reminders into daily routines: another practice for maintaining a general awareness of approaching future events is to arrange notes in the physical environment so that they directly interfere with daily routines. For example: “I have a list of what I have to do in the next days in the bathroom…right there, in front of the WC. So, you know, every morning, when I wake up…I can check what I have to do”. Or the following: “I have a wall calendar near the TV. I watch TV every afternoon, after lunch. I mark important things with a red circle. So I can see them from distance. When there are TV ads, I give a look at the calendar…”

This “remembering task” can be articulated in three different sub-tasks: recording, maintaining and recalling. Recording refers to the moment when the date, the time and the place of a specific future event are jotted down information (usually on on a piece of paper). This information need to be stored for a certain period (for example, in the form of a note attached on the fridge) and then recalled at the appropriate time to prompt the intended action. In order to deal with these tasks, elderly people have elaborated different strategies none of them based on the use of digital artifact.

Creating a new routine: several subjects do not to store notes in visible place or to embed them in daily routine, rather they invent new routines to daily check what they have to do in the next days.

Most of our subjects record the information about future events using small sheets of papers, such as Post-It, notebooks pages, etc. In general, they take a note of the future events, by writing down the date, the time and the place. In several cases, a referring person (such as the doctor or the hairdresser) writes the note and hand it to them.

For example, a lady said that she collects all the notes in a drawer bedside the table and every night, before sleeping she controls if she has to do something the next day. Another lady adopt the same strategy collecting the notes in a pocket of her wallet. Moving information from the background to the foreground: a more complex strategy is to move the notes in different location to mark the approaching of an event. For example, “I have a note for every appointment. I stick my notes on a painting near the entrance door […]. Everything is there: appointments; programs; medical examinations and so on. […] I regularly look at them and when I know I have an up-coming appointment, I move the note from the painting to the entrance door so that the morning when I open the door I see it.” (see figure 2).

If the event is far away in the future, a common practice is to transcribe the note content in the calendar or to stick the note near the appropriate date. In few cases people use a sort of dashboard: they create lists of future events and delete them once they have been attended. Another strategy consists in having a dedicated place in the home where to collect such notes. Transcribing or collecting notes in more structured and permanent places reduce the possibility of losing them and allow referring when it is needed.

This excerpt points out an important issue: remembering the time and date of future events is not sufficient to trig an appropriate course of action at the right moment. Routines are really well consolidated scripts in elderly life and they could interfere with the execution of an unusual action schema. This phenomenon is called “strong habit intrusion” [20] and the resulting error is called “capture error” [21] that happens when a frequently used schemas (like “going to work”) take over from a less frequent, less activated schemas (like “visiting a friend”). In order to deal with these kinds of failures of the prospective memory, participants have elaborated different strategies to move the peripheral notes from the background to the foreground at the right moment. The subject mentioned before uses the strategy of moving the note from the showcase to the door so that she cannot avoid looking at them before going out. Another participant is used to transcribe a note from the calendar (used as a long-term storage) and to leave this near the alarm clock.

The effectiveness of a note to remember an event depends on its being noticed at the right time, not too early and not too late. Several strategies emerged from our analysis.

Figure 1. Notes on the fridge and on a dashboard in a living room.

Peripheral visibility: Wall calendars, dashboards and place dedicated to note collection in general, are usually located in the kitchen or in the entrance of elderly people houses (see figure 1). These locations guarantee high visibility: elderly people spend a lot of time in the kitchen and the entrance is a natural home passage. The notes remain in the background and do not interfere with domestic daily

Keeping the note until the end: as noticed before, in most of the cases the note is not written by the older adults but by

202

Full Papers

Proceedings: NordiCHI 2010, October 16–20, 2010

some other person and handed them. In many cases, we fund that elderly people keep the original note even when they transcribe them on the calendar.

emotional and narrative connotation. Another lady uses the calendar more as a sort of micro-diary than as a way of remembering future events: every night, before sleeping, she records with a list of keywords the most important facts of the day, personal thoughts, feelings and emotions. During the contextual interviews, the lady told us that she never throws away a calendar; she has the calendars of the last 12 years and she is used to look at them to remember what she did in the past. “When I’m alone at home, I really like to look at old calendar and say: oh, I did it, oh I did that, oh it was really snowing…”. During the contextual interview, we noted that there were a couple of weeks without any notes. She told us that it was a very sad period of her life and she did not do anything interesting. By discussing with the lady, we elaborated the idea that the calendar works as a sort of dynamic representation of personal moods and daily rhythms; it is a sort of wall display that stimulates self-reflection and contributes to create a self-understanding. By looking at the calendar, which is in a very prominent position in the dining room, this lady has an immediate representation of how she is in the current month.

Elderly people experiences some difficulties with working memory and having the note with all the information written down even if they transcribed them make them feel safer and confident.

Another subject has four calendars, a showcase full of notes, an agenda close to the phone and an agenda in her bag. All these artifacts have not a functional specification: information about future events is distributed across them in a redundant way without following any specific pattern. This practice is a clear example of a psychological need more than an organizational strategy: the calendar, the dashboard and showcases with notes are not only peripheral display that contributes to maintain a contextual awareness of future events, they also create a pervasive context that communicate a feeling of “a life full of arrangements” (see figure 3). As already discussed in literature, being busy and have the perception that time is not wasted is one of the most important value that characterize elderly life.

Figure 2. Notes stored on a dashboard (on the left) and then moved on the door. Practices on Coordination

Most of the interviewed people live alone and are autonomous, coordination with other family member is not a crucial issue. Even when daily routines are shared with other members of the elderly community, the coordination is facilitated by informal social context and it has never been mentioned as problematic in the interviews. However, we have identified an interesting coordination practice related to the moment of taking note of the relevant future event information. As already said, during the contextual interviews we have observed that elderly people often does not personally write the notes: it is the doctor, the day center staff or other people involved in the event that does so. Older adults have often hearing problems and some time also some difficulties in writing because of fine motor limitations, this practice therefore reduces the probability of transcription mistakes and misunderstandings. Practices on Reflection

Figure 3. Calendars with annotation and post-its.

Consistently to the literature of calendar usage, our subjects use the artifacts for managing future events, such as calendar, notes, dashboards also to support reflection on past experiences. For example, a lady mentioned the fact that she decides to plan an appointment with the hairdresser only after a month from the last time she had a haircut.

VALUES

Values are an important dimension that must be taken into consideration when designing for elderly [2]: values shape the practice of use, by directly influencing people attitudes towards products, both in terms of behaviors and emotions. In this paper we have specifically investigated the values embedded in time management practices and we have

However, the most interesting aspect of this topic is that the reconsideration of the past events could have also an 6

203

Full Papers

Proceedings: NordiCHI 2010, October 16–20, 2010

described how these values affect the way people use time management artifacts.

some cases they could also refuse to use them, even if they could positively support their independence.

In our analysis, three values emerged as crucial in relation to time management: the need of proving to themselves and other that they are not wasting their time; the fear of memory loss as an evidence of cognitive decline and need of assessing themselves.

It is important to note that the practices previously described about the physical arrangements in the environments of notes are not in contradiction with this value. The people who adopt such practice declared to have a very busy life and they really stressed this point. In this case, having a busy life not only justify the use of such artifacts but also make their pervasive use an added value. In both cases the ultimate purpose is to maintain and elaborate a representation of the self that is gratifying and acceptable from a cultural and social point of view.

Making a good use of time: social engagement, outdoor physical activities (such as mountain excursion, bicycling) and volunteer work are perceived as useful and rewarding activities that is important to carry out. Spending too much time watching TV, staying alone at home or eating preprepared food are associated with bad moods or negative feelings: “After lunch, I’m used to watch TV. However, after a while I really feel something that is pushing my back…it says me: you have to go out!” Another subject said “Before sleeping I’m used to look back at my day and I’m really happy when I see that I did many things, when I did a wonderful walk in the mountains, I’m really happy”

Assessing myself: a recurrent theme with our subjects is their need to continuously assess their memory skill, for example “I still have a good memory. I remember everything, even the birthday of my grandchildren. Sometime I say: tomorrow it should be the birthday of Giovanni. Then I go to see the calendar - where I wrote down the birthday dates of all my grandchildren -…and I’m always right!” or a similar “I keep my notes in my wallet. I don’t need them. I remember things by myself. I use these notes only to see if I’m right”

From our interviews, it clearly emerges that elderly people like to be busy, to have a day full of arrangements and an intense social life. This confirms many studies that point out the importance for elderly people of being engaged with the social and the cultural world.

This value is deeply related to what we have described in the previous section and to the importance that elderly people attribute to memory skills in the construction of the self. As it possible to observe in the excerpts, this subtle affects the way elderly make use of time management artifacts: they are used not only as tools to manage the upcoming future but they also support elderly people in their activity of constantly proving their memory skills.

This value influences how elderly people use artifacts for organizing their life. It is closely related to the practice of reflection. As already discussed, artifacts used for time management not only have a functional role: they are also used to support reflection about personal lifestyle. By looking to a showcase full of notes, most of our subjects elaborate the impression that they are actively living their life and still engaged with the “real” world.

DESIGN OPPORTUNITIES

By elaborating on the analysis previously presented, we have defined a set of design dimensions that are aimed to drive the development of time-management technology in the context of elderly life.

Luckily, I don’t need it: during one of the first semistructured interview, a subject answers to the question about what artifacts she uses to remember things: “luckily, I don’t need them. Not yet.”. This lady perceives artifacts for time management as evidences of memory problems; by saying she does not need artifacts for remembering things, she is asserting her cognitive well being.

Beyond functionality

When designing digital artifacts for time management in the context of elderly life, it may be useful to widen the design space and not considering only functional aspects but also the personal, emotional, narrative and leisure dimensions.

This theme emerged quite often in the analysis in a positive or in a negative form: some subjects were proud of their memory skills while some were feared the possibility of losing memory in the next years. Several contradicted themselves by neglecting the use of artifacts for remembering things to do and at the same time “confessing” to have a calendar for the appointments far away in the future.

In general, we have observed that time management artifacts have a double nature: on one side they are used to support functional task such as remembering upcoming events or planning future appointments; on the other side they enable elders in making sense of time, by stimulating personal reflections. In respect to this latter aspect, we can identify two different uses: artifacts used to support the elaboration of a narrative account of past and present experiences and artifacts used as a sort of “gaming platform” to test memory skills.

Since artifacts for time management are often associated with memory aids, they have also a symbolic value. Some people see them as the evidence of their diminished memory skills. This has an impact on practices of use: elderly people could try to minimize the use of such artifacts, to make them as less visible as possible and in

Narrative account: task management tools, such as notes, calendars or dashboards can also used as object to think with, in order to make sense of time and elaborate a

204

Full Papers

Proceedings: NordiCHI 2010, October 16–20, 2010

background to the background” we have describe how it is necessary to manipulate notes visibility in order to appropriately support the recalling process.

narrative and emotional account of personal life. From a design point of view, it could be interesting to investigate this dimension and to explore how to enrich the experience of use by working on the aesthetics of interaction [23] or by allowing people to integrate more personal contents.

In the section “luckily, I don’t need it” we have described the symbolic value of artifacts used for time management: they could be perceived as evidences of memory impairments. As a consequence elderly may tend to minimize their use or to make them invisible.

Gaming platform: time management artifacts could be used as a gaming platform to test personal memory skills. This constitutes an interesting design opportunity to change people attitude toward time management artifacts. Rather than being perceived as passive memory aids which could have a negative value, time management artifacts could be interpreted as tools that actively support elderly in improving or maintaining memory skills.

Time management artifacts should be considered within these complex set of values and practices; the way they are concretely used and adopted critically depends on personal attitudes. From a design perspective it is important to develop flexible technology that allow elderly to configure them according to their individual preferences. In particular, in respect to this class of artifacts, it is important to consider the visibility of information related to future events as a resource. Elderly should be enabled to modify and configure such a resource by moving information from the background to foreground or even making them invisible.

Beyond the domestic space

The design space may also be enlarged beyond the domestic space by taking into consideration a variety of different context of use. The whole lifecycle of event management might be considered: since when the information related to a future event is defined and recorded to when it is stored on a more persistent support, to when the information related to the event are referred to prompt the appropriate course of actions at the right time to when such information are used to support the older adult in arriving at destination and attending the event.

Proactive devices

As we have discussed in section “Making a good use of time”, elderly people are continuously looking on how to occupy their time: indeed, programs (such as theater seasons) are often near to their showcases and calendars.

The interaction context varies along with the evolution of the time management task: during the recording of the information related to a future event, elderly could be in an office (i.e. in front of an bank employee or a member of the day center staff); the temporary storage and the recalling of a note generally occurs in the domestic environment; then elderly tend to keep the note with themselves until the end of the attended event (this is a completely mobile context where the note is used, for example, to support elderly in remembering the appointment address). In particular, in the recording phase it is important to support the coordination between the older adult and other person involved: as we have discussed in the “coordinating” session, in order to minimize the probability of misunderstanding the note is written down by the other person involved in the event and not directly by the older adult.

This practice constitutes a design opportunity. We can consider time management artifact not only as passive tools that elderly people use to organize their life but also as proactive tools that suggest opportunities and possibilities. Calendars, dashboards and even showcases can be considered as interfaces to a wide set of services: by using such a familiar interface, elderly people could have the opportunity to navigate their future event reminders together with the proposals of public or private institutions such as cinemas, theaters, local events and so on. Public institution could also use these artifacts to communicate with elderly customers and present relevant information such as periodic physical examinations in the context of their daily life.

The design of task management technology should therefore take into consideration how information related future events are recorded, manipulated and propagated across different media, physical and social contexts, and it should guarantee a fluid user experience across different spaces and situations.

CONCLUSION

In this paper, we presented the results of a qualitative research aimed at understanding the time management practices of older adults. In particular, it emerged that although a busy life is often considered an important value that characterize elderly life, time management tools can carry a stigma of a prosthesis.

Visibility / Invisibility

In the section “peripheral visibility” we have described how elderly make arrangements of notes in the physical environment to support a peripheral awareness of future events they have to attend. In the “reflection” section we have discussed the role of such arrangements beyond its pure functional role (see also “narrative account”). Furthermore, in the section “moving information from the

Although the user group is limited in size, gender and geographical distribution, we feel that the issues emerged might have a broader validity and may contribute to a greater understanding on how to design for older adults.

8

205

Full Papers

Proceedings: NordiCHI 2010, October 16–20, 2010

ACKNOWLEDGMENTS

11.Plaisant, C., Bederson, B. B., Clamage, A., Hutchinson, H., Druin, A., Shared Family. Calendars: Promoting Symmetry and Accessibility, ACM Transactions on Computer-Human. Interaction, 13, 3 (2006) 313-346

The work of this paper has been funded by the Internet of Services Joint Research Project at FBK. We would like to thank the “Centro Belenzani” day center in Trento and the people that participated in the study.

12.Hirsch, T., Forlizzi, J., Hyder, E., Goetz, J., Kurtz, C., and Stroback, J. 2000. The ELDer project: social, emotional, and environmental factors in the design of eldercare technologies. In Proceedings on the 2000 Conference on Universal Usability. Arlington, Virginia, 2000.

REFERENCES

1. McCreadie C and Tinker A. (2005) The acceptability of assistive technology to older people. Ageing and Society, 25, 1, pp 91-110. 2. Forlizzi, J., DiSalvo, C., and Gemperle, F. (2004). Assistive Robotics and an Ecology of Elders Living Independently in Their Homes. Journal of HCI Special Issue on Human-Robot Interaction, V19 N1/2, January, 2004.

13. Leonardi C., Mennecozzi C., Not E., Pianesi F., Gennai F., Cristoforetti A. Knocking on Elders’ Door Investigating the Functional and Emotional Geography of their Domestic Space. In Proceedings of ACM CHI 2009. April 2009, Boston, Mass.

3. Palen L., (1999) Social, individual and technological issues for groupware calendar systems, Proceedings of the SIGCHI conference on Human factors in computing systems: the CHI is the limit, p.17-24, May 15-20, 1999, Pittsburgh, Pennsylvania, United States

14.Eisma, R., Dickinson, A., Goodman, J., Syme, A., Tiwari, L. and Newell, A.F. Early user involvement in the development of information technology related products for older people. International Journal Universal Access in the Information Society, 3(2), 2004, Lin1-140 Academia Press.

4. Crabtree, A., Hemmings, T., Rodden, T., and Mariani, J. 2003. Informing the development of calendar systems for domestic use. In Proceedings of the Eighth Conference on European Conference on Computer Supported Cooperative Work (Helsinki, Finland, September 14 - 18, 2003). K. Kuutti, E. H. Karsten, G. Fitzpatrick, P. Dourish, and K. Schmidt, Eds. ECSCW. Kluwer Academic Publishers, Norwell, MA, 119-138.

15.Marti, P., Giusti, L., Crafting Sensorial Stimulation and Social Exchange in Dementia Care, Proceedings of the 6th Conference of the International Society for Gerontechnology - ISG08, Pisa, Italy, May 20 - 23, 2008. 16.Dickinson, A., Goodman, J., Syme, A., Eisma ,R., Tiwari, L., Mival, O. and Newell A. Domesticating Technology: In-home requirements gathering with frail older people. Proc. of 10th International Conference on Human - Computer Interaction HCI (2003), 827-831 the Home. Housing Studies, Vol. 19, No. 5 (2005) 745– 763.

5. Taylor, A. S. and Swan, L. (2004) List making in the home. Computer Supported Collaborative Work, CSCW '04. Chicago, IL, 542-545 6. Lin, M., Lutters, W. G., and Kim, T. S. 2004. Understanding the Micronote Lifecycle: Improving Mobile Support for Informal Note Taking. In Proc. CHI '04. ACM Press, 687-694

17.Lindlof T. R., Taylor B. C., (1995) Qualitative communication research methods. Thousand Oaks, CA: Sage Publications, p. 195.

7. Payne, S. J. Understanding calendar use. HumanComputer Interaction 8, 2 (1993), 83--100.

18.Hugh Beyer, Karen Holtzblatt, (1998) Contextual design: defining customer-centered systems, Morgan Kaufmann Publishers Inc., San Francisco, CA.

8. Tomitsch, M., Grechenig, T. & Wascher, P. (2006). Personal and Private Calendar Interfaces support Private Patterns: e.g. Diaries, Relations, Emotional Expressions. In Proceedings of the Nordic conference on Humancomputer interaction

19.Glaser, B. G. and Strauss, A. L. (1967), The Discovery of Grounded Theory: Strategies for Qualitative Research. New York, NY: Aldine.

9. M. Brzozowski, K. Carattini, S. R. Klemmer, P. Mihelich,. J. Hu, and A. Y. Ng. groupTime: preferencebased group scheduling. In Proc. of CHI'06, pages 1047–1056, 2006

20.Reason, J. (1990). Human error. Cambridge University Press 21.Norman, D. A. (1981). Categorization of action slips. Psychological Review, 88, 1-15.

10.Doimo, L. A. ; Derntl, A. M. , (2006) Use of time in elderly people´s daily lives: an indicator method for lifestyle in old age.. BRAZILIAN JOURNAL OF GERIATRICS AND GERONTOLOGY, v. 9, p. 37-53.

22. Turkle, S. (2007) Evocative Objects --- Things we think with, MIT Press. 23. Rullo, A. 2008. The soft qualities of interaction. ACM Trans. Comput.-Hum. Interact. 15, 4 (Nov. 2008), 1-25.

206

Full Papers

Proceedings: NordiCHI 2010, October 16–20, 2010

Studying Mobile Context-aware Social Services in the Wild Paul Holleis, Matthias Wagner

Sebastian Böhm

Johan Koolwaaij

DOCOMO Euro-Labs Landsberger Str. 312 80687 Munich, Germany {holleis, wagner}@docomolab-euro.com

net mobile AG Zollhof 17 40221 Düsseldorf, Germany [email protected]

Novay Brouwerijstraat 1 7523 XC Enschede, NL [email protected]

ABSTRACT

involves sensible context data has not been well exploited yet. In addition to technical aspects in the implementation of context-aware social services like IYOUIT, our research focuses on the usability, acceptance and market potential of such systems in practice. In this paper, we report on the results of a longitudinal user trial with 19 users from 6 countries over one month. We extend and discuss our longterm experiences made by releasing IYOUIT to the public since more than one year with the findings from this study.

We have implemented and evaluated IYOUIT, a contextaware application for the mobile phone that promotes a digital lifestyle, sharing, and life-logging approach for people on the go. The service incorporates context management technology to abstract data about and around the user into meaningful interpretations of the user‟s digital trace in the real world. Complementary to the public release of our service, we have conducted a longitudinal field study with 19 users for a period of one month. In this paper, we present findings from this coordinated user trial and provide researchers with advice on the design and implementation of similar systems.

After some related work, we present a system overview, followed by the setup and methodology of the study. After an in-depth analysis of the conducted user trial we summarize the results and derive guidelines for the design of context-aware mobile applications.

Author Keywords

Mobile Services, Context Awareness, Social Networking.

RELATED WORK

ACM Classification Keywords

IYOUIT comprises a broad range of concepts, ideas, and services into one system with various interfaces. To the best of our knowledge, no other system provides a comparable diversity of context-aware social services, also including an extensive mobile client application. A good overview on context-aware mobile computing research until the year 2000 is given by Chen and Kotz [5]. Interestingly, of the 19 projects they analyzed, only two used context information other than location (and the current time). These two added some notion of activity mostly recognized by specifically deployed sensors. In contrast, IYOUIT uses available phone sensors to sense and deduce a great variety of context information. Hong et al. [13] further report on general context-aware systems published until 2007. They categorize them according to five layers: concept and research, network, middleware, user interface, and application layer with various sub-categories. The diversity and rich feature set of IYOUIT is exemplified by the fact that it covers a total of 8 of the 19 categories of the first four layers and has elements from half of the application areas (information systems, communication systems, and web service). The authors also show that very few research projects provide a fully functional and publically available application.

H.5. Information interfaces and presentation (e.g., HCI): Miscellaneous. H.1.2 User/Machine Systems INTRODUCTION

IYOUIT is a mobile application that allows users to automatically collect context information centered on visited places and interactions with other people. It aims at making it easy to collect such data on a standard phone and facilitates instant and light-hearted sharing of personal experiences within communities together with rich contextual tagging for everyday life. In combining and aggregating context sources, IYOUIT derives a rich portfolio of personal information to be shared with others on the Web and on the mobile phone. By hooking up to Web 2.0 services like Flickr, Facebook, and Twitter, IYOUIT extends sharing experiences to social online communities commonly regarded as being widely adopted for the exchange of personal information. However, adding a mobile dimension to social communities that

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. NordiCHI 2010, October 16–20, 2010, Reykjavik, Iceland. Copyright 2010 ACM ISBN: 978-1-60558-934-3...$5.00.

In the last years, many academic and commercial social platform projects have been created. Many of them are specific to a particular type of information. For example, Picasa, Snapfish, and Panoramio concentrate on sharing images, Google Latitude, Shizzow, and Plazes help people share their current location, and EveryTrail, Dopplr, and

207

Full Papers

Proceedings: NordiCHI 2010, October 16–20, 2010

and mash-ups with other services and applications. The framework consists of two main parts. Management components, e.g., ensure secure authentication, implement finegrained access control, and allow the definition and exploitation of context ontologies. On the other hand, context provider components wrap basic context data sources and can implement aggregations and abstractions. The framework has been designed for an open integration with 3rd party services and is leveraging Semantic Web technology in various places. The platform can be accessed through a web platform and a mobile client application.

TripIt are used mainly for sharing travel plans. Obviously, several platforms exist that allow people to share messages, updates, and stories. Examples include Twitter, blogging, and other community platforms (see below) for short messages and status updates. As IYOUIT incorporates all these features, it can be used by a single person as a notebook and „digital life recorder‟ for his or her history of location and activity, encounters with others, etc. However, its true potential is unfolded with a community of users. On the same grounds, several social community platforms and applications have emerged. Most of them are specialized to some domain (e.g. LinkedIn for business; StudiVZ for education), restricted to some type of content (e.g. Flixster to movies; Last.fm to music), or are only available or popular within a certain region (e.g. Mixi in Japan; Qzone in China). Similar to IYOUIT, many of those have a web frontend and a mobile client application. However, especially the most popular platforms worldwide such as Facebook, Myspace, Orkut, and QZone do not exploit the rich feature-set of current smartphones and merely provide a simplified interface of their web portals. IYOUIT directly uses phone capabilities (see, e.g. [25] for a similar approach) and the users‟ mobility to create profiles and offer multiple services described below. In order to evaluate our system under real conditions and provide results with respect to its use, we chose a longitudinal field trial. Controlled lab trials or artificial test environments such as the one used for the evaluation of the health-monitoring application by Morla and Davies [15] cannot fully satisfy our needs as we wanted to focus less on the mere usability of the system and more on how it is perceived and used in the real world. There have in fact been very few studies examining the use of such powerful context-aware platforms with web-based and mobile interfaces. Most existing studies e.g. by Tungare et al. [23] concentrate on specific aspects or ways of applying the system or, like O‟Grady et al. [17], focus very much on the usability of the specific mobile application and its interface. In contrast, we were more interested in how users accept and use our system. A similar goal has been followed by Jacucci et al. [26]. However, their system is targeted at a quite specific usage scenario (communication at large events) and was trialed in very specific settings over a short period of time.

Figure 1: The IYOUIT component framework and connected 3rd party services. The Mobile Client

IYOUIT has been designed to run 24/7 in the background on modern phones. Our current implementation supports all phones of Nokia‟s S60 platform. Figure 2 shows some screenshots of the mobile client‟s interface. Each tab either displays a certain type of context information or accumulates various pieces of information in a contextual overview that helps assessing the situation of a user at a given point in time: Me: the home tab of IYOUIT and the standard entry point. It summarizes recently spotted information in the user‟s proximity, highlights the latest presence status, and provides pointers to more detailed information in other tabs.

THE IYOUIT PLATFORM AND SYSTEM

In order to cope with the plurality of context, IYOUIT is built upon a Context Management Framework (CMF) which hosts and combines various services and data sources [3]. As a prototype from our labs, IYOUIT is available free of charge. In addition to a mobile client available for Nokia S60 phones, we have implemented our own IYOUIT Web portal1 and open APIs to support 3rd party developments 1

Buddy: shows the real-time context of all social contacts, including their location, status updates, latest activities, shared photos, and more. It offers different views on context, various sorting orders and context-dependent groups. Places: summarizes the user‟s current whereabouts. The client uses GPS coordinates when available. Otherwise, the location is fixed using cell IDs or exploits existing location

http://www.iyouit.eu

208

Full Papers

Proceedings: NordiCHI 2010, October 16–20, 2010

Figure 2: Various tabs of IYOUIT’s mobile client application.

information stored by the IYOUIT user community (e.g. at WiFi hotspots or fixed Bluetooth devices). Using statistical learning and clustering methods based on [16], places (abstractions from exact location values) are generated. These represent frequently visited places and map to basic semantic concepts such as the user‟s „home‟ or „office‟.

Initially, the level of accessible information is defined by the manually specified relationship to a contact (e.g. „wife‟, „colleague‟). In addition, it can be controlled at various levels (e.g. for all location updates or a single photo). Further tabs exist for displaying local information on a map, for showing GPS details and statistical overviews of the personal context history. Automatic context tagging is also available for books and products (via bar code scans) as well as sound and video recordings.

Experience: allows setting IM-compliant presence attributes like activity, mood or availability. Small text notes are automatically complemented with personal context information to be posted, e.g., on Twitter. Using built-in accelerometers measuring movement and gestures, we can automatically set states such as „sleeping‟ or „walking‟. Further (semi-)automatic methods are active research [11].

The Web Portal

While the primary personal use of IYOUIT is on the mobile phone, the application also materializes on the Web in its own portal. Daily hits on the IYOUIT Web portal currently average 100.000+ requests.

Photo: added pictures are proactively tagged with available context data and can be uploaded to 3rd party services such as Flickr. Beyond widely used tags such as geographic information, IYOUIT adds semantic descriptions like nearby people with the corresponding social relationships, the current place, qualitative weather annotations, etc. As a consequence, managing and sharing photos becomes more intuitive. For instance one can ‟find all family pictures taken at the holiday cottage on a sunny day„ or ‟subscribe to photos captured by friends on the university campus‟.

The Web portal offers similar interfaces as the mobile client but can be more suitable for the display and configuration of complex data. It shows latest public observations and selected bits of collected context together with hints on how the community is evolving. Users can conveniently define social relationships and configure personal privacy policies. Furthermore, the portal offers a hub to various other online services. Users can thus decide to share their personal context with, for instance, Flickr, Twitter, Dopplr or YouTube. The IYOUIT portal also serves as test bed for selected features from our research lab. In an experimental engine, for instance, users can configure automatic context blogging or profiling.

Events: displays a timeline of recently observed significant changes in the user context. Here, entries to places of interest are listed together with social encounters, network information as well as activities observed, e.g., by the accelerometer sensor of the phone. Actions such as switching the phone profile from „outdoor‟ to „meeting‟, can be attached to places. Similarly, activities such as „working‟ may be bound to nearby people.

Given the particular focus on real-time sharing aspects of personal data in the user study, the web portal and its features to display context in an aggregated view over a longer period of time has been of minor interest in this trial.

Scan: collects technical observations in terms of scanned Bluetooth devices and spotted WLAN access points. Observed devices can be associated with buddies while scanned WLAN access points are geo-tagged. Both can be associated with rules to trigger proactive actions such as setting the user‟s experience or automatically subscribing to WLAN network access in a certain place.

Public IYOUIT Community

IYOUIT makes it relatively easy to gather and share personal context data. During the design and implementation of the system, our assumption has always been that people will appreciate such sharing features given their privacy is preserved and under their control. We have started to test this assumption by releasing the system for public and free use about two years ago. For this reason, IYOUIT has also been promoted and eventually awarded as part of the

The owner of information (the user whose context is being sensed) has full control about what is accessible by whom.

209

Full Papers

Proceedings: NordiCHI 2010, October 16–20, 2010

Mobile Experience at MobileHCI‟082. Since IYOUIT‟s public release mid 2008, more than 1.500 users from 50 countries registered with the service. More than 1.5 million location measurements were recorded, 10.000+ pictures were tagged and uploaded, and more than 55.000 local information items, such as weather forecasts, were requested. Albeit already of great value through direct feedback and observation, the global IYOUIT user community provides only limited contribution to a comprehensive and complete understanding of our system. To learn more about how mobile users perceive contextual tracking and sharing as well as other application features implemented in IYOUIT, a profound user study with a clear focus and a relatively controlled setting was indispensible.

collected feedback about the general concept, user experience and behavior, and application specific features and interfaces.

EVALUATION Figure 3: Demographics of the 19 study participants.

The study on which we report in this paper took place in 2009 and lasted four weeks. One main target of the study was to find details about people‟s acceptance of general context-aware sharing concepts and various special features within the IYOUIT system.

Final interviews: we also managed to have individual, faceto-face interviews with 14 participants to clarify some of their answers and to find reasons behind their judgment. Study Participants

Study Setup

We tried to recruit a variety of different user types from the existing user base and also encouraged additional persons who had not used IYOUIT before. In the end, we covered a range of different nationalities and age groups (26-65 with an average of 37 years, see Figure 3), 15 male, 4 female.

In the following, we briefly give an overview on the structure, methods, and tools we used. Initial tutorial: we met with all study participants to introduce the IYOUIT system, distribute compatible handsets if necessary and configure the application if not done already.

Table 1: Previous experiences and participation of the users.

Demographics: demographic information about participants was collected in this session with a short questionnaire.

Participants Characteristics Use of Facebook social platform Use of Flickr photo sharing Use of Twitter micro blogging Use of a location sharing platform

Triggering participation: as stated in Froehlich et al. [9], "participant compliance tends to decrease over time" and triggers can help to counter such trends. We provided weekly challenges and announced small prizes for those who would provide most constructive feedback. In fact, however, the consequences were very limited: only three persons attempted to address some of the tasks, and those only within the first two weeks. Thus, we can practically rule out any effect on the results of the study.

One half of the participants (47%) already knew IYOUIT (but only 21% for longer than 6 months before the study). We also aimed at a mixture of previous experiences with such platforms to study a potential correlation between the actual use of our system during the trial and previous experiences with similar services. Table 1 shows some of the characteristics.

Data Elicitation

During the study, we used three ways of gathering quantitative and qualitative data:

With respect to picture sharing, our sample mostly had experiences with Flickr or similar services. Regarding social community platforms, in total 17 systems have been in use by at least one participant. Among these, Facebook and LinkedIn were most prominent (mentioned >5 times).

Logging: the mobile IYOUIT client was fitted with additional routines to log and transmit information about actions and context experienced throughout the study in order to reinforce subjective opinions with actual behavior.

ANALYSIS

For this paper, we distilled results about four broad areas of interest and describe the instances from which we elicited the data: general use (usage data and personal feedback), context acquisition and sharing (photo sharing), context aggregation and abstraction (location and a concept called places), as well as social networking and privacy (connections to 3rd party sharing platforms and detailed treatment in the questionnaires).

Questionnaires: most subjective data has been gathered through questionnaires distributed by email as well as the IYOUIT Web portal. An initial questionnaire gathered demographic data and clarified previous experiences. After two weeks, a questionnaire checked people‟s participation and tried to spot trends. A final, elaborate questionnaire 2

Rate 58 % 53 % 32 % 21 %

http://mobilehci2008.telin.nl/?sub1=89

210

Full Papers

Proceedings: NordiCHI 2010, October 16–20, 2010

General Acceptance and Use

ly. Surprisingly however, and in contrast to what people indicated, the Experience Tab initiated more than twice as many interactions as the Location Tab. This indicates two aspects: the implementation and presentation of location information has been done in a way that needs little user interaction; and people are in fact interested in collecting and sharing more information than their position.

The application usage throughout the study was very high but varied largely between participants. As Figure 4 shows, the average uptime of the mobile client during the trial period was 38%. Given that these numbers assume a 24/7 usage, the measured average is actually surprisingly high.

Figure 4: Uptime of the mobile client application (avg. 38%). Figure 6: Types of context most important to users.

Although increased power consumption was mentioned as a major issue of the system by 47% of the participants, Figure 5 indicates hardly any correlation (r2 = 0.65) between problems due to power consumption and the actual runtime. The average number of times that people had the phone powered off because of an empty battery was only 4 times. In the end, battery problems seem to explain only 12% of the cases in which people had IYOUIT not running for some time. This is supported, e.g., by Froehlich et al. [9] who show that people most often recharge their phones considerably before the battery reaches a low level anyway. In addition, IYOUIT also offers several opportunities to conserve energy, e.g. cell triangulation instead of GPS for location estimation and an offline mode.

Still, location-based services are of high importance as the appearance of the Location, GPS, and Map tabs within the chart of Figure 7 proves. Less interesting seem to have been the possibilities to generate statistics (Statistics and Graphs) about past history of context of oneself or others. This correlates to the fact that no one described context history to be of top interest (Figure 6). However, this could potentially be improved with a better and more interesting user interface or more features such as comparing statistics across users. In fact, in the interviews, 43% pointed out that they would be interested in using IYOUIT as a life-logging application and a dedicated diary service was even requested as additional feature. This indicates a strong need for automatically generating and presenting context to the user in an appropriate way (cf. concept of places explained below). One interesting result from our final questionnaire was that people stated they learned habits of their buddies (2.9 on a Likert-scale from 0 to 4, σ = 1.0): “I just found out that a colleague’s hobby is airplane spotting! This buddy-based sharing is definitely a worthwhile feature for the application.” Japanese, male, 34 years.

Figure 5: Number of empty battery occurrences per users.

Context Acquisition and Sharing – Photos

To see which context categories are of most interest to users, we had the participants specify the top three most important types of information (cf. Figure 6). Location and pictures head the list followed by weather information. Curiously, people‟s activities and mood have not been mentioned very often. One advantage of running this study is that we were able to compare these statements with the actual behavior within the system.

The IYOUIT system provides an excellent framework for photo sharing. Pictures taken with the mobile client are automatically augmented with context information such as location, activity, or mood gathered by the application. Additionally, the user can manually associate tags and descriptions with photos. In conjunction with querying and grouping mechanisms, all this information helps in later retrieving specific pictures (using, e.g., information about the weather conditions or nearby people). As IYOUIT can be directly and seamlessly connected to 3rd party services such as Flickr and Facebook, features from these systems like Flickr‟s photo sets capability can directly be exploited.

Figure 7 provides more details about the most frequent use of the IYOUIT client. As expected, people were attentively following the context of their buddies. Also the Photo Tab for taking, tagging, and sharing pictures was used frequent-

211

Full Papers

Proceedings: NordiCHI 2010, October 16–20, 2010

Figure 7: Relative amount of use of the client’s different tabs.

Figure 9: Photos with manual tags and transferred to Flickr.

During the trial month, 540 photos were uploaded allowing us to derive interesting insights into the way people share and describe pictures. First, the number of uploads is not related to the uptime of the client. Second, as expected, our current system could not fully challenge dedicated picture viewing applications such as Flickr as it was not designed for this purpose. However, its feature set was rich enough to be attractive and often used. Automated tagging and the transfer to Flickr were very much appreciated. This can be seen in Figure 9 that shows the percentage of photos with manual tags and descriptions. The right bars show only those users that have manually tagged or described at least one picture during the month. The numbers are unexpectedly high as adding descriptions requires some effort: use of the menu and entering text on the phone. This indicates that adding information to pictures is important and IYOUIT can help greatly by automatically adding tags and a title 3.

One main issue that arose within the feedback through the questionnaires and interviews was that many people prefer their digital camera to the mobile phone camera, e.g. because of image quality, flash, and zoom capabilities. Therefore, it has been very well received that IYOUIT also allows for post-processing Flickr images: pictures uploaded from an external camera can be automatically augmented with related context tags at a later point in time.

Figure 10: Categories of manual tags and descriptions. Context Abstraction and Presentation – Places

IYOUIT collects much data while in use. This raw data is often only of limited value to the user and becomes meaningful only after it has been combined and abstracted. IYOUIT uses context ontologies and Semantic Web technologies to implement key abstractions and reasoning features. We refer to Böhm et al. [4] for the details. We exemplify context abstraction through the concept of places: an IYOUIT place is a semantic concept that abstracts from a geographic position and describes it by type and name. Places may expand to cover whole areas. The concept of places is needed for three reasons: first, location measurement on the phone has to rely on imprecise data (Fagerberg et al. [8], note that “place labels are useful no matter the granularity of the positioning system”); second, several real world locations consist of the same place – e.g. within an office building; third, abstractions are necessary to give a more appropriate meaning to locations, e.g. “office” instead of a street (see, e.g. Ashbrook and Starner [1]). As has been stated before, location is an import type of context and people often rely on and share such knowledge. Grandhi et al. [10], e.g, found that up to 77% of people observed in New York over a 3 weeks period were willing to share their location data (17% even with total strangers).

Figure 8: Number of photos shared using IYOUIT (avg. 30).

Out of all tags that have been manually assigned by the users, five categories of tags have been identified. A large number of tags detail the location where the photo has been taken (Figure 10). The mobile application actually could have derived 53% of those (such as city, proximity to a recreational area, or the position close to some landmark). Two other main categories describe some detail depicted on the photo (such as a person or event) or some activity. Descriptions are longer but used for the same purposes, but favor detail over location (e.g. if covered by a tag). These findings strengthen the need for a system providing smart algorithms for automatic tagging that, at the same time, allows for manually adjusting or adding information. 3

See http://bit.ly/IYOUITpic for an example picture

212

Full Papers

Proceedings: NordiCHI 2010, October 16–20, 2010

The IYOUIT system holds an ontology of places shared by everyone to coherently share places between users (e.g. „home‟ vs. „office‟). Participants used 39 different place types for 235 places indicating that this mechanism is both appreciated and used. Similar to setting experiences manually, we suspected that the manual specification of places would be too much work for most users. Therefore, complementary to manual places, IYOUIT runs specialized data mining algorithms (cf. Nurmi and Koolwaaij [16]) over aggregated location traces and proactively generates place suggestions. Similarly to the reasoning by Ashbrook and Starner [1] as well as Hightower et al. [12], this is mostly based on the amount of time the user stays within a certain region. We also abstract from concrete positioning methods taking inherent (in)accuracy into account. The user is provided with such place suggestions and can easily decide to name and typify, to ignore, or delete them. This was very well received (about two-thirds of all suggestions adopted) and acknowledged in the questionnaires and interviews.

Although places are obviously very personal, we introduced a feature that enables people to inspect and subscribe to places of others (only for places explicitly marked as visible by buddies or the public). Within the trial group, already about 6% of the places have been used in such a way. This is one of the features we think has a high value, especially as soon as the IYOUIT community reaches a certain size and more places are being created.

Figure 12: Time users spent within a specific place (avg. 69%).

As a side result, it is interesting to see that many people seem to move between places quite often. The data gives an average of 8.9 times (σ = 7.6) that a user enters another place per day. Still, this seems to be quite specific to individual users (also indicated by the high standard deviation). Judging from the available data, one could categorize users into the three clusters „rather stationary‟, „average mobile‟, and „very mobile‟ with the distribution within the trial group being 8-5-3. This also suggests that specifying rules based on places, e.g. automatically setting one‟s status or activity, can provide a constant stream of sensible data.

Figure 11: Manually named and subsequently visited places. 4

Figure 11 shows the distribution of named places of each user with an average of about 15 places per user. The same chart also displays the number of places that have actually been visited during the trial period (this includes unnamed places, see below). Our usage data over a longer period of time shows that, for the majority of people, an average of 12 places seem to cover most regularly visited areas of interest. This is in accordance with previous research, e.g. by Hightower et al. [12] who found that an average of 7 to 8 places are visited regularly within a year and Zhou et al. [24] who discovered on average 15 significant places.

Privacy and Social Networking

From the user‟s point of view, IYOUIT is primarily a sharing platform for personal information. Here, the social aspects of sharing data and communicating with people, indirectly with context updates and directly by sending messages, blogging, or commenting on other‟s context are vital to the application. Privacy Policies and Directives

A Privacy Manager is an integral part of the IYOUIT system (for details see Böhm et al. [3]). In contrast to most platforms such as Facebook that allow full access to personal information for one category of „friends‟, IYOUIT extends to more defined categories of social contacts and makes sharing fully bound to the personal social network through a fine-grained access control. Previous studies, e.g. Palen and Dourish [19], suggest giving users full control over privacy directives. In IYOUIT, this has been fully implemented by controlling access to personal information with a general system of rules defined by the user and resource specific access directives as an exception to the general rules. These fine-grained access control mechanisms allow for the definition of rules for groups of people or specific buddies, for distinct context categories as well as the level of detail that is being disclosed (a requirement also

Furthermore, people overall spent over two-thirds (69%) of the time they used IYOUIT in the defined area of a place (Figure 12). In fact, some people have spent about 75% of their time within one specific place (provided one can meaningfully extrapolate beyond the uptime of the client). This statistic is actually very similar when considering the whole IYOUIT community during the trial period with an average value of 67% to be within a defined place, thus further confirming the validity of the place concept. 4

Note that in this and the following chart, only the data of 16 participants are visualized as the short usage times of the 3 users who had IYOUIT running for less than 2% of the time could lead to inappropriate values.

213

Full Papers

Proceedings: NordiCHI 2010, October 16–20, 2010

stressed by Consolvo et al. [6]). This means that, for example, access to the user‟s current location can be prohibited completely or allowed at a country, city or concrete address level. Users may also separately choose to grant access to historic information. Resource specific directives, on the other hand, provide means to explicitly choose the level of disclosure for specific items (such as a picture) case-bycase, independent of the general directive for this context category.

seem to fit the way people interpret these and want to have them: the user interface of the privacy manager was actually only accessed by 9 of the 19 users and only 6 of them modified the default directives. On the other hand, privacy directives could be adapted individually from the mobile client as well. The users‟ explicit opinions varied: 3 out of 17 people said to be fully comfortable in sharing their context with their buddies. However, in the trial, only one user actually granted full access for his buddies. 4 persons felt uncomfortable with sharing but again only one completely restricted access to others. This is similar to the findings reported by Consolvo et al. [6] who state that the Privacy Segmentation Model [18] into unconcerned, pragmatists, and fundamentalists through interviews does not seem to be a good predictor for actual behavior.

As Sadeh et al. [22] state, giving the user such a flexible and powerful control over privacy and sharing settings can increase the adoption of context-aware services. They also show ways of incorporating more complex rules for granting or denying access to information and provide guidelines in this direction. However, as they and Palen and Dourish [19] correctly point out, this can always be only partly a solution since (location) privacy is a "dynamic process of managing boundaries". Interestingly, the willingness to choose the level of disclosure for specific items rather than just relying on the previously defined rules or default settings varies greatly for different context categories. Our study participants often wanted to specify the sharing level of photos individually, whereas the sensitivity of location has been seen as rather static. For example, the majority of 62% of pictures were uploaded to be publicly visible, while 30% were for buddies only, and the remainder was only visible for the authors themselves. This indicates that there is a definite need for privacy preserving mechanisms and that they should be adapted according to the type of data.

Figure 13: Relationships used to categorize buddies. Social Networking Platform

In contrast to some systems for which a direct connection between usefulness and number of users is well known as the critical mass (see Rogers [21]), IYOUIT is already valuable without any or with only a small number of buddies. Yet, after the end of our study, everyone had a relatively large network of 2 to 25 buddies with an average of 10 and a total of 184 explicitly defined relationships. shows how study participants categorized their buddy relationships with 63% being tagged as business contacts, 7% as family, and 30% as friends. The high number of business relationships could indicate a potential use in the corporate setting, an application area that has in fact been suggested by several interviewees (Bardram and Hansen [2] show an application for hospitals using context to trigger social awareness between colleagues).

One finding derived from personal feedback and interviews is that it seems to become easier to define meaningful privacy directives when changing from low-level data to a higher level of abstraction. For example, most users easily understood how to judge the sensitivity level of places, whereas only more „expert‟ users were able to express the corresponding rules for spatial data in general. As a positive result, we regard that 79% were comfortable with sharing context with their buddies through IYOUIT. Furthermore, 63% of all trial participants expressed no need for a more sophisticated privacy management procedure. Others mainly argued for adding control about which picture is transferred to which 3rd party platform with which privacy setting or have not found the opportunities that the current system offers.

In order to make the most out of the IYOUIT platform and to increase added value, IYOUIT can be configured to connect to existing 3rd party services such as photo sharing platforms (Flickr), blogging platforms (Twitter), and social community platforms (Facebook). The automatic distribution of information across services was valued highly by many users. However, the actual representation or configuration of information within, e.g. Facebook needs to be revised as the following comment made clear:

We can support the discovery by various works [6, 20, 22] that people‟s sharing behavior depends on types or groups of people such as co-workers or close friends. To this end, IYOUIT allows to define personal social networks of buddies by defined and typed relationships such as „friend‟, „brother‟, „colleague‟, etc. which are more comprehensive than in most other systems. Relations are pre-configured in privacy settings that can be adjusted individually but mostly

“My friends really wondered why I suddenly started to comment on me waking up!” Italian, male, 26 years.

214

Full Papers

Proceedings: NordiCHI 2010, October 16–20, 2010

RESULTS AND OBSERVATIONS

The vast majority of uploaded pictures (90%) were also automatically transferred to Flickr. As Flickr employs a categorized privacy mechanism similar to IYOUIT, the access labels „public‟, „private‟, and „buddy only‟ are directly mapped (the buddy groups differ, though).

In the following, we distill and abstract from the study findings reported in the last sections to provide a set of outcomes that have been proven successful in the design of IYOUIT and can therefore also be relevant for those active in the area of context-awareness and social networks.

For Facebook, e.g., not having this mechanism, only pictures uploaded as public are transferred. In the questionnaire, 55% were very positive towards the automatic addition of status updates in their Facebook accounts and several of them requested an even tighter connection with the IYOUIT application.

General Usage

Combining data and meta-data from various sources and with various services is an important feature, live and also after data collection (e.g. post-tagging pictures on Flickr). Most important types of context as expressed by the users are location, pictures, weather and presence information.

Trends throughout the Study

We also tried to spot trends and changes in attitudes during the course of the study. However, besides a slight general decrease in use, there were no significant changes to be observed. We cannot clearly say whether this is due to the duration of the study or the mere absence of changes.

Power consumption is an issue although, in our study, this had only minor impact. Still, users see which features impact battery life and should be able to turn those off. Proactive versus Manual Actions

The concept of places has been widely understood and used. Users highly valued automatic place suggestions. In general, automatic abstraction mechanisms from raw (sensor) data proved to be an effective tool for leveraging personalized information exchange.

As one example of analysis of the quantitative data we collected, we let people judge whether IYOUIT increased their communication between buddies before and after the study. The plus of 0.4 points on the Likert-scale (to 2.1, σ = 1.3) was not significant (F = 0.71, p > .2).

Automatic context tagging can relieve users from manual input. However, what the system can derive automatically must be shown, with the possibility to manually adjust it.

Some behavior also seems to be an individual preference, independent from the amount of use. For instance, there was no correlation between the number of photo uploads and photo tagging per user (r2 = 0.07).

Automatic activity recognition needs more research before the process can (if ever) be fully automated. However, recommendations from which a user can choose can be a meaningful shortcut to express assumed presence states.

Some interesting results appeared with regard to thoughts about privacy. Our questionnaires before and after the study revealed that, on average, the use of IYOUIT made people more confident to share data with close family members and friends (see Figure 14). However, people also became aware that the information being disclosed can be rather delicate and detailed and the readiness to share dropped slightly for more distant relationships.

Privacy and Sharing

Explicit sharing is more subject to privacy concerns than automatic sharing: people tend to be reluctant to share manually entered information (such as mood) in detail. Automatically gathered, yet personal data (e.g. spatial information), however, has been regarded as less sensitive.

On the other side, two people acknowledged that they learned to restrict the amount of information made available and thus also became more comfortable to provide some data even to lose contacts.

3rd party services offer high potential, especially a connection to social platforms and automated information updates provide a strong added value for users. However, the user must remain in control of what is transferred and generated content should be explicitly marked or separated. A dedicated privacy mechanism that is accessible, understandable, and modifiable by the users is essential. Although people like to be in control and use an item-by-item strategy for some data such as images, privacy settings per type of buddy relation are accepted. Thus, we suggest changing the opinion of Consolvo et al. [6] and Lederer et al. [14] that "who is inquiring about one's location" is a strong factor to "the relation to the one who is inquiring". This implies a greater number of relationship types than available in most current systems is beneficial.

Figure 14: Willingness to share context with specific groups. People generally shared more information with family members or friends and became more careful in sharing with more distant persons.

215

Full Papers

Proceedings: NordiCHI 2010, October 16–20, 2010

SUMMARY

11. Györbíró, N., Fábián, Á. and Hományi, G. An activity recognition system for mobile phones. Mobile Networks and Applications 14(1). 82-91. 2009

We have evaluated IYOUIT, a freely available platform and mobile application for social, context-aware computing. A trial has been performed with 19 users over four weeks. Results have been presented in the categories of general acceptance and use, context sharing, context abstraction, as well as privacy and social networking. We collected and presented results in four categories about use, context sharing, context abstraction, and privacy.

12. Hightower, J., Consolvo, S., LaMarca, A., Smith, I. and Hughes, J. Learning and recognizing the places we go. In Proc. UbiComp’05. 159-176. 2005 13. Hong, J-Y., Suh, E-H. and Kim, S-J. Context-aware systems: A literature review and classification. Expert Systems with Applications 36(4). 8509-8522. 2009

We feel that this study has greatly enhanced our own understanding of context-aware social systems. In the future, we plan to further investigate and study the potential and acceptance of enhanced profiling algorithms to derive patterns about users, which promise more targeted personal services. As a starting point from a commercial point of view, 76% of the participants would accept targeted advertisements within the application. Complex derivations of suggestions and profile data will then, however, need further research to enhance their understanding and acceptance on the user side (cf. Dey and Newberger [7]).

14. Lederer, S., Mankoff, J. and Dey, A. K. Who wants to know what when? Privacy preference determinants in ubiquitous computing. In CHI’03 Extended Abstracts. 724-725. 2003 15. Morla, R. and Davies, N. Evaluating a location-based application: A hybrid test and simulation environment. IEEE Pervasive Computing 3(3). 48-56. 2004 16. Nurmi, P. and Koolwaaij, J. Identifying meaningful locations. In Proc. IEEE MobiQuitous'06. 2006 17. O'Grady, M., O'Hare, G. M. P. and Sas, C. Mobile agents for mobile tourists: A user evaluation of Gulliver's Genie. Interacting with Computers 17(4). 342-366. 2005

REFERENCES

1. Ashbrook, D. and Starner, T. Learning significant locations and predicting user movement with GPS. In Proc. ISWC’02. 101-107. 2002

18. P&AB. Consumer privacy attitudes: A major shift since 2000 and why. Privacy & American Business Newsletter 10(6). 2003

2. Bardram, J. E. and Hansen, T. R. The AWARE architecture: supporting context-mediated social awareness in mobile cooperation. In Proc. CSCW’04. 192-201. 2004

19. Palen, L. and Dourish, P. Unpacking “privacy” for a networked world. In Proc. CHI’03. 129-136. 2003

3. Böhm, S., Koolwaaij, J., and Luther, M. Share whatever you like. In Proc. Workshop CAMPUS’08. 2008

20. Patil, S. and Lai, J. Who gets to know what when: Configuring privacy permissions in an awareness application. In Proc. CHI’05. 101-110. 2005

4. Böhm, S., Koolwaaij, J., Luther, M., Souville, B., Wagner, M. and Wibbels, M. Introducing IYOUIT. In Proc. ISWC’08, 804-817. 2008

21. Rogers, E. M. Communication technology: The new media in society. 1 st edition. The Free Press. 1986

5. Chen, G. and Kotz, D. A survey of context-aware mobile computing research. Technical Report UMI TR2000-381. Dartmouth College. 2000

22. Sadeh, N., Hong, J., Cranor, L., Fette, I., Kelley, P., Prabaker, M. and Rao, J. Understanding and capturing people's privacy policies in a mobile social networking application. Personal and Ubiquitous Computing 13(6). 401-412. 2009

6. Consolvo, S., Smith, I. E., Matthews, T., LaMarca, A., Tabert, J. and Powledge, P. Location disclosure to social relations: why, when, & what people want to share. In Proc. CHI’05. 81-90. 2005

23. Tungare, M., Burbey, I. and Pérez-Quiñones, M. A. Evaluation of a location-linked notes system. In Proc. ACM-SE 44. 494-499. 2006

7. Dey, A. K. and Newberger, A. Support for contextaware intelligibility and control. In Proc. CHI’09. 859868. 2009 8. Fagerberg, P., Espinoza, F. and Persson, P. What is a place? Allowing users to name and define places. In CHI’03 Extended Abstracts. 828-829. 2003

24. Zhou, C., Frankowski, D., Ludford, P., Shekhar, S. and Terveen, L. Discovering personally meaningful places: An interactive clustering approach. ACM Trans. Inf. Syst. 25 (3). 12-42. 2007

9. Froehlich, J., Chen, M. Y., Consolvo, S., Harrison, B., and Landay, J. A. MyExperience: A system for in situ tracing and capturing of user feedback on mobile phones. In Proc. MobiSys’07. 57-70. 2007

25. Raento, M., Oulasvirta, A., Petit, R., & Toivonen, H. ContextPhone: A prototyping platform for contextaware mobile applications. IEEE Pervasive Computing, 4(2). 2005

10. Grandhi, S. A., Jones, Q. and Karam, S. Sharing the big apple: A survey study of people, place and locatability. In CHI’05 Extended Abstracts. 1407-1410. 2005

26. Jacucci, G., Oulasvirta, A., Ilmonen, T., Evans, J., & Salovaara, A. Comedia: Mobile group media for active spectatorship. In CHI‟07. 1273-1282. 2007

216

Full Papers

Proceedings: NordiCHI 2010, October 16–20, 2010

Negotiating Privacy Boundaries in Social Applications for Accessibility Mapping Harald Holone Østfold University College, Norway [email protected]

Jo Herstad University of Oslo, Norway [email protected]

ABSTRACT

Typically, the approach to sharing of such information is largely based on personal communication, and the trust implied through personal connections. Making this process more public allows for greater reach of the information, and the collective gathering of accessibility information that can form the basis for route planning and other applications. On the other hand, exposing ones whereabouts, preferences, needs and opinions in a more public sphere raises issues not only related to trust, but also about aspects of privacy.

Privacy is often used as an abstract concept, and negotiating what information to disclose to whom, where, at what times, and in what situations is a challenging one. In this paper we apply a previously proposed framework from Palen and Dourish for understanding and discussing privacy to a setting of sharing and acting on information about physical accessibility. We do this by describing existing practice for sharing of such information among wheelchair users, and compare that with new practices emerging from the use of ICT, the mobile, collaborative route planning concept OurWay. Through highlighting these changes, we discuss concrete privacy issues, and hope to provide a contribution to users, designers and analysts for creating and using mobile, networked technologies for accessible navigation of urban and built environments.

We draw on research from Computer Supported Collaborative Work (CSCW) and Human Computer Interaction (HCI) to discuss the role of privacy and negotiation in collaborative systems. Further, we take a look at relevant accessibility literature to explore the need for user involvement in assistive technologies. To form a basis for our discussion of privacy, we build on the work by Palen and Dourish on privacy in a networked world [11], which in turn is adapted from Altman’s privacy theory from the 1970’s. The framework suggested by Palen and Dourish is useful both for illustrating the complex notion of privacy, as well as a vehicle for discussing the role of negotiation and privacy in a social application for accessibility mapping. We are using a concrete example ﬁrmly founded in an existing practice to discuss the boundaries and negotiation of information disclosure. In this context, the notion of privacy is not about law, it is about the experienced privacy.

ACM Classification Keywords

H.5.2 User Interfaces: User-centered design General Terms

Human Factors Author Keywords

CSCW, Social Navigation, Accessibility, Trust, Privacy, Negotiation INTRODUCTION

The contribution of the paper is twofold: First, we problematize the general use of social technologies for speciﬁc applications like accessibility mapping, and second we build on the work by Palen and Dourish to discuss the notion of privacy in social applications. The main research question we address in this paper is What privacy issues emerge when social software technologies are applied for speciﬁc situations like accessibility mapping?

The disability research area covers many aspects, including physical accessibility and e-accessibility. In previous work, we have suggested the use of ideas from social applications on the web to create and share information about physical access to built and urban environments. In this paper, we introduce the issue of privacy that arises when these technologies are applied in an attempt to solve the gargantuan task of collecting, disseminating and maintaining such information. We refer to this process Accessibility Mapping.

The paper is organized as follows: First we review relevant literature within the ﬁelds of accessibility, social applications, HCI and privacy. Then we re-cap the work with OurWay, a collaborative route planner, including our latest work relating to the re-framing of OurWay as a social application, and it’s potential for accessibility mapping. The discussion focuses on the notion of privacy in general, as well as speciﬁcally within the realm of accessibility mapping. Finally we conclude the paper with a summary of the role of existing practice and the

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. NordiCHI 2010, October 16 - 20, 2010, Reykjavik, Iceland. Copyright 2010 ACM ISBN: 978-1-60558-934-3...$5.00.

217

Full Papers

Proceedings: NordiCHI 2010, October 16–20, 2010

negotiation of privacy in mobile, networked systems for accessibility.

The role of negotiation in CSCW systems is put forward by Schmidt et al. as important for coordination and articulation work [14, 15]. A complementary form of negotiation of contexts created by introduction of new technologies is introduced by Satchell and Dourish on their study of non-use in HCI [13]. This perspective is useful for understanding mechanisms by which new technologies are introduced, but not necessarily accepted.

RELATED WORK

We now brieﬂy review relevant literature in the ﬁelds of accessibility, social software and privacy. This literature provides a background for next section’s discussion of privacy in mobile, networked ICT for collaborative, accessible route planning.

Privacy

Accessibility

ICT-related privacy research often focuses on technology and implementation, and related to rules and regulations (see for example [2]). Palen and Dourish have proposed a framework for a more nuanced understanding and discussion of privacy in a networked world [11]. Building on the seminal work of Altman, they identify three boundaries central to the negotiation of information disclosure. The ﬁrst and basic boundary is the disclosure boundary, that is, what information to reveal or share, and what to keep from others. Secondly, the identity boundary is deﬁned by the role taken on by the user. For instance, if the user is representing herself personally, or if she represents a group such as a company or organization. Finally, within the temporal boundary the eﬀects of persisted information is discussed. That is, the information left behind in a networked system can be interpreted by unintended recipients at a later time, and there is little or no way of controlling the interpretation of information, or the context in which it is interpreted.

Accessibility is a term with many connotations, and covers a wide range of topics, such as the accessibility and usability of ICT, as well as to accessibility in the urban and built environment. In the work with OurWay, the primary focus is on physical accessibility, and the communication of information relevant to moving around in the physical world. Universal design and accessibility in general is currently receiving a lot of attention, as can be seen for instance in programs within the European Union with regards to accessible technology and urban planning. Our current focus is on use of mobile ICT for sharing information about accessibility in the physical world. Also in the physical world, accessibility can have diﬀerent meanings. For instance, in results from the AUNTSUE project [5], social aspects like fear of crime are mentioned as important factors when users assess the overall accessibility of an area. V¨ olkel et al. [17] describes requirements for accessibility annotation of geographic data, with the aim to collect detailed information for diﬀerent needs and preferences.

Altman’s insight is that privacy is not a static set of rules, rather it is a dynamic process, a constant negotiation depending on the situation. Grudin puts this in the context of situated action, which is precisely what allows the constant negotiation just mentioned [6]:

Existing literature on accessibility focuses mostly on the interaction between a single user and a computer system, with some notable exceptions, e.g. [18] and [8]. Hedvall [8] argues that the accessibility ﬁeld is lagging behind the HCI ﬁeld, focusing mostly on regulations and predictability. His work bridges work in the HCI ﬁeld and the accessibility ﬁeld, and by comparing the evolution of the two ﬁelds, he suggests that the accessibility ﬁeld has much to learn from the work in HCI and CSCW. In particular, the focus on the individual is key to Hedvall’s argument, in what he calls experienced accessibility.

Why then the uneasiness, the widespread attention to privacy? It may reﬂect an awareness at some level of something more fundamental than privacy that is being challenged: The steady erosion of clearly situated action. We are losing control and knowledge of the consequences of our actions, because if what we do is represented digitally, it can appear anywhere and at any time in the future. We no longer control access to anything we disclose. The role of negotiation, according to Schmidt, is a key component for a successful CSCW system. It seems obvious from the privacy work cited above that this is particularly true when it comes to issues of privacy. In fact, we would argue that the experienced accessibility discussed by Hedvall has a parallel in privacy, in what may appropriately be dubbed experienced privacy. This experience is diﬃcult to assess in situ, since the control over interpretation is no longer in the hands of the user.

Social Software

Grudin, although not using the term social software, puts forward four characteristics he considers key in these kinds of technologies [7]: . . . they 1) can be extremely lightweight, 2) make information and activity highly visible, 3) provide individual and group beneﬁts, and 4) are grassroots, self-organizing phenomena. Grudin refers to studies of using project blogs and wikis as well as the use of hash tags for coordinating and maintaining project-related content and activities. We have used Grudin’s characteristics for discussing OurWay as a social application.

218

Full Papers

Proceedings: NordiCHI 2010, October 16–20, 2010

RESEARCH SETTING AND METHOD

Dashboard [16] projects, where exposing the source and history of information is a key point. Perhaps the main drawback of the current OurWay prototype is that it only indirectly provides this information, through the resulting route suggested to the user. In other words, there are no clues provided as to who provided the annotations that led to the suggested route, or what context they were captured in.

We will now provide the background for our case, OurWay. First, the concept and early work is presented, before we summarize a discussion of OurWay as a social application. Finally, we present ﬁndings from interviews with wheelchair users to illustrate existing practices for sharing of accessibility information. The OurWay concept

Individual and group benefits

OurWay is a collaborative route planner, where users are providing feedback on accessibility through their interaction with the system. Users are equipped with mobile phones running a client application, which connects to a central route planning and feedback server over a mobile Internet connection. The client displays a map of the area, and allows the user to ask for a route between two locations, which is then displayed on the map. The user can at any time provide feedback to the central server about the part of the route being traversed. To keep the threshold for contribution low, we have used only three levels of feedback: good, uncomfortable, and inaccessible. The submitted ratings are attached to route segments, and are used as weights by the route planning server when calculating new routes.

Some individual beneﬁts of a social software system for accessibility mapping are obvious, such as the opportunity to provide feedback on suggested routes and get immediate reward in the form of an alternative route from the system. The beneﬁts for the group(s) are, as we have shown through the work with OurWay, more accessible routes over time. Another beneﬁt for the group as a whole, is that this way of collecting and maintaining accessibility information is a potential resolution to the resource challenge mentioned in the introduction of this paper. Further, the social awareness of mobility challenges and special needs can be raised, also outside of the group of core users. Self-organizing grassroots phenomena

Social applications for accessibility mapping

There is obvious value in the networks, agendas and political power represented by established interest organizations. At the same time, parts of what make social applications work is the ability for users to rapidly form groups and processes, often to inﬂuence the organizations or establishments they are taking part in. There should be a mutual interest from institutions, individual users and user groups to make use of social software tools, however the suitable balance of initiative and power remains to be established. See for instance Borchorst et al. [3], where they look at the application of Web 2.0 technology in the interaction between citizens and municipalities in Denmark.

The OurWay project started as a technological prototype, without extensive work on designing it for collaboration. Throughout the project the focus has shifted towards the social aspects of use, and by using Grudin’s four characteristics we have re-framed the concept as a social application. A summary of this discussion is presented here, for more details see [9]. Lightweight

From the end user’s perspective, OurWay can be considered extremely lightweight. The feedback mechanism is a vast simpliﬁcation of the typically form based inquiry taking place in traditional accessibility mapping. We do not propose that the OurWay concept should replace such detailed approaches, rather that the information generated by users can augment existing information. One obvious challenge is the use of mobile phones for people with special needs. Although mobile phones are ubiquitous among all user groups, the necessary HCI challenges related to use in the ﬁeld must be considered carefully. One key argument for using an open, lightweight infrastructure for a social accessibility mapping application is that the opportunities for adaption to diﬀerent needs and requirements are moved away from the core of the system and towards the end user.

Existing practice

We performed in-depth semi-structured interviews with three wheelchair users early in the OurWay project. The goal was to assess the interest in such technology, and the privacy issues related to use. Two of them have been active participants in organizations for the disabled over many years, and knew about the OurWay idea. The interviews were fully transcribed, and analyzed using an open coding process. This material is now used to describe existing practices for navigating the urban landscape with wheelchairs. Then we discuss the changes in practice introduced with new technology to provide a background for the discussion of negotiating privacy.

Information and activity made visible

For anyone, planning is an important part of a trip to an unknown location. The most common practice for planning a trip in a new or unknown urban landscape, according to our subjects, is direct communication with other trusted people who understand the needs of the person who asks:

A tool which provides route suggestions must adhere to high standards when it comes to dependable information and social transparency. There is little doubt that allowing the user to see the ratings and context which is used to calculate the routes is important. Some suggestions can be found in the WikiTrust [1] and Wiki-

219

Full Papers

Proceedings: NordiCHI 2010, October 16–20, 2010

“When you are in a situation like ours, as wheelchair and crouch users, we become very creative in ﬁnding solutions, and we always have to think ahead. We can’t just go.”

mous] says that the criteria for universal design has been followed, I know that it works for me. But I also know that if Mrs. Olsen comes along, and she has problems with her hip, I’m able to say that you shouldn’t trust that these toilets will suit you. Because they suit me.”

This communication often implies an intimate knowledge of each other, including previous common experiences and knowledge of each others respective needs: “If I know that he has been there, right, and I know that he and I share the same needs, if he says this worked for him, then I can go there, too.”

Some urban features, like shopping malls, are considered accessible “by default” by many users. Both because of easily accessible and designated parking spaces, and because of thoughtful design of the built environment:

“I would use a mobile phone, right. And talk to them and then ask [. . . ] are you sure it will work with my chair? Of course, if [friend] was the one sharing the information, I would have done it anyway, right. Because I know him so well.”

“When it comes to shopping malls, they have trolleys, shopping carts, baby strollers, and they build for them. Everyone must have access, because they need to shop. And if baby strollers and trolleys can get in, a wheelchair user can also get in.”

This is emphasized by the fact that many wheelchair users have diﬀerent assistive technologies for diﬀerent occasions, be it motorized chairs (light and heavy), manual chairs and in some cases crouches. Says one user of an electric chair:

“. . . and the ﬁre regulations. These regulations demand quite a bit with regards to escape routes, the size of those, that there shouldn’t be stacks of goods placed everywhere, and so forth. This makes it secure enough for you to use it.”

“I sometimes use a manual chair, but then I need someone along to push me. [. . . ] I can make two trips to the center and back between charging the battery. Then I’m not using this chair, I’m using the large chair.”

Thus, we get the impression that shopping malls are preferred over shops in the streets because of the general accessibility: “You know that if you’re going shopping you go to the shopping malls. You don’t go looking for the smaller local shops, right?”

One frequently recurring theme from the interviews is that of individual needs and the issue of trusting information coming from others. This anecdote aptly illustrates what can happen if the shared understanding of needs is absent, or diﬀerent reference points exist:

Several initiatives to map out the accessibility of urban areas has been undertaken, mostly (so far) based on paper maps and by the help of volunteers. Further, other less formal activities have been carried out, like this attempt to signal the accessibility of individual shops:

“I called a hotel to book rooms for the National Association of Disabled, and they told me that their hotel was accessible to disabled, and they knew that because a Paralympics team had staid there. [. . . ] There were no obstacles, right? These people even used the stairs to get to the dining hall. [. . . ] The receptionists experience was that she had seen many people in wheelchairs, and it all worked well in her hotel.”

“We had this campaign once where we sold accessibility stickers. We made the criteria, but we were obviously not suﬃciently speciﬁc about the criteria, and then we sold these stickers [to the shops]. So when you came to a shop and saw the sticker, you knew that everything was OK. That was the intention. However, after a while, when diﬀerent people did this [sold the stickers], we blurred the criteria, and we knew each other, and it became more important to sell stickers than to [indicate accessibility].”

Even interpreting standardized accessibility, for instance by signposting toilets as accessible, requires the similar knowledge. In this situation, intimate knowledge of the formal requirements are needed:

In summary, existing practices for sharing accessibility information is often based on individual needs and personal relations. The information is local, and is interpreted based on the individual and the situation. As the examples presented indicate, even formalized regulations and knowledgeable people can create uncertainty regarding the experienced accessibility. Having presented existing practice as a background, we now move on to the main discussion.

“The toilets, for instance, are a problem for many. Some want them high, others want them low. I want them low, as I’m 145 tall. There are no regulations on height for the toilet seats. A whole group in the Association of Disabled of course want to change the regulations so that all accessible toilet seats should be high, right. [. . . ] I know the regulations so well. It’s impossible to make a toilet seat both low and high, right. [. . . ] So if [anony-

220

Full Papers

Proceedings: NordiCHI 2010, October 16–20, 2010

DISCUSSION

conditions.

Introducing new assistive technologies will change the practice of getting around. Introducing aids for sharing of accessibility information will change the way people share this information. It will most likely not replace the existing practice, and it is key to our discussion that the technology is augmenting existing practice, not substituting it for something completely diﬀerent. In the ﬁrst part of the discussion we identify privacy issues that can result from use of a collaborative navigator, before we focus on negotiation of privacy in this context using Palen and Dourish’ framework in the second part.

Further, simply introducing a device like a mobile phone into the process of negotiating accessibility on the spot has practical implications, such as simply managing the extra device in an already demanding situation. Further, use of such a solution may be regarded as a special case of using a public terminal [4], which in itself has impact on the use situation. This is diﬀerent from using navigation equipment in a car, where the car provides a private sphere in which interaction with the technology can be negotiated.

Privacy issues

Reach and persistent information

Staying focused on the individual, we now attempt to highlight some of the changes that will arise with the introduction of new assistive technology. We do this to answer our main research question.

Existing practices for sharing of accessibility information are largely local, personal and ephemeral. A mobile collaborative system can extend the reach of such information in more than one way. First, the potential number of people who can access and provide information to the system is increased. This means that information left behind can be used by people who would not normally have easy access to the information (they are not part of a local community, or lack personal relations in the area in question). It also means that users expose their activities to strangers, which raises privacy issues. Further, the exposure of activity can provide new and perhaps awkward situations, illustrated here by a quote from one of the interviews. The topic was whether annotating a goods lift as accessible could be justiﬁed if the beneﬁts of getting to another ﬂoor were signiﬁcant:

Change in practice

As we have shown, identifying accessible routes and locations for the individual is largely accomplished by intimate knowledge of respective accessibility needs and relations between people, and understanding of the criteria behind standardized accessibility. With the introduction of ICT for assistance, the communication forms changes, and the old practices cannot be directly applied. For instance, relying on persisted accessibility information provided by strangers will require support for sense-making, both through the technology itself, and by traditional means. However, to allow for this negotiation of meaning, the activity of other users in the system must be made visible, including information about the user(s) who provided the information.

“This is very dangerous. Because, as you say, this system could also be used by the municipality. If it [the use] in any way reﬂects that the group that I represent [when annotating the lift] accepts goods lifts as equally good alternatives to ordinary lifts, they might conclude that they won’t bother about [installing] an ordinary [accessible] lift.”

Providing details about other users in the system is a balancing act between full disclosure and privacy. Will full identities, activities and assistive technologies in use be required for this negotiation? Perhaps building trust networks and social relations into the application can be one way forward. In this way, some form of anonymity can be maintained, whilst still providing cues about the trustworthiness or relevance of information.

This statement highlights several privacy aspects, such as the representation of self and other, and the persistence of information over time and the interpretation of the persisted information. We will return to these issues in the next section.

In the existing practice, this information has an ephemeral and personal nature. Individual needs are communicated on a trusted need-to-know basis, and often it is not even an explicit part of the negotiation, as the knowledge is tacit and does not need to be addressed speciﬁcally. Another potential change in practice comes with the new opportunity for more spontaneous trips, since the information is readily available on the mobile device. There might be a shift from detailed pre-planning of routes to more ad-hoc route planning. The mixed reality of an annotated map with route planning capabilities provides the ability to adjust the route along the way, taking into consideration local and temporal changes in accessibility, such as construction work or weather

One of the premises for a collaborative navigator like OurWay is that users leave behind traces of their activity, in other words, that parts of their interaction with the system is recorded to facilitate calculation of accessible routes with updated information. Persistence of information is perhaps the most important source of privacy issues to be discussed in the next section. Persisting traces of user activity extends the reach, not only in terms of audience, but also over time and use contexts. Based on these observations about changes in practice and the introduction of technology to extend reach and persisting accessibility information, we switch gears and discuss these privacy issues with the help of the frame-

221

Full Papers

Proceedings: NordiCHI 2010, October 16–20, 2010

work proposed by Palen and Dourish.

the need for negotiation of what information goes out to whom, and what activity is triggered by interaction with the system. Some activity might be extra valuable to make public, for instance to make visible ones own contributions to the system and community of users in order to gain credibility or good reputation. Making activity visible can also be a means for limiting vandalism or improper use. However, as we shall argue later, there are instances where visibility of activity needs to be kept private.

Negotiating privacy

Our focus in this part of the discussion is on the need for negotiation of privacy in diﬀerent situations, emerging from the introduction of ICT for sharing of accessibility information. Disclosure: Privacy and Publicity

The way we interpret the framework, the fundamental boundary is that of privacy and publicity, in other words what information users in a situation regard as private, and what information it is desirable or necessary to share with others. First, we need to identify the core types of information at stake. Obviously, positioning information is fundamental in a navigation system. This might be geographical coordinates, or approximate information such as venues used in the social application FourSquare. Time can also be crucial, for instance for taking into account the temporality of certain obstacles (like weather conditions), or time of day (related for instance to opening hours etc). The level of accessibility experienced by the user must also be recorded. In previous research, we have observed that it is largely inaccessibility which is recorded, and mostly when users have to provide feedback in order to get an alternative route from the route planner [10]. This might change, though, if such a collaborative navigator is used in a campaign setting.

In summary, the negotiation of what information to disclose must be left to the user, and whether or not activity should result in public traces must be negotiable by the user at any time. Identity: Self and other

The ﬁrst question a potential user must answer is whether to join the service or not. Trivial as this might seem, it is a fundamental question, and one that is well illustrated by the social pressure to join and participate in successful on-line communities like Facebook or Twitter. Once a participant, it must be possible to quit the system, and potentially delete traces of ones own activity. Quitting or changing ones association with the service will be a public activity, which again touches on the social obligation to be a participant in the community. As an example of the publicity of private actions we present an anecdote about Facebook from one of our colleges: Gisle joined Facebook when it ﬁrst opened for the international audience. He added many friends, including his wife who he identiﬁed as such. Realizing that this resulted in public information on his proﬁle revealing his private family relations, Gisle decided to change the relationship from “spouse” to ordinary friend. The unintended eﬀect of this change was that all his friends got a system generated status message from Gisle that said “Gisle is no longer listed as in a relationship”, which in turn resulted in a number of concerned friends contacting him to investigate what they interpreted as a terminated relationship.

Who you are, who you represent and the role you have in a situation are key to the negotiation of disclosure. For instance, some obstacles might be considered to apply in a speciﬁc context, and not be valuable to others. A user who considers an obstacle to be “private” must still be allowed to use the system, for instance for getting an alternative route, without this activity being persisted for later retrieval. Even when deciding to share the information with the community, the role has to be negotiated. Does the user represent “all” wheelchair users, or only users of motorized chairs or manual chairs? Does the user represent an interest organization with an agenda, or is he a casual user only using the system as a navigation tool? These are all questions that in the existing practice is negotiated from situation to situation, and similar ability for negotiation must be in place in the new practice. The role taken on by the individual is negotiated in the situation, and evolves over time as well. For instance, the amount of eﬀort put into the “cause” can change: “[These days,] I don’t sit down and write to the management and tell them how it should be. I don’t, because I get so angry, I go home because I’m so upset. [. . . ] So I don’t go to that church, because I don’t like it. I think the ramp is ugly. Now, in my younger years [. . . ] I wrote an angry letter to the council down there, right. But I left it at that. I was very angry.” The experienced accessibility is personal, however it can trigger actions taken on behalf of a larger group: “[. . . ] I don’t think much about the fact that other people are going to that church later. In a way, I have told them what I think. [. . . ] Well, I react on behalf of all [the other users]” And if the context is explicitly coordinated by a group, the role taken on can be very diﬀerent:

In this story, it was not Gisle who decided to make this a public announcement, it was the system. It illustrates

“Then, sometimes, the Association of Disabled has

222

Full Papers

Proceedings: NordiCHI 2010, October 16–20, 2010

decided to run a campaign against it [church accessibility]. And then I’ve gathered the church council. But I’m not taking on that ﬁght regularly.”

it’s users. Again, Facebook is a prime example, where current analysis shows recent dramatic changes in privacy policy, where Facebook Inc. has made changes to their policy for providing access to user information for third parties1 .

Negotiation of identity is an involved and dynamic process. Capturing this process in an ICT system is challenging, and requires careful attention in the system design process. We suggest that the negotiation will take place mostly outside of the system itself, which means that the system must be open to the result of the negotiation, for instance by allowing anonymous or non-traceable activity, and the ability to revoke information at a later stage.

This also raises an important question about who owns the service and associated contributed data, and who can change the terms under which they are shared with others. By agreeing to conditions of use whereby the service provider can modify the rules without consent from the users, the user is eﬀectively waiving the opportunity for negotiation of these issues. It is our recommendation that these issues must be publicly addressed and communicated to users prior to joining such a social service, and that any user should be allowed to easily opt out of the service and potentially remove traces of their activity and get their own data with them if they choose to do so.

To summarize, place, time and situation are key components in the process of negotiating ones identity - the answer to the question Who do I represent here, now?. An extra layer of concern is added when the reach and persistence changes are taken into consideration. The question then changes to Who do I represent here, now and later?. The time boundary is discussed next.

Another aspect of the time boundary is that of interpreting current activity in the light of previous activity. Again, an example from one of our subjects:

Temporality: Past, present and future

“. . . we’re very good at complaining when things are not OK, but we’re not very good at telling people that things are OK. [. . . ] I never go to the newspaper and tell them the new ramp is in place [. . . ] I like making a bit of fuzz, but I’m not very good at giving people a pat on the back.”

Arguably the most inﬂuential change to existing practice by introduction of a mobile, collaborative navigation system is that of persistence of information. Palen and Dourish address persistence by using the temporality boundary. In existing practice, it is largely taken for granted that communication about accessibility is of local and ephemeral nature. Sharing accessibility information through technology implies persistence of information, and thus the impact of sharing traces of activities becomes diﬃcult to assess, as is the negotiation of what to share with others.

By constantly and publicly being complaining about accessibility, there’s a risk of later being regarded as a negative person. This is true in existing practice as far as actively public statements (like in the newspaper) is concerned. By the introduction of ICT, the aggregated statements made by single users can portray a negative image of a user, even though the user is only leaving traces by making use of technology as a tool to solve local, personal navigational problems. The converse is also true, of course, as in the example with the goods lift, where a positive attitude towards a undesirable solution can be interpreted as legitimizing sub-standard solutions - thus implicitly making the activity trace represent the user group collectively, and not merely as coming from an individual user.

The example about accessible goods lifts is an example of this, and the user who presented this continues: “. . . basically because we see that there are, like, economic interests at stake, and they would eﬀectively create a barrier for many of us, in contradiction of rules and regulations. An we do deﬁnitively not want to be part of legitimizing that. You will ﬁnd that attitude, guaranteed.” This user has been active in organized accessibility work for many years, and clearly comes across as reﬂected about the possible interpretations of information, made by others at a later time. There are numerous examples of lack of this kind of reﬂection, perhaps most notable in current use of on-line social applications like Facebook. For many, the idea that someone “outside the group” will have access to relatively private information seems elusive, and one likely cause is the apparent ephemeral nature of on-line conversations, and in particular the lack of technological understanding by users [12].

One could argue that aggregating and anonymizing information could limit the risk of standing out from the crowd as a person who provides critical feedback, however the use of aggregation also introduces new problems. These includes a higher risk of the information being interpreted as representing a consensus view withing a group, and it also makes it more diﬃcult to assess the reliability of the information, which today is largely based on personal connections.

Further, changes in privacy policies for such on-line communities often introduces breaches of contract with

1

223

http://w2.eﬀ.org/deeplinks/2010/04/facebook-timeline/

Full Papers

Proceedings: NordiCHI 2010, October 16–20, 2010

CONCLUSION

3. Borchorst, N.G. and Bødker, S. and Zander, P-O. The boundaries of participatory citizenship. In ECSCW 2009: Proceedings of the 11th European Conference on Computer Supported Cooperative Work, 7-11 September 2009, Vienna, Austria, page 1. Springer, 2009.

The ﬁrst question that has been addressed in this paper is speciﬁcally about what privacy issues that emerge when social software technologies are applied for situations like accessibility mapping. Introducing new technologies is not only about what the speciﬁc technology “does”, but also about what existing practices are changed with the new technology in use. We have seen that there is a balancing act between the disclosure of information and managing privacy when using social systems for route planning. What was local and only disclosed to trusted others in an existing route navigation practice, can potentially be disclosed to all other users of the system now and in the future, and thereby changing the reach of the information.

4. I. Bratteberg and S. Kristoﬀersen. Irreversibility and forceback in public interfaces. In Proceedings of the 5th Nordic conference on Human-computer interaction: building bridges, pages 63–72. ACM, 2008. 5. G. Evans. Accessibility, Urban Design and the Whole Journey Environment. Built Environment, 35(3):366–385, 2009. 6. J. Grudin. Desituating action: Digital representation of context. Human-Computer Interaction, 16(2):269–286, 2001.

Through this study, it is seen that the everyday practice of route navigation is fundamentally changed when introducing social applications. This view is of importance, since it is often lost when discussing the potential beneﬁt of new, networked technologies. The second research question is about what forms of negotiation emerge in accessibility mapping. The framework presented by Palen and Dourish was applied in order to discuss the negotiation of boundaries related to disclosure, identity and temporality.

7. J. Grudin. Enterprise knowledge management and emerging technologies. In Hawaii International Conference On System Sciences, volume 39, page 57. IEEE Computer Society, 2006. 8. P.O. Hedvall. Towards the Era of Mixed Reality: Accessibility Meets Three Waves of HCI. In HCI and Usability for E-Inclusion: 5th Symposium of the Workgroup Human-Computer Interaction and Usability Engineering of the Austrian Computer Society, USAB 2009, Linz, Austria, November 9-10, 2009, Proceedings, pages 264–278. Springer-Verlag New York Inc, 2009.

The everyday practice of negotiating information disclosure is changed when introducing social software. The negotiation will both take place within the system, but also by using technologies and communication channels outside the system. Thus, the system must be open to the result of such negotiation.

9. H. Holone and J. Herstad. Social software for accessibility mapping: challenges and opportunities. In Unitech 2010, Oslo, Norway. Tapir Forlag, 2010.

Privacy issues are both about enforcement of laws and the practice of responding to speciﬁc circumstances, like when planning and making a journey. The impact that mobile, networked technologies have on the practice of handling privacy issues is important. Our understanding of privacy issues, and the way this changes with introducing new technology is of crucial importance for users, designers and analysts. By addressing speciﬁc use situations of social software, we hope to contribute to this important topic.

10. H. Holone, G. Misund, H. Tolsby, and S. Kristoﬀersen. Aspects of personal navigation with collaborative user feedback. In Proceedings of the 5th Nordic conference on Human-computer interaction: building bridges, pages 182–191. ACM New York, NY, USA, 2008. 11. L. Palen and P. Dourish. Unpacking” privacy” for a networked world. Proceedings of the conference on Human factors in computing systems, pages 129–136, 2003.

ACKNOWLEDGMENTS

We want to extend thanks to the ICCHP 2010 and Unitech 2010 anonymous reviewers, and to Sigurd Dahlseide and Torleif Gravelsæter for performing one of the interviews.

12. S. Patil and A. Kobsa. Uncovering privacy attitudes and practices in instant messaging. In Proceedings of the 2005 international ACM SIGGROUP conference on Supporting group work, page 112. ACM, 2005.

REFERENCES

1. B.T. Adler, J. Benterou, K. Chatterjee, L. De Alfaro, I. Pye, and V. Raman. Assigning trust to wikipedia content. In WikiSym 4 rd Intl Symposium on Wikis, 2008.

13. C. Satchell and P. Dourish. Beyond the user: use and non-use in HCI. In Proceedings of the 21st Annual Conference of the Australian Computer-Human Interaction Special Interest Group: Design: Open 24/7, pages 9–16. ACM, 2009.

2. V. Bellotti and A. Sellen. Design for Privacy in Ubiquitous Computing Environments. Proc. of the European Conference on Computer-Supported Cooperative Work, 273, 1993.

224

Full Papers

Proceedings: NordiCHI 2010, October 16–20, 2010

14. K. Schmidt and L. Bannon. Taking CSCW seriously. Computer Supported Cooperative Work (CSCW), 1(1):7–40, 1992.

17. T. V¨olkel, R. K¨ uhn, and G. Weber. Mobility impaired pedestrians are not cars: Requirements for the annotation of geographical data. In Proceedings of the International Conference on Computers Helping People with Special Needs (ICCHP’08), pages 1085–1092. Springer, 2008.

15. K. Schmidt and C. Simonee. Coordination mechanisms: Towards a conceptual foundation of CSCW systems design. Computer Supported Cooperative Work (CSCW), 5(2):155–200, 1996.

18. F. Winberg. Supporting Cross-Modal Collaboration: Adding a Social Dimension to Accessibility. In Proceedings of Haptic and audio interaction design: ﬁrst international workshop, HAID 2006, Glasgow, UK, August 31-September 1, 2006, pages 102–110. Springer-Verlag New York Inc, 2006.

16. B. Suh, E.H. Chi, A. Kittur, and B.A. Pendleton. Lifting the veil: improving accountability and social transparency in Wikipedia with wikidashboard. In Proceeding of the twenty-sixth annual SIGCHI conference on Human factors in computing systems, pages 1037–1040. ACM, 2008.

225

Full Papers

Proceedings: NordiCHI 2010, October 16–20, 2010

Transferring Qualities from Horseback Riding to Design Kristina Höök Mobile Life @ Stockholm University Forum 100, 164 40 Kista, Sweden [email protected] focus. The renewed HCI interest lies in designing for experiences – beyond efficiency and task completion [19]. In some respects the ergonomists were more sensitive of the body than we are.

ABSTRACT

We see more and more attempts to design for bodily experiences with digital technology, but it is a notably challenging design task. What are the possible bodily experiences we may aim to design for, and how can we characterise them? By analysing a horseback riding experience, we came to identify the following themes: (1) how certain kinds of bodily experiences are best understood through experiencing them yourself – the bodily ways of knowing, (2) how rhythm and balance create for particularly strong physical experiences of this kind, (3) how movement and emotion coincide in these experiences, (4) how the movement between seeing our own bodies as objects vs experiencing in and through our bodies is one of the ways we come to learn the language of expressing and understanding bodily action, and (5) how this in turn lets us describe the sensitive and delicate relationship of wordless signs and signals that represent, in the case described, two bodily agents – a human and a horse. When the human-horse relationship is really successful, it can be described as rare moments of becoming a centaur. We translate these themes into design considerations for bodily interactions.

As digital products have become an integral part of the fabric of everyday life, the pleasure they give, the contribution to our social identity, or their general aesthetics are core features of their design. We see more and more attempts to design for bodily experiences with digital technology, but it is a notably challenging design task. With the advent of new technologies, such as biosensors worn on your body, interactive clothes, or wearable computers such as mobiles equipped with accelerometers, a whole space of possibilities for body-based interaction is opened. But this begs for answers to three questions. First, what kinds of experiences are we aiming to design for? Glossing them over as all being about designing for flow [3] or inducing a game play experience is too vague [8]. We need to drill deeper and better understand exactly what experiences we are talking about. Are we designing for pleasurable or unpleasurable ones? Are we designing for those that are subjective and unique, or ones that are common and shared? Ones that deliver serendipitous experiences or ones that are evocative and emotional? These are not all the same, even if all may emphasise aspects of bodily experience.

Keywords

Embodiment, experiential qualities, bodily interaction, autoethnography ACM Classification Keywords

H.5.2 Information interfaces and presentation (e.g., HCI): Miscellaneous.

A particularly difficult issue lies in understanding how these experiences may unfold over time – both in the particular interaction with and manipulation of the artefact but also as parts of our everyday on-going lives. As Löwgren [17] puts it, a gestalt for interactive artefacts is defined as a dynamic gestalt which “we have to experience as a dynamic process” (p. 35 - 36).

INTRODUCTION

Originally, HCI spun off from ergonomics through its focus on the cognitive aspects of interaction. With the turn to third-wave HCI with its focus on experiences [2], we are now in a sense turning back to the body in interaction. But contrary to the main focus of ergonomics, instead of documenting and altering routines, finding error-free ways of involving operators in the complex beast of man and machine as a unity, or measuring how the body has its limitations, this turn back to bodily interaction entails a new Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. NordiCHI 2010, October 16–20, 2010, Reykjavik, Iceland. Copyright 2010 ACM ISBN: 978-1-60558-934-3...$5.00.

226

Second, once we know what kind of experience we are aiming to design for, we need to describe them in a form that makes sense and that we can share within a design team. Ways of knowing can arise from your bodily acts without any language translation in-between. The feel of the muscle tensions, the touch of the skin, the tonicities of the body, balance, posture, rhythm of movement, the symbiotic relationship to objects in our environment – these come together into a unique holistic experience. It is not the ability to fulfil a task, but the experience of the corporeality of doing so that matters here. Those descriptions also need to be shared with the users that we invite to test our designs, or even participate in the design process.

Full Papers

Proceedings: NordiCHI 2010, October 16–20, 2010

Thirdly, if we try to design interaction that builds on bodily movement, seeking certain experiential qualities, many different aspects of the interaction have to be fine-tuned to enable the experience, as, for example: •

•

•

the riding experience after each of the seven lessons. These notes were juxtapositioned with the written transcripts of the videos. I also discussed the material with two colleagues who helped me analyse the data. The accounts I provide below are sometimes quite detailed as I am trying to come to the essence of the riding experience.

the timing of interaction: movement has to render response in exactly the right moment for exactly the right kind of length of time in order to create for a particular experience [27] linking emotion and movement: certain movements and body postures are more likely to coincide with certain emotional experiences [4, 13, 25] harmony of modalities: the modalities of the interaction, such as graphics, haptics or gestures, all have to speak together – harmonize [26]

At the time, I was re-learning how to ride in English style as I had been riding Icelandic style for many years. The Icelandic style of riding is quite different from English: the Icelandic horses are smaller, have more gaits (including tölt and pace), the saddle is placed further back on the Icelandic horse, the stirrups are kept longer, and the horses are trained to interpret the signs and signals the rider emits (from legs, hands, body posture) differently. The way to ride English style is to try to be relaxed and balanced, not disturbing the horse through squeezing your legs too tightly or pushing with your rear muscles. The aim is not to put any tension into the horse – while on Icelandic horses, it is necessary to put tension into the horse in order to enable (unusual) gaits such as tölt. The account given here is therefore the story of an instructional situation. Teaching and learning are at core.

To shed some light on these issues, we decided to study a non-digital activity that is unambiguously about the body’s involvement – a mundane activity to some of us – horseback riding. Also, as it happens, horseback riding is an activity the author has a passion for. There is a plenitude of activities that we do for the pleasure of moving – dancing, sports, jogging, cycling. The pleasures of these activities are of course not only soft, nice movements, since some of the activities involve pain, applying yourself really carefully to make your body do them, adjusting your own body in various ways, even making your body build certain muscles that you normally do not use so much, embarrassment when you do not get it right, and so on. But they are still rewarding. The pleasure of moving a muscle that is aching, or the adrenalin rush when you ride your horse and almost loose control, makes some of us feel that we are living life to its fullest – away from the safe office environment were the excitements (and there are of course plenty of them) will not, to the same extent, move your muscles, bones, joints and stimulate the nervous system of the whole body.

THE FELT EXPERIENCE OF RIDING

There are many ways I could look at the data from this horseback-riding experience. I could look upon it mainly as a learning experience, analyzing each painful step of getting my body to behave in the English-riding style, and figuring out the ways by which my teacher Christian is trying to help me ‘see’ and correct my posture. This is similar to a study by Laurier and Brown, where they talk about Laurier’s father teaching him to see the various phenomena involved in fly fishing [14]. I could describe the ergonomics of the interaction between me and the horse in terms of errors, task completion time or bodily muscle acts. Or I could analyze how the two agents (me and the horse) are communicating. I will attempt to give some accounts of all those situations, but I want to approach the difference between describing various moments in the interaction (from what can be seen e.g. in a video of my riding), to instead describing aspects of the feeling of my experience. Rather than glossing over that experience as flow or other experiential qualities, we would like to describe the unique experience of horseback riding with Christian and his horses, named Blue, Henry and Liberty in terms that hopefully preserve the essential elements of that experience. Here I do not want to suggest that there is a unique subjective experience that I want to describe, so much as preserve what the American philosopher Quine called the inscrutability of reference: the actual thing that is empirically existent, for any and all persons [22]. I would like to describe how we sometimes become part of each others’ lived experience [5], how I experienced my own body, the horse’s body and the rare moments when we were in synch and I became totally absorbed by the situation.

STUDY SET-UP

The study is done by the author, who is not a professional horseback-rider. Instead, we portray a mundane activity of someone doing her favourite leisure activity, an amateur. From here on, I will therefore use the pronoun ‘I’ when I speak about my subjective, personal experiences of riding. The account is an autoethnography, that is a form of autobiographical personal narrative that explores the writer’s experience of life. It has been used in the sociology of new media, journalism and communication studies. Autoethnography (though controversial) has recently started to be used in HCI [15], most notably by Boehner and colleagues who also use it as a design method [1]. The study was done through video taping and taking notes of seven riding lessons. The videotaping was done both by the riding instructor, Christian, and by cameras placed next to the paddock where the riding happened. The instructor also watched one of the videos and made comments to it that I used in the analysis. All of what was said in the videos was transcribed, and snippets of video were cut out and analysed in detail. I wrote down detailed accounts of

The following themes are extracted from the analysis of my experience. They have to do with the various orientations to

227

Full Papers

Proceedings: NordiCHI 2010, October 16–20, 2010

my knees tightly to the horse’s sides and pushing with my sitting bones into the saddle, I could not understand what he meant. I tried making my heel the lowest point of my body rather than my toes, but I was still not getting it. It became a contrived position for my body, my heels were forced down and as soon as I forgot about pushing them down, they crept up again. Christian decided to help me understand where my weight should be through a special exercise. He asked me to stand up in the saddle, see Figure 1, at first while the horse was standing still, then while walking and finally while trotting. Standing in the saddle like this forced me to try and find my own balance without squeezing my legs to the sides of the horse whatsoever. Suddenly I realized what Christian had meant by putting weight into my heels. My whole weight had moved down to my heels and it was from there that I found my balance. I felt my own weight travelling down, gathering around my ankles, and the sole of my feet became like a floor. When I was allowed to sit down back into the saddle, I knew what the feeling should be – which muscles should be activated, what my ankles and feet should feel like and I experienced, inside my body, inside my balance, my own weight.

Figure 1 Standing in the trot to feel the weight in my heels, finally loosening my legs from Blue’s body.

experience that are constituted in the domain of horse riding: •

• • •

•

•

how certain kinds of bodily experiences are best understood through experiencing them yourself – the bodily ways of knowing, how rhythm and balance create for a strong physical, aesthetic experience how movement and emotion may coincide in this particular instructional setting how I move between seeing my own body as an object vs. experiencing in and through my body is one of the ways we come to learn the language of expressing and understanding bodily action how this in turn lets us describe the sensitive and delicate relationship of wordless signs and signals that represent, in the case described, two bodily agents – a human and a horse rare moments of becoming a centaur – that is feeling as ‘one’ with the horse

While I can rationalise what was needed afterwards and decompose exactly which body parts needs to be loose or tensed, it is not this decomposition that I learnt. I learnt a whole body schema, a wholeness that only made sense if I got all parts at the same time. This was a key element in the vocabulary of rider-horse relationships, how the sense of being a centaur is realisable. This insight did not come from being told a visual metaphor (I had already been told, repeatedly, in various ways to put weight into my heals), or from Christian showing me through riding himself what it should look like. In my case, I had to experience it myself in order for it to be meaningful to me. This is not to say that it is impossible to be told how to move your body or simply imitate and thereby constructing meaning. The more experienced you are in listening to bodily instructions, as a rider, dancer or in other bodily activities, the better you will be in following the instructions and thereby experiencing the movement. But the knowing itself does not come from the description of movement – it comes from the experience of it. Or in this case, the concept “putting weight into your heels” did not make sense to me until I had experienced it – the meaning of the words was intrinsically tied to the action. Without the action they conveyed insufficient knowledge to let a person ‘do’ the action in question

Bodily ways of knowing

The interaction with a horse is obviously not word-based. It happens through physical signs and signals: the riders use the muscles in their legs, the placement of their sitting bones, bodily balance, head movement, hand and arm connection to the horse’s mouth and sometimes tone of voice. The horse talks back through its movement, direction, pace, activations of muscles that can be felt throughout the horse’s body, its head movements, tail movements, flipping ears, bend of neck and noises. In order to be a good rider, you need to learn this wordless language. As in any language, understanding and communication arises in interaction over time. When you have experienced a particular bodily schema or concept yourself this understanding may arise. Let us provide one example of such a process.

Out of rhythm and embarrassment

As horse and rider move together, they create a rhythm. Depending on the gait, it can be a two-beat (trot, pace), three-beat (canter), or four-beat (walk, gallop, tölt), in different paces. To allow the horse to keep the beat in a balanced way, the rider needs to make herself invisible in the saddle, not disturbing the rhythm.

During the lessons, Christian kept saying that it was important that I put weight into my heels. This was where my balance was going to come from. As I had been squeezing

228

Full Papers

Proceedings: NordiCHI 2010, October 16–20, 2010

In one of the earlier lessons, as I was repositioning my body from Icelandic style to English, I totally lost everything. In particular, I lost my ability to follow the rhythm of the horse. I was ‘behind the movement’ as Christian phrased it when commenting on the video of me riding: “As you come into better balance the horse is more forwards. But then you lent forward and pulled on the reins so that he slowed down and fell in on the circle. There is still the tendency with the leg to go too far forwards so the seat of the rider is pushed to the back of the saddle and the shoulders come forwards so the rider is turned as what we would call ‘behind the movement’. You are not in time with the horse but you are always trying to catch up with the horse. You can actually see that quite clearly there. He is trotting in one rhythm and the rider is in a slightly different rhythm.”

Figure 2 Feeling very embarrassed by how badly things are going. I make a stupid remark, lean forward as if to release tension, sit down again, and pull my shoulders up, face upwards, giggling awkwardly.

horse compared to the small Icelandic horses. The movement was more of a huge big ship moving up and down, rather than the fast up and down rising on Icelandic horses. I had to make my body listen to and get into this bigger rhythm. As I was trying to adjust my way of sitting, experimenting with where my weight was, loosening the grips of my knees, avoiding to flex my back too much, pushing my heals down, and avoiding to push my sitting bones into the saddle, the rhythm did not come ‘naturally’. I did not have the ability nor prior experience of the feeling that would allow me to attune my body posture to how the horse responded, and make it respond the way I wanted.

The experience of being ‘behind the movement’ is very awkward. The trot is a 2-beat movement in which the horse’s legs move in diagonally opposite pairs separated by a moment of suspension. In a rising trot, the rider needs to rise from the saddle in every second stride. You need to let the movement come from the horse moving you upwards. When it does, you continue the movement forwards, rather than straight up. In the next stride of the horse, you then come back down, and if you are in rhythm, you will come back into the saddle, briefly, at exactly the right beat, immediately going up again. Your whole body needs to listen to the rhythm of the horse, because unless you are in perfect synch with the horse, the horse’s strides will not be perfectly even – your movement disturbs the horse.

Being inside a dynamically rapidly changing situation where you are simply not getting it right, it is very hard to make sense of what to prioritise, what to fix. When Christian stopped me, I was therefore totally devastated by the situation. I was out of synch with the horse and I had no hypotheses about what to do or what was actually wrong. I tried joking about it and made some embarrassed movements with my body, as you can see in Figure 2.

The problem was that I was sitting back into the saddle with a ‘splat’ slightly out of rhythm with the horse. Given how many years I had been riding before going to lessons with Christian, it was horribly embarrassing for me to be out of synch. Following the rhythm of the horse is one of the most important pleasures of riding. As discussed by others [21] rhythmic movement as in dance or riding, moves us in way which are immediately appealing. But just as it can be very awkward to watch someone dancing out of rhythm, it is very awkward to experience it. Being out of rhythm on the horse, is especially awkward as it inflicts on the horse to have the weight of an adult human rider bouncing down into its back at the wrong moment, just as the horse is about to rise.

In a sense, this is similar to Ingold’s description of plank sawing [9]. He nicely shows that no matter how experienced you are, sawing through yet another plank will never be exactly the same. There are number of different sensations that synchronise in a particular riding experience. Rapidly adjusting to each one of them, will in every new instance be a unique experience. Getting into the rhythm – being one with the horse

Towards the end of the third lesson, after having been embarrassed, out of rhythm and very confused, I had, in a way, given up all my good intentions of putting my body in the right place and working hard to make it work. Instead, I allowed myself to be distracted by Christian who was talking about something irrelevant, I relaxed and then I finally got the rhythm of the horse. Suddenly, without consciously trying to, I shifted my weight further back in the saddle and suddenly it worked. I rose at the right moment, I was relaxed in the way I was sitting and the whole thing felt fleeting and nice.

But my embarrassment did not arise only from knowing that Christian could see me and condemn my riding. Nor from my worry about how the horse had to deal with my weight bouncing into his back at the wrong moment, even if this also contributed to it. The embarrassment came from the actual physical experience of being out of rhythm. Our bodies are used to rhythms, our own bipedal swagger [25], our mother’s heart beat, waves beating the beach, music and dancing, and, for those who are fortunate enough to experience it: the horses’ different gaits.

At this point, as I was relaxing, just feeling the rhythm of the trot, I was not really thinking of where my different body parts are placed, nor was I thinking about the rhythm as such. I cannot say that any clearly formulated thoughts flew through my mind. Indeed, one could say that I am not

This particular incident happened when I was still trying to get to terms with how much bigger the stride was with a big

229

Full Papers

Proceedings: NordiCHI 2010, October 16–20, 2010

Figure 3 With my body in balance, more or less straight back, weight in my heals, in rhythm with the horse’s trot, rising at the right moment, and overall relaxed.

thinking at all. Thinking is precisely not what is required, I was simply entirely absorbed by the situation. My muscles were doing what they should be doing, my eyes were directed towards where we should be going, my hearing followed the rhythm of the horse’s hooves on the ground, and I felt as one with the horse. We were no longer separated but moved together with one intention, one will. My whole self was simple there, in synch with the movement.

trying to do then their body takes over their reflexes kick in and then people tend to ride in a more relaxed and better balanced way because they are not thinking too much about what they are doing, they are just doing it and that is better. So sometimes you just chat about going to the hairdresser or something.” When Alan Kay talked about how he worked through the ideas of the desktop metaphor and the use of the mouse for drag-and-drop, point-and-click, he picked up on the same insight [10]. He had been inspired by a TV-show of how a tennis teacher was able to distract his pupils by making them sing songs while learning to do backhands, forehands and serve. Through these distractions, the pupils were distracted and their bodies simply imitated the right physical movements without giving it much thought.

In Figure 3, we can see that my body is not leaning forwards, my legs are not too much forward (though still not perfectly in line with shoulder and hip), and I am looking in the direction of where we are heading. I asked Christian to comment on the video of me, and at this point, when I get into rhythm, he said: “Much better rhythm. And again, the rider is better balanced the horse goes forwards more fleeting. He is more even in his rhythm. A lot of this you can see straight, anybody watching this can see the difference. There is much better. You can argue that yes ok you are still a little bit forwards and the rider is still a bit behind the movement. But overall it is much better.”

Seeing my body as an object vs experiencing in and through it

When learning bodily practices such as tennis, golf or skateboarding [29], you occasionally will have to change your posture or aspects of your movements in order to correct some aspect of your performance. These changes can be very painful as they feel unnatural – the prior, faulty, behaviour keeps interfering. Those situations require that you consciously reflect upon different body parts, where they are, what they feel like in that position and how to move them into the new, required position or tension level.

He also confessed that he had used one of his strategies to get me into this state: “Very often, especially with adult riders or nervous riders, if you stop them thinking too intensely about what they are

Figure 4 My leg actively lifted away from the Henry’s side, without putting any weight into the heel, leaving me unbalanced (left). Leaning far too much forwards (middle) annoying Henry. Not putting any weight into my heel on Blue (right).

230

Full Papers

Proceedings: NordiCHI 2010, October 16–20, 2010

Christian, talked me through one of these re-positionings of my posture:

tion, see Figure 6, he touched or pointed to parts of my body that needed to relax or be more tensed, he rode the horse himself so that I could watch his body and imitate, and he made me do some ‘extreme’ exercises to provide me with a bodily experience (as described above when standing in the stirrups), or as he phrased it, helping me to create “a muscle memory” of where he wanted certain parts of my body to be placed or what it should feel like.

“Sit in the centre of the saddle, balance squarely on both seat bones, your back should be straight, your legs should relax along the horse’s sides, and the idea is that there is a straight line from your shoulder, through your hip to your heal. Your arms should fall softly by your sides. There should be a straight line running from your shoulder, through your elbow, through your wrist and down the rein to the horse’s mouth. There should be a ‘spring’ between your heal knee and thigh that makes it possible to do a rising trot. You should put your weight in your heels and not lean your body forwards but be straight.”

Through these exercises, Christian occasionally helped me to get into a position where I was balanced, experiencing the kind of interaction between me and the horse that he wished to see. Being in synch with the horse, following its rhythm, and being perfectly balanced myself, happened rarely. But at those moments when I went from actively thinking about where a certain body part was placed, to actually feeling that “this is a good position”, I also typically would switch from seeing that body part as an object from the ‘outside’, to instead being ‘inside’ my body, feeling the interaction from inside my body. In Figure 5, we can see one such moment. Again, my body is not leaning forwards, my legs and knees are relaxed, and my head is turned in the direction of where we are going. The horse also has a nice forward movement. It gave a sense of relaxed concentration – I am totally inside the experience, but in a relaxed way.

You may note how each body part has to be in the right position and how each muscle needs to be activated or relaxed exactly in the right way. In this description, there is an emphasis on being soft, relaxed and balanced. Christian kept emphasising that I should keep my own balance, that I should not disturb the horse. Christian used some visual metaphors to describe what it should feel like, such as “the horse should be flowing like a river through the reins, your arms, and between your legs”. As I had been riding in the Icelandic style for most of my adult life, the re-learning of how to sit proved to be very hard. The problem for me was that as soon as I tried to do something with the horse like steering it or asking it to move somewhere, rather than just thinking hard about how I sat or where my different limbs where, I immediately fell back into squeezing my knees tight to the horses sides, pushing my legs to be straighter and more forwards, and pushing my butt hard down into the saddle. My whole body was screaming that Christian’s way of sitting was not the way I was used sitting and I comfortably moved into the position my body ‘knew’ best.

Learning a movement, a posture, a rhythm, as was required of me here, is to try learn a habit or as phrased by Game [5], “the movement of music, riding, writing, lives in us as we live in it.” Being In the World Together

When things go really well, I feel the presence of the horse’s way of being in the world as part of my own and we act together. This sense of another agent being present relies on a process of recognition. The horse and my self become one, as the centaur mentioned above. But sometimes – indeed in my case quite often as I struggled with learning to ride in a new way with horses unfamiliar to me – my orientation to other, the animal in this case, evokes the idea of the mind and how it is somehow separate from the body. To be able to merge with a horse, one has to get the feeling that the horse’s mind and its body are as one, and the mind is not being mischievous in a way that leads the body to convey misleading signals.

When re-learning how to position my body, my body become an ‘object’ that I controlled with my will. Or rather, what has hitherto become a matter of habit, a way of the body that one takes for granted, now becomes something that I focused on. My mind starts to think again, about how the body should be. During my third lesson riding with Christian, he had decided that he was going to change my way of sitting and thereby influencing the horse. He did so by lounging the horse around him so that I could focus entirely on how I was sitting without having to steer the horse or control it. Halfway into the lecture, I was still sitting all wrong. Since Christian had asked me to relax my legs and last time he had complained about my knees squeezing, I was now letting them hang down totally free from the horse’s sides – even actively lifting them away from the horse’s sides, see Figure 4. To overcome my difficulties Christian had to find a way to show me what the problem was. Describing it through the visual metaphors he used had not helped. In the seven lessons I took with Christian, he used a range of ways of showing me: he showed with his own body what my body was looking like when I was in the wrong posi-

This sense of another mind being present relies on what Jane Heal describes as an empirical exercise of the imagination [6]. I recognize the individual horse, how I imagine it is thinking, it’s personality and I ‘talk’ to it. Through this operation, I come to see that it is not alien to me at all, but an individual that I emphatically recognise through its horselike way of thinking and being in the world and how it is trying to listen to me. The problem with Christian’s horses is that I did not know them as I was a newcomer to his stables – my ability to construct in my mind’s eye, a sense of them, of their personality, of their mind, could not work. The ultimate test of this failure: I did not recognise what they were thinking.

231

Full Papers

Proceedings: NordiCHI 2010, October 16–20, 2010

Figure 6 Christian showing how my butt is sticking out and my legs are pointing forwards (two leftmost pictures). Then showing what my body position should be, and how I can be rising from the position (three rightmost pictures).

Their stride was much bigger than the Icelandic horse and it made me feel that maybe this is a different kind of animal. Not a horse as I know them but some other kind of intelligence. These horses do in fact have quite different behaviours compared to Icelandic horses as they are more nervous, higher strung (a quality that breeding has emphasised as these horses were used in war and therefore bred to be very fast and have quick reactions).

interested in me. I therefore tried to distract him, make him listen to me. I pulled at the reins to say “hello, I am here – listen to me”. I shifted in my seat, and I made my legs ‘present’ in the interaction. But it felt as the more I tried to get in contact, the more he closed off and did not want to listen. He just became more silly, more easily spooked. I did not feel that I was in control of the horse, something that makes any rider very nervous. As I did not feel that I could decide what to do on the riding track (Christian is in charge of what we do after all), I could not use the tricks that I would have liked to use – going into smaller volts, changing direction, distracting Liberty and getting his attention through saying “here I am, I want to do these things, let us have fun together”.

Christian’s way of teaching and my focus on the way I was sitting did not give me much room to establish this kind of relationship or explore who the horse was. His horses were also very well behaved, allowing me to concentrate on my way of sitting rather than controlling the horse. An exception was perhaps the horse named Liberty. Liberty was a huge, big horse that kept chewing on everything in a playful style, mirroring a teenager. Not dangerous, but a bit nervous and silly. The kind of horse who like to pretend to be spooked by some dangerous shadow in a corner, allowing it to get away with jumping here and there in a way that makes riders like me totally distraught and immensely nervous.

As I was doing all of this ‘give me attention’-business, I fell back into a bad habit of watching Liberty’s head. By doing so, you can get some more signs of where the horse is heading, what it is thinking or looking at, and whether it is listening to you. But the rider should not be watching the horse’s head. My head should be directed towards where we are going, since the balance of my head and the position of my shoulders tell the horse where we are heading. I also need to plan ahead where we should be going next, preparing the horse and myself for what is going to happen. If I look in the direction of where we are going, I will still see the horse’s head in the periphery. And by looking at where we are going and planning our route, I stay in control, rather than acknowledging and giving the horse the control. On and off, I tried to stop watching Liberty’s head, but because of this lack of contact or dialogue between us, I kept looking back at his head to be prepared for any action he would take. We were not ‘one’, but two that were not collaborating.

At the beginning of the lesson with Liberty I was immensely tensed. Liberty has already on our way towards the riding track decided that there were ghosts everywhere and was ‘spooked’. I could feel his tension and how it increased by my tensed movements. My butt-muscles and armmuscles were extremely tensed. I tried to make them relax, but failed. The tension in my arms made them straight and stiff rather than bent at the elbow and relaxed along the sides of my body. Christian could see how tensed I was and asked me to stop being so silly. During the lesson, I felt that Liberty was not listening. My feeling was that I needed to take more control, provide more signs to make him listen to me and start collaborating. I did not feel that Liberty was testing me, more like he was genuinely into his own thing, being spooked, and not that

Towards the end of the lesson we finally connected. I was asked to canter, and for some reason, this made a lot of my tensions go away. I took more control in the canter, was not

Figure 5 Finally, I am in the right position, balanced, and Henry is “flowing freely” with a forward gait (more or less).

232

Full Papers

Proceedings: NordiCHI 2010, October 16–20, 2010

worried that he would start being spooked. As the speed increased, some of the nervous energy was let loose, and the steering was easier. The speed and the rhythmic beat of the canter are exhilarating in themselves – it became physically enjoyable. In particular, I felt that the canter helped me to get into rhythm and after each canter both Liberty and I were more loose, relaxed, flowing in our joint stride. Liberty’s canter was big in its stride. It had a nice big, rolling rhythm. Together we went from canter to rising trot on and off. Both felt really good and relaxed. Liberty became more interested in the dialogue. The experience of it was fun, which in turn made the whole thing work. I only had to think ‘forwards’ or ‘let’s canter’ and it happened.

agents with quite different ways of being in the world – my own human body’s perceptual way of being in the world, vs. the horse body’s perceptual being in the world. I learn to express with the horse so that I, as a rider, can come to emphatically understand how the horse lives in its world – our world in common. When I do, and when I manage to capture the horse’s interest and attention, we sometimes get into an experience where there is only one being, one center of experience, a centaur, an animal of two in a world made the same for both: for the horse and for me. But when this is achieved, the two do not entirely merge. Important in horse-back riding is to leave space for the horse do its job – there is this space that I need to open for the horse to prefer to go there. I need to be out of the way, balanced in my own body, to not disturb the horse and its movement. As a rider, I found it very interesting to hear Christian’s account of how he could see when I was sitting wrongly or inflicting in a bad way on the horse:

To make this happen, true sympathy is required [5]. You have to recognise the otherness and difference in the horse and create a lived experience together. You have to forget about your own human self and instead turn yourself into a centaur self – consisting of two agents acting together. It is not without risks or wilderness, you might fall, the horse might take off in an unexpected direction. But when the connection and collaboration between the two minds happen, we both enjoy it.

“I watch the horse. The horse tells me what the rider is doing. The shape of the horse. The dynamic confirmation. The shape of how the horse is moving. We saw that earlier on this video clip earlier when you were out of balance and the horse was going very short in his stride and he was coming into the middle of the circle all the time. And then when you became better balanced he opened his stride and went into the bigger circle.”

TRANSFERRING LESSONS LEARNT TO DESIGN

In my view, there is no need to replace the horseback riding experience with a digital riding experience. This is not why I wanted to describe this experience. Instead, I want to transfer some of the qualities in the human-horse interaction into other applications we might want to create with digital technology – what Ljungblad and Holmquist name transfer scenarios [16]. Their idea is that we can learn from people’s specific practices and use the qualities of those practices and transferring them into innovative design that can attract a more general audience. They have tested this process on e.g. studying reptile owners and transferring qualities of their relationships with their animals to the design of home robots. The difficult step in their method is narrowing down on (1) what the qualities are, and (2) which are worth transferring and (3) how to implement those in an innovative design. Here, I want to suggest what some of the more important qualities from my horseback riding experience that we may want to transfer to design.

This could be a quality of design that we might find interesting to transfer to some kinds of human machine interactions – such as robot-interaction or interaction with various intelligent systems in our environment. Rather than making that relationship an explicit language, it could build on bodily signs and signals, allowing robot and machine to create a mutual understanding (cf. the work by Sengers [24] and discussions on machine intelligence by Taylor [28]). Bodily Learning

Many of the lessons learnt above where about bodily training. I found learning processes that to a large extent are already known from the literature on learning: •

I also want to point out that while others have also considered the importance of the body in design, e.g. [7, 12], their focus is most often on how to improve a function or serve a another purpose in the interaction, not on the aesthetics and enjoyment as an end-goal in itself (with some notable exceptions [21, 27, 30]).

•

•

A sensitive and delicate relationship of wordless signs and signals

The relationship between the horse and myself is based on non-verbal, very subtle signs and signals – a language of sorts even if not lexographical. There is a mutual space of signs and signals that we are involved in – to some extent translated by Christian, the instructor – but to a large extent happening between the two of us. This language worked to bring us together even though I assume that we are two

If you distract the conscious self, you can make people use their ‘muscle memory’ or imitate and thereby get it right [10] That you can get into an all-absorbing state as I did towards the end of the lesson – which is similar to flow, being in the zone in sports, and a range of similar descriptions That we move back and forth between states where we can think about body parts and where they are placed or what they are doing as objects or instruments (present at hand) and states where we are inside the experience doing the thing (ready to hand)

But when we design interactional systems involving our bodies, we rarely discuss how this bodily learning process is supposed to happen. Intuitively, many designers build these kinds of processes into their systems, but there is no systematic description of how to do it. In a sense, HCI-

233

Full Papers

Proceedings: NordiCHI 2010, October 16–20, 2010

research has been very concerned with zero-learning time. But as we could see from my account of riding, a great deal of the pleasure comes from applying yourself really hard, learning the movements that are needed in order to get to states of aesthetically appealing rhythmic interaction. Rhythm – an aesthetic experience

I have kept coming back to rhythm, as it is an important aspect of riding. At the same time, it is an interaction quality we rarely talk about in our design processes even if we know that any interaction, be it bodily or a more traditional desk-top system, reveals its dynamic gestalt over time and space [17]. Any such interaction will have its rhythm – its ebbs and flows, intense or more slow reflective parts. But in the horseback riding, rhythm is a more intense experience, strongly intertwined with all the different parts of the riding activity. Many of our everyday bodily experiences that we may want to mirror in interface design for sports, life style and heath applications will be rhythmically oriented. They may follow our circadian rhythm (biological processes following our 24-hour day), heartbeat, movement-oriented rhythm (picking up on walking or running behaviours), or everyday events (sleep, work, lunch, work, home). In my group, we have designed one such system named Affective Health [18, 23]. It picks up on bio signals from sensors placed on the body, transmits them in real-time via Bluetooth to your mobile phone were they are portrayed as an animated, pulsating torus. Similarly, the artist George Khut has created a range of art installations where visitors can see their bodily states mirrored in captivating, colourful animations [11]. These are just a couple of examples of interactions where our own bodily rhythm makes the interaction come alive – obviously we could imagine many more. Especially interesting are rhythmic interactions where the technology has some agency, moving us rather than us moving it. Describing Movement Experiences

Above, I tried to provide an account of exactly how I was sitting, which muscles were tensed, how I balanced on my sitting bones, the direction of my head and vision, the position of my legs and the tension in arms and hands. Still, the experiences I am trying to describe are wordless, and putting detailed descriptions of them still fails to cover the complexities and the uniqueness of my embodied experience. We rarely describe bodily, experience-oriented, system interactions in such a detailed way (in CSCW, there are system interaction descriptions based on conversation analysis, but their focus is mostly on the functionality, not on the subjective experience.) But in order to design for such interactions, we will have to do so. In particular, we need to start considering the aesthetics of movements. Why, for example, certain rhythmic movements feel awkward, and other states makes us leave our analytical selves behind, and instead allows us to be all-absorbed by the experience. Through my description of riding I want to show how those bodily postures corresponded to my experience of being in those lessons with Christian. That experience was a mixture

234

of my bodily posture, my expectations on my own expertise and embarrassment of having to relearn so much, my interaction with the horse, with Christian, and with the physical location – my embodied, lived experience. When MerleauPonty writes about the body he begins by stating that the body is not an object [20]. It is instead the condition and context through which I am in the world. Our bodily experiences are integral to how we come to interpret and thus make sense of the world. Our experience of the world depends on our human bodies, not only in a strict physical, biological way, through our experiential body, but also through our cultural bodies. Merleau-Ponty [20] attempts to get away from the perspective of the doctrine that treats: “perception as a simple result of the action of external things on our body as well as against those which insist on the autonomy of consciousness. These philosophies commonly forget – in favour of a pure exteriority or of a pure interiority – the insertion of the mind in corporeality, the ambiguous relation without body, and correlatively, with perceived things”. (p. 3-4) Through integrated descriptions of both our experiential and cultural bodies, living everyday, mundane, practices such as horseback riding, we may be able to reach a body of work that we can use in the design processes. Similar to any artists, as designers we have to practice our sensitivities in seeing, feeling, doing, hearing bodily experiences. SUMMARY

From a detailed account of my horseback riding, I have extracted a number of experience-oriented qualities. I have hinted at how those might be considered in design processes, such as: • • • •

Designing spaces for mutual wordless understanding between human and machine Letting bodily learning take time and be a pleasure in itself Putting more emphasis on the aesthetic pleasures of rhythm when designing for bodily interaction Finding ways of describing experiences of bodily interactions that can serve as inspiration to design

Hopefully these kinds of insights can take us beyond designing for the experiences of the body to the narrow domain of games and entertainment. In my view, we need to return to being more concerned about our bodily experiences, returning to the care that ergonomics shows the body, but with a stronger focus on experience, and not only function. Acknowledgments

This study was done while I was a visiting researcher at Microsoft Research in Cambridge, U.K. I would like to thank Richard Harper and Alex Taylor for comments on this study and their help in the analysis of the data. REFERENCES

1. Boehner, K., Sengers, P. and Warner, S. (2008). Interfaces with the ineffable: Meeting aesthetic experience

Full Papers

Proceedings: NordiCHI 2010, October 16–20, 2010

on its own terms. In ACM Transactions on ComputerHuman Interaction, 15 (3).

16. Ljungblad, S. Holmquist, L-E. (2007) Transfer scenarios: grounding innovation with marginal practices, Proc. of the SIGCHI conf. on Human factors in computing systems, San Jose, California, ACM Press.

2. Bødker, S. (2006) When second wave HCI meets third wave challenges. Keynote paper, Proc. of NordiCHI.

17. Löwgren J. (2001). From HCI to Interaction Design. In Chen, Qiyang (ed.), Human-Computer Interaction: Issues and Challenges, Hershey, USA: Idea Group Inc.

3. Csikszentmihalyi, M. (1990). Flow: The Psychology of Optimal Experience. Harper Perennial, London. 4. Darwin, C. (1872/1998) The expression of emotions in man and animals, Third ed. by Paul Ekman. Oxford University Press, 1872/1998.

18. Kosmack Vaara, E., Höök, K., Tholander, J. (2009) Mirroring bodily experiences over time, Proc. of the 27th international conf. extended abstracts on Human factors in computing systems, Boston, MA, ACM Press.

5. Game, A. (2001) Riding: Embodying the Centaur, 7 (4): 1, Body & Society, 2001; 7; 1.

19. McCarthy, A. Wright, P. (2004) Technology as Experience. Cambridge, MA: The MIT Press.

6. Heal J. (1989). Fact and Meaning: Quine and Wittgenstein on Philosophy of Language, Blackwell, Oxford.

20. Merleau-Ponty, M. (1962). The Phenomenology of Perception, C. Smith, trans., London: Routledge & Kegan.

7. Hillerup Fogtmann, M., Fritsch, J., Kortbek, K. J. (2008). Kinesthetic interaction – revealing the bodily potential in interaction design, Proc. of the 20th Australasian Conf. on Computer-Human Interaction: Designing for Habitus and Habitat, 2008, Australia.

21. Moen, J., (2006). KinAesthetic movement interaction: designing for the pleasure of motion. Doctoral Thesis, KTH, NADA, Stockholm, Sweden. 22. Quine, W.V. (1960). Word and Object, MIT Cambridge.

8. Isbister, K. Höök, K. (2009). On being supple: in search of rigor without rigidity in meeting new design and evaluation challenges for HCI practitioners, In 27th international conf. on Human factors in computing systems, Boston, MA, ACM Press.

23. Sanches, P., Höök, K., Kosmack Vaara, E., Weymann, C., Bylund, M., Sjölinder, M. (2010). Mind the Body! Designing a Mobile Stress Management Application Encouraging Personal Reflection, In Designing Interactive Systems, DIS 2010, ACM Press.

9. Ingolds, T. (2006). Walking the plank: meditations on a process of skill, in J. R. Dakers (ed), Defining technological literacy: towards an epistemological framework New York: Palgrave Macmillan, 2006), pp 65-80.

24. Sengers, P. (1998). Anti-Boxology: Agent Design in Cultural Context. Ph.D. Thesis, School of CS, Carnegie Mellon University. Technical Report CMU-CS-98-151.

10. Kay, A. (2003). Education in the digital age. (27:45) http://video.google.com/videoplay?docid=2950949730059754521&hl=en.

25. Sheets-Johnstone, M. (1999) Emotion and Movement: A beginning Empirical-Phenomenological Analysis of Their Relationship, Journal of Consciousness Studies, 6, No. 11-12, pp. 259-277.

11. Khut, G. (2006) Interactive Art as Embodied Enquiry: Working with audience experience. In Edmonds, E., Muller, L., & Turnbull. D. (Eds.) Engage: Interaction, Arts & Audience Experience. University of Technology, Sydney, Creativity and Cognition Studios Press.

26. Ståhl, A., (2006). Designing for emotional expressivity. Unpublished Licentiate Thesis in Design, Umeå Design Institute, Umeå University, Sweden.

12. Klemmer, S., Hartmann, B., Takayama, L. (2006). How bodies matter: five themes for interaction design, In Proc. of DIS 2006: ACM Conf. on the Design of Interactive Systems. State College, PA, 2006.

27. Sundström, P., Ståhl, A., and Höök, K. (2005). eMoto Affectively Involving both Body and Mind, In Extended abstract CHI2005, Portland, Oregon. 28. Taylor, A. S. (2009). Machine intelligence, Proc. of the 27th international conf. on Human factors in computing systems, Boston, MA, ACM Press.

13. Laban, R., Lawrence, F.C., (1974). Effort Economy of Human Effort, Second ed. Macdonald & Evans Ltd., London, UK.

29. Tholander, J., Johansson, C. (2010). Bodies, boards, clubs and bugs: A study of bodily engaging artefacts. In Extended Abstracts 28th ACM Conf. on Human Factors in Computing Systems, Atlanta, ACM Press.

14. Laurier, E. and Brown, B. (2004) Cultures of seeing: pedagogies of the riverbank, Manuscript, Institute of Geography, Edinburgh Univ., 2004 online: www.geos.ed.ac.uk/homes/elaurier/texts/Laurier_cultur es_of_seeing.pdf

30. Tweneboda Larssen, A., Robertson, T., Edwards, J. (2007). The feel dimension of technology interaction: exploring tangibles through movement and touch, Proc. of the 1st Intern. Conf. on Tangible and Embedded Interaction, ACM Press.

15. Ljungblad, Sara (2009) Passive photography from a creative perspective: “If I would just shoot the same thing for seven days, it's like...What's the point?”. Proc. of 27th International Conf. on Human Factors in Computing Systems, Boston, MA, USA, ACM Press.

235

Full Papers

Proceedings: NordiCHI 2010, October 16–20, 2010

Controlling the Use of Collaboration Tools in Open Source Software Development Heli Ikonen, Netta Iivari & Henrik Hedberg Department of Information Processing Science, University of Oulu P.O. BOX 3000, 90014 Oulu, Finland [email protected], [email protected], [email protected] ABSTRACT

model relies on communities where large amount of people can improve the software, adapt it to their personal or organisational needs, and fix defects. [31] The most wellknown OSS products are Linux operating system, Apache web server, OpenOffice.org application suite, and Mozilla Firefox web browser.

This paper analyses control in the open source software (OSS) development context, focusing specifically on how the use of collaboration tools, such as bug trackers and mailing lists, are controlled in OSS projects. The tools are vital for the functioning of distributed OSS projects. One OSS case project was analysed in the paper. The findings show a surprising amount of control in the OSS project. In addition, when compared to traditional information systems (IS) development projects, different control modes and mechanism were found. Strong evidence for informal ‘clan’ and ‘self-control’ were found, but also a lot of formal control was in use. The results are discussed in connection to OSS literature and critical IS literature.

The OSS project is often depicted as a hierarchical or onion-like structure [5, 6, 13]. Usually, the decision-making power is centralized on a core team consisting of one to a few skilled and recognized developers. Other active participants positioned in the next layer are often called 'codevelopers' or simply 'developers' emphasising the importance of the programming. However, it must be noted that there are also other tasks in OSS projects, such as documenting, web page and other infrastructure maintaining, and testing. The outer layer of the OSS project structure consists of users. The furthermost individual is solely using the software, but some users may also communicate with the project by, for example, filing bug reports. The granularity of the structure may vary from project to project, but in most cases it forms the same hierarchical setting with very few developers in the centre making decisions and increasing amount of individuals around them with lower activity and communication levels.

Author Keywords

Control, open source software, collaboration tools ACM Classification Keywords

H.5.3 Group and Organization Interfaces INTRODUCTION

This paper examines control in the open source software (OSS) development context. More specifically, the research question addressed in this paper is: 'How is the use of collaboration tools controlled in OSS projects?' OSS means software of which source code is available for everyone to read and modify. Open Source Initiative defines OSS through its license, which has to comply with the given criteria (see http://www.opensource.org/docs/ definition.php). The key idea is to let software evolve freely without any restrictions to its usage or development. Traditional software development is carried out in closed settings, where only a few paid people can access the code, and end users have just the binary form of the software enough for using it. On the contrary, OSS development

The OSS development process has been introduced as a solution to software industry's problems of slow and expensive software development that does not fulfil quality expectations [10]. It has been argued that OSS can even challenge commercial software products in the quality of the product as well as with their development speed and cost [31]. OSS communities are in many cases based on voluntary work [9]. However, OSS development has been noticed to evolve into a more commercial direction. OSS developers are increasingly getting paid to do their jobs and the users are more often willing to pay for open source products or the supplemental of open source products. [11]

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. NordiCHI 2010, October 16–20, 2010, Reykjavik, Iceland. Copyright 2010 ACM ISBN: 978-1-60558-934-3...$5.00.

In OSS development it is usual that the members work at geographically dispersed locations. Carmel and Agarwal have argued that geographically dispersed organizations cannot work efficiently without control and collaboration. Collaboration to these researchers is "the act of integrating each task with each organizational unit, so the unit contributes to the overall objective" and control is "the process of adhering to goals, policies, standards or quality

1 236

Full Papers

Proceedings: NordiCHI 2010, October 16–20, 2010

levels." [4: pp. 23] A geographically dispersed organization can form social relations and create organizational tacit knowledge. It has been argued that this happens through a collaborative development environment. [3] Due to geographical dispersion, OSS developers need to have reliable and efficient collaborative development tools that offer them a channel to communicate and to distribute the work tasks between them [23]. This way of working also requires collaboration systems to work efficiently [9]. However, open source communities are organized quite freely and due to this, methods, processes and tools are heterogeneous [5]. Most commonly used development tools in OSS communities are OSS solutions themselves. Widely known and used collaborative development tools are bug trackers (e.g. Bugzilla), revision control systems (e.g. CVS Concurrent Versions System), automatic code builders (e.g. Tinderbox) and different way to communicate via Internet (e.g. mailing lists). [22]

developers. [25, 32, 34] Therefore, OSS development has made the long-standing boundary between design and use very fuzzy [1, 15]. Furthermore, OSS development has blurred the boundary between leisure and work, since it typically relies on voluntary developers with motivations such as gaining a reputation among peers, loving to code, altruism, fighting against proprietary software, learning and scratching own itch, but also companies and paid developers have entered the scene with economic motivations [2]. In this paper, we complicate the picture even further by examining the OSS developers as users of their collaborative development tools, and focus particularly on how these users and their tool usage are controlled in this voluntariness and openness emphasizing context. This control is partly enabled by the way these tools have been designed, which bears some implications for HCI research and practise. The paper is structured as follows. The next section discusses existing research on control in traditional IS and OSS development contexts, as well as outlines some studies critically addressing the issue of control in IS context. In the third section, the research setting and data collection and analysis methods are outlined. The fourth section presents the findings of our empirical examination, the fifth section discussing their implications. The final section summarizes the results and their limitations and identifies a number of paths to future work.

There is a lack of research on the adoption and use of these collaborative development tools in OSS projects. However, there is an ample literature base on the adoption and use of different kinds of information systems (IS) in organizational context. This literature has long ago indicated that the success of IS implementation is not selfevident. Orlikowski, for example, has shown in numerous studies (e.g. [26, 28]) that there might be huge differences between the goals of the management and users, and users might be very powerful in obstructing the implementation process. This may be the case even if the system was technically of good quality. Users may still choose to use it in different, surprising or even in harmful ways. [28]

RELATED RESEARCH Previous studies about control

Control covers all the methods and processes that aspire to predictable behaviour. In IS context, the target for control is the user and his/her way of using it so that it would be consistent. [16] Control is usually divided into two subcategories: formal and informal [29, 16, 4]. Formal control typically consists of budgets and explicit guidelines. Informal control could be for example peer pressure. [16] In this study, we have chosen to use Kirsch's portfolio of control to recognize and classify different forms of control. Kirsch was chosen for real examples of control in commercial information systems development (ISD) projects. Kirsch’s definitions are based on Ouchi's view of control, which is widely used also in IS studies. Kirsch has divided the two subcategories of control into four modes: behaviour, outcome, clan and self-control. The control mechanisms that are used to exercise the control mode overlap among the different modes. The implementations of these four modes are introduced next.

The traditional way to control how an IS is to be used is to provide user trainings in the implementation phase. In groupware systems’ case, this has been said to be particularly efficient. [27] In training sessions, users can compare their experiences and create shared work practices for using the system. Also guidelines, walk-throughs and recommendations by expert users may be used for establishing the desired usage patterns. [28] Furthermore, project plans and meetings can be used for this. [16] Supervisors are typically assumed to exercise control, but actually it is not always related to a relationship between a supervisor and a worker, but control can also exist between same level workers, even in hierarchical organization [17]. OSS development process is a complicated phenomenon that is influenced by technology, human behaviour, economics, culture and society [32], offering a new and fascinating research context for Human Computer Interaction (HCI) research. Traditionally, software has been developed by technically skilled engineers for people without these skills, but in OSS development the boundary between developers and users has been blurred. The developers tend to be the users of the software they develop, and all users need to be considered as potential

Behaviour control is a formal control mode that is implemented with rules and procedures. There are rewards based on following those rules. Behaviour should be visible and observable. Outcome control is also formal control mode that is implemented by the outcomes and goals. Also with this control mode rewards are based on producing

2 237

Full Papers

Proceedings: NordiCHI 2010, October 16–20, 2010

those goals. Outcomes should also be measured against those goals. [16]

regulating relations, and where the network consists of individuals rather than institutions." So he does not see control or regulations as important factors in OSS development model. [19]

Clan control mode is implemented in common values, beliefs and problem solving philosophies. Also the identification and reinforcement of acceptable behaviours is implemented. New members are selected with care and taught to follow the clan regulations. These regulations are commonly unregistered. It is also noticed that specific task goals evolve over life of the task. Finally, self-control mode is implemented with task goals and procedures that individual define. Individual also monitors, rewards and sanctions self. Rewards are based partly on individual's self-control skills. Self-control is visible in individual's efforts. [16]

In another study, the results are somewhat the same. Persson et al. concluded in their OSS project case study of Argo project that control is weaker in an OSS project than in a typical commercial systems development enterprise. They specify this as lack of predictable time scales. They also notify that the developers in Argo have wider access to the code base that they would have in a commercial project and this can be seen as a burden to developers as they cannot focus on their specific modules. [30] Thus, it seems that previous studies have come up with contradictory conclusions about control in OSS communities. The way of working openly has been seen as an advantage and a disadvantage. The aim of this study is to clarify OSS communities’ way of controlling by focusing on their collaborative development tool usage, as using them is essentially important to OSS communities as many researchers have concluded.

Also users of IS can be involved in controlling an ISD project [16] In addition, communication is nowadays a mediating factor in both control and collaboration [4]. In overall, it is important to understand that control is directed to the interaction between the user and the system. This is based on the view that qualifies technology as a neutral element and suggests that all the factors that determine and constrict the usage of the system are coming from the user's attributes or from the organization's attributes in which the user is a part of. [26, 28]

Critical studies

In this study we also approach controlling from a critical point of view. Studies concerning controlling have introduced many different critical views to it. In this chapter, we introduce some of them, which have also inspired us completing this study.

Previous studies about open source and control

OSS development process has been characterized as meritocratic. Decision power is normally concentrated on only a few people. This has also been seen as a problem: for governance and quality practices trusting only to a few people's know-how the quality of the software is depended on how good these few people are. [24]

It has been said that control is based on invisible social norms and relations. Control can therefore provide an efficient way of using a system consistently or as an opposite; control can also completely prevent the usage of a system. Doolin has carried out an empirical study of how an IS can be used in controlling a workplace. According to him, the thought that IS users are only items that the system monitors is far too simplistic. Users can always choose to work in another way. Interesting comment that Doolin represents is that the introduction of an IS can lead to an organization-wide debate about what the system can monitor. The users can question the entire system and the need for it when they question the monitoring and controlling aspects of the implemented system. Doolin has said that using too many resources for governance can add the possibility for the resistance of the system. Too much control can also make users questioning the purpose and usage of the whole system. [8]

Latteman and Stieglitz have proclaimed OSS communities to be highly depended from different support and communication systems. They mention revision control systems, news groups and mailing lists as examples. They particularly emphasize the importance of these systems during the community’s growth. [18] Gallivan has conducted a study where he examined the results of nine different OSS case studies and focused on the findings of control and trust. His discovery was that OSS projects have clearly used a variety of control mechanisms and policies. Gallivan also suggests that the OSS projects cannot operate effectively based only on trust. [14] Lundell et al. have studied communication, cooperation, and control in the OSS development context. They conclude that control in OSS projects is generated through maintaining a shared vision that guides developers to share a common understanding of the goal of the project and protects it from forking (branching). [20]

Zuboff has also argued that too strict control can lead to a deskilling of user and the freedom to use a system differently should be seen as a learning possibility and may lead to an increase of new skills. Zuboff has stressed the possibility for the negative consequences of technologyassisted control and surveillance. IS can store all tasks and operations and there may be no one supervising them. [35]

Ljungberg argues that OSS project "could be seen as a virtual organization, but where trust has replaced law in 3 238

Full Papers

Proceedings: NordiCHI 2010, October 16–20, 2010

Earlier we noticed that OSS development process is a complicated phenomenon influenced by technology, human behaviour, economics, culture and society [32]. In a larger perspective it is worthwhile to notice how control is a part of the society and its trends. Many scholars (Zuboff, Lyon, Deleuze) have used a prison type designed by Jeremy Bentham, the Panopticon, as a metaphor for computermediated surveillance and control. The first who made Panopticon well-known was philosopher Michel Foucault who saw it as "both a sign of and metaphor for this new disciplinary society" [35]:

One OSS project was selected as a case for this study. Before selection the following criteria that the case should fulfil were defined: the project should have publicly available information about project management and it should have a public bug tracking system. The project should be medium sized so that there would be enough data for analyzing without being over-exhausting. Too large and too small projects were discarded. Other criteria were that the project should be active one and develop an open source licensed software product. The selected case project is based completely on voluntary work. The project has forked from different project in 2002 and it publishes software under LGP-license (Lesser General Public). Project has documented their project activities remarkably well in project's wiki. The project has also defined its purpose in vision document in the wiki. It is argued that this community should be driven by “innovation” and “experimentation” and that it should be “fun”, “stimulating” and “educational” (Project's Community Vision, wiki, 2009).

"Hence the major effect of the Panopticon: to induce in the inmate a state of conscious and permanent visibility that assures the automatic functioning of power. So to arrange things that the surveillance is permanent in its effects, even if it is discontinuous in its action; that the perfection of power should tend to render its actual exercise unnecessary; that this architectural apparatus should be a machine for creating and sustaining a power relation independent of the person who exercises it; in short, that the inmates should be caught up in a power situation of which they are themselves the bearers." [12: pp. 201]

The data was collected in a time period of May to October 2009. At the time there were about 20 developers in the project with two of them as lead developers. The project stored most of its development related data in the wiki. It also functioned as the project’s public web pages. In addition, the project used several other collaborative development tools: mailing list, bug tracker Mantis, revision control system Git and IRC for communication. The project had stored mailing list conversations to public archives since 2006. The mailing list and Mantis were examined totally. There were 1813 bug reports on Mantis and about 1500 e-mails in mailing list archive at the time of data gathering. Between the years 2006 and 2009 project had organized several IRC meetings. The minutes from these meetings were stored to a website which was publicly available. All these minutes were collected and examined for this study. From the wiki, 8 different web pages with their history data were collected and analysed. The collected pages represented guides and definitions of different work tasks. In addition to all material being publicly available, the permission to use data was asked from a lead developer of the project.

Deleuze argues that the disciplinary society that Foucault represented is already changing to a control society where people are confined by continuous control and instant communication. Control for him has especially taken place in the continuous monitoring of people. He also suggests that control in society cannot be explained just by the machines and computers that are used for control; it must be seen as collective arrangements where the machines are only one component. [7] Also Lyon has made a significant contribution in the area of surveillance and control. He writes "... to a virtual reality where the spaces and times to be controlled are manufactured ones. The environment can be perfectly controlled. Indeed, it no longer requires control; the environment is control". [21] RESEARCH DESIGN

This research was conducted as a qualitative case study. The case study method aims to understand complex social phenomena such as organizational processes. Case study method also aims to investigate a phenomenon within its real-life context and within multiple sources of evidence. Case study can benefit from previous theoretical propositions: they can guide data collection and analysis. [33] In this case study previous theoretical contributions were used as sensitizing device guiding and inspiring empirical analysis.

The data analysis was executed as follows. The email archive and bug tracking system data were examined in two phases. First phase was to make a swift look on the collected material and make a selection of seemingly interesting e-mails or bug reports. In a second phase, this selected material was inspected more carefully and Kirsch's definitions were used to identify different kinds of control methods and their implementations. For IRC meeting minutes and wiki pages there was no need for the first selection phase as their overall content was smaller so they were inspected with Kirsch's definitions as a classifier. A

OSS projects use many kinds of collaborative development tools efficiently and are actually highly reliant on them due to the distributed nature of the projects. It has been noticed that as a by-product the tools offer valuable research data [23]. All the data in this research was gathered from Internet sources that are public to all.

4 239

Full Papers

Proceedings: NordiCHI 2010, October 16–20, 2010

number of quotes were collected during this phase as examples of different control modes and related issues.

reporting system Mantis and in the mailing list to refer to the wiki documentation and to ask to read the instructions from there:

The data that was used as a source for the empirical results was collected from several sources. From the project's mailing list archive, totally 103 e-mail were analyzed. The archive consisted of emails written during the years 20062009. From Mantis, the project's bug tracking system, 51 bug reports were analyzed. In Mantis archives the oldest reports were written in year 2005, yet the newest were from the summer 2009. Of the project's IRC meeting minutes, 10 of them were analyzed. These meetings were conducted in years 2006-2008.

You should check out the wiki related to installation to get more information. (Lead Developer, Mantis).

The project participants did also look after that the Mantis users were following the documentation: See http://project.sourceforge.net/index.php/Reporting_bugs which is a reference that contains useful information related to what should go into bug reports. (Developer, Mantis).

It was frequent to refer to the wiki. The referred pages included also other pages that were not related to Mantis or a bug:

From the project's wiki the following pages were analyzed: • Changelog Format • IRC-Meeting • Release Checklist • Submitting Code • Development Processguide • Wikipolicy • Project's Vision • Reporting Bugs The wiki pages were written in years 2005-2009.

Next time you should at least read the style guidelines :) (Developer, Mantis).

The guideline refers to project's Coding Style page. It is noticeable that the advice to read the wiki for more information was given between developers and between developers and users. However, the strictest control was in use in the project IRC meetings:

EMPIRICAL RESULTS

I propose that we have a regular (e.g. every week or so) project meeting for contributors and other interested parties, on #channel @ Freenode.... (Developer, project's mailing list).

This section examines how the OSS project participants as users of their collaborative development tools are controlled. The empirical results are grouped based on the control mode that they are identified with.

The project had formed a very precise guide to communicating in IRC meetings. The guide defined that all should prepare a one-line status report, which they should present in the beginning of an IRC meeting:

Behaviour control

Behaviour control is a formal control mode that is based on rules and procedures and rewards based on following them. Mechanisms for this mode in Kirsch’s study were project plans, meetings, progress reports, walk-throughs, ISD technical documentations, organizational roles, conference calls, management travel, and systems demonstrations. This control mode was the most important one in Kirsch’s cases: all projects had project plans and different types of reports and meetings. However, only one project had ISD technical documentation. [16]

I have updated on the wiki the IRC Meeting page and added that several items should also appear in one line. Our goal is to keep things as succinct as possible, so think catefully your statements in advance and condense them into one line. (Developer, project's mailing list).

This line should contain information about what the developer has done (DONE), has there been any problems (BLOCKING), what the developer will do next (TODO) and whether he had anything he wanted to discuss about (DISCUSS). The line could look like this:

Based on the empirical analysis, we claim that behaviour control was the strongest control mode in the case project as well. The project had even more this type of controls than Kirsch’s cases. The most important finding was the wiki documentation. There was a considerable amount of documentation stored in the wiki (see the previous section). There were checklists for different work phases, like Release Checklist for publishing a new version of the software. There was also a guide to using the Git revision controlling system and policies for using and editing wiki and for committing a bug report.

DONE: pack libjava together with classfiles into the jar, much x4x development, TODO: have a look at the new full methods, probably useful, BLOCKING: none, DISCUSS: javabindings in next version. (Project's IRC meeting log)

However, as an IRC meeting went on, the discussion changed to be more informal. The project did also quit having these meetings: the last meeting minute was from year 2008. As mentioned, behaviour control is connected with observability: project members’ actions should be visible. This indeed happens; the tools the case project uses make possible for anyone to observe how the project is working.

Actually, the wiki can be seen as the most used tool in controlling the project. It was a common event in the bug

5 240

Full Papers

Proceedings: NordiCHI 2010, October 16–20, 2010

Another observation related to behaviour control is that in Kirsch's definition there were rewards as a result of following the rules. However, there is no evidence that the case project used anything particular to reward the developers. Still, this does not ensure that there weren't any.

Kirsch cases there were only some findings of clan control: two cases had IT culture related findings but the two other cases had no clan control mechanisms identified. [16] This mode appears as a strong control mode in the case project. The strongest mechanism for this mode seemed to be peer pressure. Findings in the case project show that it had a particular culture of its own and also open source culture can be seen as an effective factor. Physical and IRC meetings were used as mechanisms of clan control. The project members had met face-to-face once a year in the years 2008 and 2009 at open source developers' meeting:

Outcome control

Formal outcome control is executed through measuring outcomes and rewards based on achieving the goals. The mechanisms for this mode in Kirsch’s study were project plans, requirement documents, standards, organizational roles, presentations to user community, system testing and 'walking around'. [16]

Still another idea that came up in the FOSDEM [Free and Open Source Developers' European Meeting] is that we ought to document the development process somewhat better and have a good discussion about it to ensure that everybody is one the same level. (Lead developer, project's mailing list).

This type was the least used mode of control in the case project. However, comparison between a traditional ISD project and an OSS project is difficult, since the case project didn't have budget or time related targets or requirement documents. The case project had the Vision document that described the aims of the project at general. They also had some organizational roles defined. One of the developers conducts himself as a person who is in charge of the wiki:

As the quote shows, there were discussions about the project’s ways of working in those meetings, so the meetings were identified as part of the control mechanisms of the project. For this mode it is characteristic that the rules are not articulated, but they are still followed [16]. There were some findings of this kind of action:

My approach up to now has been that I have allow people to edit and add anything they want, and then I may have reformatted the content and polished it a little, if required. It might be the case that I am the one who takes care of the wiki, but I do not like to think that I am alone responsible for all that is there. (Developer, project's mailing list)

Please don’t change reporter to me in case I haven’t got any idea how on earth it concerns me :) (Developer, Mantis).

The bug report that was directed to this developer was not in his area of responsibility. However there were no findings indicating that these areas were defined publicly.

Surprisingly he incites others to use the wiki without following the wiki policy.

There were also some indicators that suggested that the participants were not supposed to question the project’s way of working:

The project's Vision document, as mentioned, defines that the target goal for the case project is to be an innovative and experimental project. The document can also be seen as a high level agreement of the common goals for the project. Furthermore, it should be noticed that the extensive documentation found in this project could be seen also as an outcome control type. The documents aim for the outcomes to be something that has been defined beforehand. Thus, some of the wiki documents can also be seen as outcome control: for example filing a bug report was defined strictly in the Bug Reporting page and there was also very precise checklist for publishing a new version of the software in Release Checklist page.

Keep in mind that there aren’t guarantees in this business. It is very simple. In case project sucks, you just go somewhere else. :p (Developer, project's mailing list).

There also seemed to be implicit rules maintaining that using personal email is not desirable although it was not forbidden in any of the documents. A developer, replying to another person in the project’s mailing list, claims with an exclamation mark that the other person had exhibited “bad behavior” and commands the recipient to go “sit in the corner”. This case was about replying to a personal email address and not to a mailing list address, but as mentioned, there was no written policy guiding towards the proper behaviour in the project’s mailing list.

Clan control

Clan control is an informal control mode based on unregistered information. Control is based on common values, beliefs and problem solving philosophies. It is also influential how the new members are selected and taught to follow the clan regulations. One mechanism for clan control is also the clan culture guiding the clan’s actions. Other mechanisms identified are the culture and the norms of the organization’s IT department, peer pressure, photo exchange, team member travel and conference calls. In

Self-control

Self-control is visible only in individuals' efforts and is an informal control mode. An individual monitors, rewards and sanctions himself. Rewards are based partly on individual's self-control skills. Kirsch has defined this mode to be implemented through organizational roles, culture of the IS department, and project goals and procedures. In the

6 241

Full Papers

Proceedings: NordiCHI 2010, October 16–20, 2010

findings by Kirsch only two projects had some mechanisms used and the other two had none whatsoever. This was the least used control mode in the Kirsch's study. [16]

members to do certain tasks, meritocracy being widespread in OSS projects [24]. It was mentioned earlier in connection with the behaviour control mode that typically there are rewards as a result of following the rules, but in the case project no evidence related to this was found. On the other hand, there is a lot of research on OSS developers’ motivations, which may explain why they don't need any rewards for following the rules of the project or that they gain rewards but they differ radically from those in the traditional IS development. Based on this study, it appears that peer pressure and selfcontrol seem to be important factors contributing to the compliance with the rules.

On the contrary to Kirsch's findings, there were many findings of self-control in the case project. The findings can be connected with organizational roles and project culture, but self-control could also be seen to operate via the methods and collaborative development tools of the project encouraging the developer to control himself. The working methods and the tools used could be seen as a part of the project culture. Evidence of self-control can be found for example in a comment in the bug tracker in which a developer is asking for the lead developer to close an incorrect bug report filed by the developer himself. The problem in the report was that the problem did not actually exist; the developer just misread the log. The developers is requesting for closing the bug report and argues that the reporter (i.e. himself) is either “a brainless idiot” or “illiterate”.

Results compared to previous OSS studies

Some results of this study seem to be in line with the previous studies, but also some exceptions were found. Gallivan concludes in his study that OSS projects use a variety of control modes and suggests that the earlier assumption that the work in OSS projects is mainly based on trust is not durable. He argues that control in OSS projects is implicit and invisible and therefore it is hard to notice. He also states that OSS projects cannot use only trust to organize the work, on the contrary, the project members observe each others' actions. To sum up, he concludes that there is a surprisingly great amount of different control mechanisms used in OSS projects and many of them are social and self-control sort of control. [14] The results of this study are in line with Gallivan's view: lots of control was found in the case project. Some were characterized as social and self-control modes, but besides there was also strict behaviour control identified in the way that the case project used formal documentation.

Another, somewhat similar example of self-control can be found from another comment in the bug tracker, in which the developer requests to delete his “stupid patch” and asks someone else to do it. In this case the developer has made a software patch for a bug that was reported but he is not satisfied with it and asks publicly to remove his patch. DISCUSSION

All the control modes identified in the earlier IS literature were evidenced in this study. Some of the control modes even seemed to be more evident in this OSS project than in the traditional IS projects. This is somewhat surprising, since OSS development is praised as a voluntariness and openness-emphasizing context.

Latteman & Stieglitz [18] propose that social controls are especially important in the OSS context, even more important than in commercial projects. This view also equals with the results of this study. However, they promote the importance of trust among social control methods, which in the light of this study is questionable. In the case project there seemed to be little left for trust: most of the important tasks were described in the wiki with details. In addition, the developers monitored each other’s actions carefully. Also Lundberg [19] has said that OSS projects have replaced direct orders with trust. This statement cannot be supported with the results of this study.

On the other hand, there were some difficulties in applying the Kirsch’s [16] framework in this new context and in comparing the results. Next few observations related to this are outlined, after which the results of this study are discussed in connection to the existing OSS research as well as in connection to critical IS tradition. One problem in applying the Kirsch’s framework is that in it ‘organizational roles’ was associated with three different control modes: behaviour, outcome and self-control. In this study organizational role is discussed in connection to outcome and self-control. However, there was only one finding related to an organizational role in the case project, the developer who mainly was responsible for wiki. There surely are more informal roles in the project but only one explicit role was identified from the gathered data. Also, some code related tasks were concentrated only on lead developers to do (i.e. merge), but actually this could be seen more as an evidence of the clan control mode; the culture in this OSS project as well as in OSS projects in general seems to allow only the highly experienced and recognized

On the other hand, the results of this study seem to support Latteman's and Stieglitz’s view that social control exists in different kinds of social rituals, ceremonies, morals, and peer reviews. They linked the motivations of OSS developers with the use of these kinds of controls, as the normal monetary rewards are not used in OSS projects. [18] They also make an interesting claim: ”Even if direct and indirect monitoring may be possible, direct and indirect control is hardly feasible” [18: pp. 7]. In the case project,

7 242

Full Papers

Proceedings: NordiCHI 2010, October 16–20, 2010

there were many signs of indirect and direct monitoring and the project also used control successfully as they controlled whether the project members followed the instructions stored in the wiki. So, based on this study, one could claim that both direct monitoring and control can and do exist also in OSS projects.

It was interesting to notice that the metaphor also fitted to this case. One of the researchers that linked this metaphor to technology-enabled supervision and control was Zuboff. She argued that one of the threats of technology is the expansion of control and surveillance which technology can assist. Systems record every action people execute without any supervision. [35]

Lundell et al. emphasize the importance of a shared vision as a part of project control. They suggest that this is an easy way to maintain a common understanding of the goals of the project and in their opinion a shared vision also protects against project forking. [20] The case project had a shared vision: the project's vision document in the wiki. However, there was no evidence that this document supported control methods in any way. On the contrary, the vision document seemed to be conflicting with the way the project actually worked. In the project's vision document the lead developer emphasised fun, innovation and experimentation. However, the culture in the case project seemed to be based more on control and discipline than having just fun and experimenting.

In Deleuze's post-Foucauldian society, control society, people are confined by collectively arranged continuous monitoring and control. As he emphasizes it is not only the machines that monitor and control, it is the whole arrangements. [7] The case project participants did work efficiently as part of the monitoring and controlling processes which were computer-mediated, but which also needed the human actions for the real control to be implemented. Furthermore, Lyon's view that the environment is control [21] is also linkable to the collaborative development tools that the OSS projects use. These tools are designed and implemented by other OSS projects [34]. Maybe due to that background and the conventions of OSS projects, they are designed to be very open for monitoring as well. As the OSS projects do not work in co-located setting due their members’ dispersed locations, control might be built into their virtual environments. Gallivan concluded that control in OSS projects could be implicit and hard to notice. In the light of this study, it could be argued that the collaborative development tools that the OSS projects use are essentially the source for control and controlling the use of them is almost equal to controlling the whole project. Maybe the implicit decisions made in the design and implementation of these OSS tools is the key to understanding of the OSS projects ways of working and controlling, altogether.

Regarding the possibility of forking, in OSS development it is a typical way to resolve a conflict related to organizational or technical disagreements: if one founds a new project he can control it as well as continue from the existing foundations. However, it is not given that other developers will follow, and thus, established projects usually stay alive and successful branches are rare. This situation may be changing as companies are entering into OSS development. They have resources to ensure the progress of the development, but the project leaders may not be recognized and trusted, as they are just paid workers. Persson et al. conclude based on their OSS project case study that in OSS projects control is weaker than it is in a typical commercial systems development enterprise. They specify this weakness as lack of time related control and time related goals in comparison to a commercial ISD project. [30] In the case project, there was no evidence of time related goals or controls either. Another similar observation is related to the openness of OSS projects working. Also Persson et al. notice that OSS projects are more open to feedback and monitoring because the OSS projects way of working is so transparent. All developers can access and review each other’s code. [30] This was true also in the collaborative development tools of the case project: they were accessible even for outsiders.

On the whole, it seems that monitoring and control are essential parts of OSS project's working. It is surprising as the OSS projects are still in many cases based on voluntary work. Interestingly, there was no visible critique among the participants against the controlling and monitoring. This differs from the traditional view that strict control leads to criticism and the more resources are used to maintaining control the more critique against it appears. As Doolin has put it in IS context, the critique can lead users to start questioning the whole purpose of the system [8]. In the case project analyzed in this paper, instead, one might claim that the users of these OSS tools were controlled and they were willing to allow that – they voluntarily subjected themselves to that. However, maybe the OSS developers as users differ from the user population of many other IT tools. This is a question to be answered by future research.

Results connected to the critical studies

The way OSS projects operate is very open and exposed to anyone who is interested. The case project allowed a free access to documents, logs and conversations. This openness for anyone to observe can be linked to a view that technology can enable continuous observation that can be seen also as disciplinary power, much quoted Foucault's metaphor Panopticon as a sign of disciplinary community.

CONCLUSIONS

This paper inquired control in the OSS development context, focusing specifically on how the use of collaboration tools is controlled in OSS projects. The tools

8 243

Full Papers

Proceedings: NordiCHI 2010, October 16–20, 2010

are vital for the functioning of distributed OSS projects. One case was analysed in the paper. The findings show a surprising amount of control in the OSS project, even more than in traditional ISD reported by Kirsch. All identified control modes were found in this case, but emphasis was different: traditional ISD had mostly behaviour control whereas clan and self-control were strong in this case. Surprisingly much formal control was also found. Different control mechanisms were in use than in traditional ISD projects. In OSS projects, control focuses more on outcomes and behaviours but not on time related objectives. Maybe because of that, Persson et al. did not found much control in OSS projects, as opposite to this case. However, in more commercial based OSS project this finding could be very different. Regarding the existing research, this study contradicts the view that trust is an essential factor in OSS projects. Instead, control was very dominant; it seems that there wasn’t much space left for trust.

mentioned here. It has been proclaimed that innovation and learning in IS context is based on the user's possibility to "choose to do otherwise”. [28] It would be an interesting area of research to try to investigate how OSS projects could release their full potential in innovative and educational way of working. In addition, it would be interesting to study how OSS projects led by companies can control their communities: is it money that ensures resources, or should they invest on recognized individuals who lead the project following the OSS philosophy. REFERENCES

1. Barcellini, F., Detienne, F. & Burkhardt, J. (2009): Participation in online interaction spaces: Design-use mediation in an Open Source Software community. International Journal of Industrial Ergonomics 39. Pp. 533-540. 2. Bonaccorsi, A. & Rossi, C. (2006): Comparing Motivations of Individual Programmers and Firms to Take Part in the Open Source Movement: From Community to Business. Knowledge, Technology, & Policy 18(4). Pp. 40-64.

The publicly open environment OSS projects work in is an ample environment for controlling and monitoring and realizes Lyon's view of environments that are the control. As the critical opinions argued, controlling can be also seen as undesirable and harmful phenomenon. In this light it is surprising that the developers voluntarily put themselves under monitoring through the open collaboration environment and they did this without any noticeable resistance.

3. Booch G. & Brown A.W. (2003): Collaborative Development Environments. Advances in Computers, Vol. 59. 4. Carmel E. & Agarwal, R. (2001): Tactical Approaches for Alleviating Distance in Global Software Development. IEEE Software, 18(2). Pp. 22-29, March/April, 2001.

Regarding implications for design, this study highlights that design of tools always entails also the design for monitoring and control - in the workplace as well as nowadays in the everyday life and during leisure activities. The designers should always keep this in mind and consider its consequences in their specific design context. However, it is not only the designers responsible for this, but it also always depends on the users and on their tool use. Users may choose to use them in different, surprising or paradoxical ways (cf. [28]), or even choose not to use them at all. In our case, the users voluntarily subjected themselves to the continuous monitoring and control, but this surely is not the case in every design context.

5. Crowston K & Howison J. (2005): The social structure of free and open source software development. First Monday, 10(2). 6. Cox, A. 1998. Cathedrals, Bazaars and the Town Council. Available from: http://slashdot.org/features/98/10/13/1423253.shtml. Accessed 22 March 2004. 7. Deleuze, G. (1995): Negotiations 1972-1990. New York: Columbia University Press. 8. Doolin, B. (2005): Information Systems, Power, and Organizational Relations: A Case Study. Information Systems Journal, 14(4). Pp. 343-362.

Regarding the limitations of this study, only IRC meeting minutes were examined, but not actual conversations and other synchronous communication. Maybe critique towards strict control could be found there. Furthermore, OSS projects vary in every characteristic. As a single case study, this does not aim to be a comprehensive view of control in OSS projects. Therefore, different kinds of OSS projects should be examined and compared to this study. Finally, interviews made amongst the case project participants would have complemented this study tremendously. Interviews were planned and the intention was to interview the lead developers but this didn't come true.

9. Feller, J. & Fitzgerald, B. (2000): A framework analysis of the open source software development paradigm. In Proceedings of the 21st International Conference on information Systems (Brisbane, Queensland, Australia). AIS, Atlanta, GA. Pp. 58-69. 10.Fitzgerald, B & Ågerfalk, P. (2005): The Mysteries of Open Source Software: Black and White and Red All Over?, Hawaii International Conference on System Sciences, p. 196a, Proceedings of the 38th Annual Hawaii International Conference on System Sciences (HICSS'05) - Track 7, 2005.

Interesting paths for future work have been already identified in this paper. In addition, few new topics are 9 244

Full Papers

Proceedings: NordiCHI 2010, October 16–20, 2010

11.Fitzgerald, B. (2006): The Transformation of Open Source Software. MIS Quarterly, 30(3). Pp. 587-598.

24.Michlmayr, M. & Hill, B. M. (2003): Quality and the Reliance on Individuals in Free Software Projects. In Proceedings of the 3rd Workshop on pen Source Software Engineering. Pp. 105-109.

12.Foucault, M. (1977): Discipline and Punish: The Birth of the Prison. New York: Peregrine books.

25.Nichols, D. & Twidale, M. (2006): Usability Processes in Open Source Projects. Software Process Improvement and Practice 11. Pp. 149–162.

13.Gacek, C., Lawrie, T. & Arief, B. (2001): The many meanings of Open Source. Technical report. Centre for Software Reliability, Department of Computing Science, University of Newcastle, Newcastle upon Tyne, UK.

26.Orlikowski, W. J. (1991): The Duality of Technology: Rethinking the Concept of Technology in Organizations. Organization Science, 3(3).

14.Gallivan, M. (2001): Striking a balance between trust and control in a virtual organization: a content analysis of open source software case studies. Information Systems Journal. 11(4). Pp. 277-304.

27.Orlikowski, W. J. (1992): Learning from Notes: organizational issues in groupware implementation. In Proceedings of the 1992 ACM Conference on Computer-Supported Cooperative Work (Toronto, Ontario, Canada, November 01 - 04, 1992). CSCW '92. ACM, New York, NY. Pp. 362-369.

15.Iivari, N., Karasti, H., Molin-Juustila, T., Salmela, S., Syrjänen, A.L. & Halkola, E. (2009): Mediation between design and use – revisiting five empirical studies. Human IT – Journal for Information Technology Studies as a Human Science, 10(2). Pp. 81126.

28.Orlikowski, W. J. (2000): Using Technology and Constituting Structures: A Practice Lens for Studying Technology in Organizations. Organization Science 11 (4). (Jul. 2000). Pp. 404-428.

16.Kirsch, L. J. (1997): Portfolios of Control Modes and IS Project Management. Information Systems Research. 1997. 8. Pp. 215-239.

29.Ouchi, W. G. (1979): A Conceptual Framework for the Design of Organizational Control Mechanisms. Management Science 25(9). Pp. 833-848.

17.Kirsch, L. J., Sambamurthy, V., Ko, D-G. & Purvis, R. L. (2002): Controlling Information Systems Development Projects: The View from the Client. Management Science. 2002. 48. Pp. 484-498.

30.Persson, A., Lings, B., Lundell, B. Mattsson, A & Ärlig, U. (2005): Communication, Coordination and Control in Distributed Development: an OSS Case Study. In Proceedings of the 1st International Conference on Open Source Systems (Scotto, M. and Succi, G, eds). Pp. 8892. Genoa, Italy.

18.Latteman, C. & Stieglitz, S. (2005): Framework for Governance in Open Source Communities. In Proceedings of the 35th Hawaii International Conference on System Sciences. 2005.

31.Raymond E. S. (1999): The Cathedral & the Bazaar. California: O'Reilly.

19.Ljungberg, J. (2000): Open source movements as a model for organizing. European Journal of Information Systems. 9(4). (Dec. 2000). Pp. 208-216.

32.Ye, Y. & Kishida, K. (2003): Toward an understanding of the motivation Open Source Software developers. In Proceedings of the 25th international Conference on Software Engineering (Portland, Oregon, May 03 - 10, 2003). International Conference on Software Engineering. IEEE Computer Society, Washington, DC. Pp. 419-429.

20.Lundell, B, Lings, B, Ågerfalk, P & Fitzgeard, B. (2008): The Distributed Open Source Software Development Model: Observations on Communication, Coordination and Control. In Proceedings of ECIS'2006, Gothenburg, Sweden. 21.Lyon, D. (2001): Surveillance Society. Monitoring everyday life. Buckingham: Open University Press.

33.Yin, R. K. (2003): Case Study Research Design and Methods. 3rd edition. California: Sage Publications.

22.Michlmayr M., Hunt F. & Probert D. (2005): Quality Practices and Problems in Free Software Projects, International Conference on Open Source Systems, Genova.

34.Zhao, L. & Deek, F. (2005): Improving Open Source Software Usability. In N. Romano (ed.), Proc. of 11th Americas Conference on Information Systems – A Conference on a Human Scale, 11.-14. August, 2005, Omaha, USA, pp. 923-928.

23.Michlmayr M. & Senyard A. (2006): A Statistical Analysis of Defects in Debian and Strategies for Improving Quality in Free Software Project. Bitzer, J. & Schröder, P. (ed): The Economics of Open Source Software Development, Published by Elsevier B. V.

35.Zuboff, S. (1988): In the Age of the Smart Machine: The Future of Work and Power. New York: Basic Books.

10 245

Full Papers

Proceedings: NordiCHI 2010, October 16–20, 2010

Piles, Tabs and Overlaps in Navigation among Documents Mikkel Rønne Jakobsen and Kasper Hornbæk Department of Computer Science, University of Copenhagen Njalsgade 128, Building 24, 5th floor, DK-2300 Copenhagen, Denmark {mikkelrj, kash}@diku.dk ABSTRACT

Much research has aimed at supporting general window switching and switching between tasks or activities. General window switching techniques [26,29] help switching between applications, which are often visually distinct. However, switching between documents often takes place within one application and documents switched between may be visually similar. Also, while users’ frequent switching between tasks or activities needs support [7,18,27,28,30], often users must switch between multiple documents related to one task.

Navigation among documents is a frequent, but ill supported activity. Overlapping or tabbed documents are widespread, but they offer limited visibility of their content. We explore variations on navigation support: arranging documents with tabs, as overlapping windows, and in piles. In an experiment we compared 11 participants’ navigation with these variations and found strong task effects. Overall, overlapping windows were preferred and their structured layout worked well with some tasks. Surprisingly, tabbed documents were efficient in tasks requiring simply finding a document. Piled documents worked well for tasks that involved visual features of the documents, but the utility of recency or stable ordering of documents was task dependent. Based on the results, we discuss the effects of spatial arrangement, visibility, and task-dependency, and suggest areas for future research on document navigation and its support by piling.

Problems with existing switching mechanisms

One facility for supporting navigation among documents is overlapping windows, standard in most operating systems. Hutchings et al. [19] found that 78% of the time people had eight or more windows open, making locating a particular window within a group of overlapping windows difficult. Switching between windows using the taskbar is also hard because users may only see a short part of the windows’ titles and because the taskbar occasionally collapses documents into one tile [13].

Author Keywords

Document navigation, window switching, overlapping windows, piled windows.

Another facility, using tabs to represent multiple documents within a single window, is seen in many web browsers, spreadsheets, and programming environments. But tabbed interfaces can only show a limited number of documents at any time, necessitating further interaction with scrollbars or drop-down lists. When several documents are open, tabs provide minimal visual cues for finding a particular

ACM Classification Keywords

H.5.2 User Interfaces (D.2.2, H.1.2, I.3.6): Graphical user interfaces (GUI). INTRODUCTION

Users frequently navigate between multiple documents, for instance to copy-paste text, compare web pages, or capture notes when reading. But navigation between documents is hard. Multiple windows or tabs are increasingly used in web browsing, for example, but users find it hard to manage multiple web pages [31]. In programming, source files are often viewed one at a time, requiring frequent navigation between files. Tabbed document interfaces that are common in widespread programming environments, such as Eclipse and Visual Studio, make navigating among many source files cumbersome [21]. Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. NordiCHI 2010, October 16–20, 2010, Reykjavik, Iceland. Copyright 2010 ACM ISBN: 978-1-60558-934-3...$5.00.

Figure 1: Interface with two piles of spatially arranged Figure 1: Interface with two piles of spatially arranged documents. documents.

246

Full Papers

Proceedings: NordiCHI 2010, October 16–20, 2010

document. Although recent studies show increased use of tabs in web browsing [15,31], attempts at improving or reinventing tabbed interfaces are needed [3]. BeaudouinLafon [6] suggested techniques for interacting with tabbed documents, such as leafing through tabs, but these techniques have to our knowledge not been widely adopted.

!"#"$"%"&'( !"#$%#&&'( #()&")%*+,+-.'( #))#*+,-(./)( !0#**%*+( ./01!2",)%"3,)'(

Still other facilities support navigation among multiple documents by a notion of piles. People pile and spatially arrange paper documents to see many of them at once, to structure their tasks, or to remind [22,25]. Although techniques for working with piles of electronic documents have been explored [4,5,6,23], questions about the design of interfaces that use piling remain unanswered. For example, how can a pile be spatially arranged so as to support navigation between its documents? Furthermore, empirical studies that investigate the usefulness of piling electronic documents are lacking.

9/014,*$!(

(

#))#*+,-(

4#*1#&&'( )/&-+)&"0)%%'(

$/(%*0),#!,(1!,)(0/*$)/&( $/(),-10,(1!,)(,../)$!(

5%,6,-(

(,.+)1,1&%'( $)#*!%,*$&'(

$/(")/5%-,(5%!1#&(01,!( $/(),-10,(1!,(/.(!"#0,(

.1&&7!%8,( ./)(0/*$,*$(%*$,)#0$%/*( ),"),!,*$,-( 6%$3(0/*$,*$( ./)(/5,)5%,6( #0)%,220)/"",-( ( #))#*+,-(%*(

3"4,2* /)-,)( )2)(&"!,*

Table 1: Taxonomy of document interface designs that use piling. The terms describing the interfaces used in the experiment are highlighted in bold.

This paper explores the design space of interfaces that support document piling and investigates possible benefits of spatially arranged piles (see Figure 1). In an experiment, we compare variations of interfaces for supporting navigation between documents, including two that arrange documents in a pile. The results contribute insights about the effect of visibility and spatial arrangement in different tasks involving navigation between documents. These findings may inform design of techniques for navigating between documents and indicate areas where further empirical research is needed.

hiding the title bar of windows underneath, as illustrated in Figure 2 (a). Spatial memory can be utilized to help people organize and find documents. Figure 2 (b) illustrates a spatial arrangement of documents in a pile that makes the spatial location of individual documents more distinct. Data Mountain, which allows users to arrange web page thumbnails on a perspective plane, showed faster and more accurate retrieval of pages compared with a text-based bookmark list [14].

SUPPORT FOR DOCUMENT NAVIGATION

Studies of office work have found that people benefit from piling as a way of spatially grouping documents [22,23,25]. Arranging and piling electronic documents may also help users navigate among documents. However, methods for spatially arranging documents and techniques for interacting effortlessly with them are needed.

Scanning through documents in a serial manner may be easier if the documents are aligned. Compared with the arrangement in Figure 2 (b), the alignment in Figure 2 (a) and (c) may provide for easier scanning of the documents. Agarawala et al. explore pile interaction techniques that use various aligned layouts for leafing through piles in a serial manner [4].

The use of electronic piles has been researched mainly as an alternative to filing documents by storing them as named files in hierarchical folders, and work on iconic document representations [4,5,23]. Instead, we focus on using piles for supporting navigation among documents that are in use. Based on a review of the literature, we have developed a taxonomy that incorporates important dimensions in the design of document piling interfaces (see Table 1). We relate each dimension to previous work and we explain the approach taken in the design of the piling interfaces used in the experiment presented later in the paper. Spatial arrangement

(a)

Documents can be spatially arranged to help users navigate among them. Four concerns of spatial arrangement in design of piling interfaces are visibility, spatial memory, scanning, and focus/periphery.

(b)

(c)

Figure 2: Spatial arrangements of four documents. (a) Pile of documents with titles and part of content visible, aligned diagonally. (b) Pile of documents with titles and part of content visible, arranged irregularly. (c) Document window with titles partly shown in tabs; tabs aligned horizontally.

Visibility of documents reminds users of the documents and allows them to interact with the documents. In overlapping window interfaces, windows open in a cascade to keep from

247

Full Papers

Proceedings: NordiCHI 2010, October 16–20, 2010

Focal/peripheral areas seem to play a role in the spatial layout of documents [25] and how users manage space in large or multiple displays [8,16]. Robertson et al. divide the display into focal and peripheral regions in a window interface that supports task switching. Their Scalable Fabric uses animated transitions to shrink windows to thumbnails as they are moved into the peripheral region [27].

Transient views that can be called up temporarily can support navigation without permanently using display space. Widespread examples of transient views are the Exposé feature on Mac OS X [1] and the - window switching technique. Several techniques have been explored for browsing and leafing through piles that use transient views of the piled documents [4,5,23]. However, we are unaware of studies that compare how users navigate among piled documents with permanent and transient views.

In this paper, we arrange a pile of documents vertically so as to make the upper part of each document in the pile visible, including its title. To further improve visibility in piles with many documents, we reduce overlap by staggering the documents, which may also help users find documents by spatial location.

In this paper, we arrange documents in a pile to provide permanent visual cues for navigation so as to compare piles to permanent views techniques for navigation between documents that are common in widespread interfaces.

Manual vs. automatic layout

Document interfaces must provide the user with control of the layout to arrange the documents in ways that are appropriate for different tasks, with as little effort as possible.

Document representation

Full-size views of documents allow the user to interact directly with the documents’ content. In contrast, small document representations using thumbnail views, icons, or text make many documents visible in limited display space.

Manual layout in overlapping window interfaces allows users to pile windows manually—users have full control over moving, resizing, and overlapping windows. However, arranging windows and switching between overlapping windows require time-consuming window management [10]. Compared with overlapping windows, automatically tiled windows may increase performance, but users may prefer overlapping windows [9,20].

Windows can be reduced for instance by scaling the windows [11,27,28], or by scaling or cropping the content of windows [24]. Using a 3D document space, The Web Forager showed a document at a focus position in full size for direct interaction between user and content, whereas documents that are not the immediate focus are placed at greater z-distances (and thus smaller) [11]. Aiming to reduce screen clutter, Miah and Alty studied an adaptive window management system that reduced the size of windows [24]. They found that identification of a window became difficult when the window’s size was below 30% screen utilization. Their results also indicate marginally higher accuracy in identification tasks when a window’s content is cropped rather than scaled. Users may identify a document by a thumbnail view from among visually distinct documents (e.g., pages from different web sites). However, if documents are visually similar (e.g., source code files or pages from digital library), a thumbnail view contains no salient features for identifying the document.

Automatic layout of documents in piles may help users arrange documents and navigate among them with fewer efforts. Task Gallery provides an “ordered stack”, in which windows are automatically laid out with fixed distance between them, as well as a “loose stack”, which users lay out manually [28]. Clicking on a window in one of these stacks moves the window forward to a selected window position. The study focused on the system’s task management support and it is not clear how the stacks were used. Lightweight interactions have been explored for automatically gathering a selection of items into a pile on the desktop, for manipulating piles, and for arranging piles in different layouts with few user efforts [4]. However, it is not clear how these techniques would work for piles of document windows.

Semantic zooming or use of colors can make visually similar document thumbnails distinguishable. Other features of the document representation can help navigating among documents in piles. For instance, to help finding one among many equally-sized windows piled on top of each other, Beaudouin-Lafon suggests rotating windows [6].

In this paper, we explore piling interfaces that automatically maintain the layout so that all documents in a pile, and in particular the documents’ titles, are visible. When the user opens or closes a document, or drags a document from one pile to another, the piled documents rearrange to maintain a consistent layout of documents. We thus aim to reduce the user efforts required for rearranging and navigating among documents in piles compared with overlapping windows.

In this paper, we change the size of documents in a pile so that documents at the bottom of the pile are smaller. To reduce a document’s size, we scale the document’s content, but show its title in a fixed font size so that the title remains readable. The readable title is aimed at helping users in searching among visually similar documents.

Permanent views vs. transient views

Permanent views that support navigation among documents provide visual cues of the documents. For instance, in many desktop environments the task bar often shows windows as icons or text, including windows that are hidden.

Adaptive vs. fixed ordering

The order that documents are shown in the display may influence how users navigate between the documents.

248

Full Papers

Proceedings: NordiCHI 2010, October 16–20, 2010

In most desktop environments, windows overlap each other in the order they were last recently used—and windows are shown in this order when switching using -. Recent research adapts the order of windows used in window switching techniques based on semantic or temporal information about the windows [26,29]. These examples use adaptive ordering: the order of windows changes as the user switches between the windows. Alternatively, documents can have a fixed order. In this case, the user may order documents manually or documents may be ordered automatically by the system (e.g., by creation date). In this study we investigate different ways of ordering documents in a pile to see how that influences the way users navigate among the documents. We use a fixed ordering where documents are piled in the order they were opened and an adaptive ordering where documents are piled in the order they were most recently used. Summary

We have described the design space of interfaces for document piling based on five dimensions in the taxonomy shown in Table 1. Using this taxonomy, we have explained the approach taken in the interfaces used in the experiment. Earlier studies of document piling have mainly focused on piling document files in desktop environments, use of iconic representations, and use of transient view techniques to provide visibility of a pile’s documents. In contrast, we focus on piling documents in use, documents represented as zooming windows, and spatial arrangement to provide visibility of a pile’s documents. We are unaware of earlier studies of automatically arranging piles of documents that aim at providing permanent visual and spatial cues for navigating among the documents.

Figure 3: The Overlapping interface. The bar at the bottom of the display contains a tile for each document.

brings the document on top of other documents. Tiles appear from left to right in the bar in the order the documents were opened. All tiles have the same size and all fit in the bar, but only about 10 characters of a document’s title is shown. A tooltip with the entire title appears if the mouse cursor hovers over a tile. Rationale: This interface is included in the experiment because overlapping windows are standard in most systems and graphical user interfaces of many systems include a task bar. Studies have found that the task bar is often used for switching between windows [19,29]. Hutchings et al. found that 78% of the time people had eight or more windows open and consequently, users may experience problems with using the taskbar because only a short part of the windows’ titles are visible [19].

EXPERIMENT

To investigate the influence of spatial arrangement and visibility of documents on how users navigate among documents, we conducted a controlled experiment in which four interfaces for viewing multiple web pages were compared. We use web pages because the Web is a widely used source of information, for instance in sense-making of research literature using digital libraries [32], and because users often have many web pages open simultaneously [31].

The Tabbed interface

The Tabbed interface (shown in Figure 4) contains one window fixed in the center of the display. The window contains multiple documents represented by equally sized tabs under the title bar (see below). Tabs appear from left to right in the order the documents were opened. If there are more documents than tabs, which are minimum 120 pixels wide, a chevron appears to the right of the rightmost tab. Clicking the chevron opens a drop-down list of the titles of all the documents in the order they were added. Documents not visible in the tabs are at the bottom of the list on a gray background. This tab behavior resembles that of the Safari browser, and drop-down lists are common in tabbed document interfaces.

Interfaces

The interfaces consist of windows, where each window contains only the view of a web page, framed by a border and a title bar at the top. The active window is indicated by blue. Windows have a fixed size of 800 x 700 pixels. The Overlapping interface

In the Overlapping interface (shown in Figure 3), windows can be moved by dragging the title bar. Document windows open in a cascade to avoid hiding title bars of windows underneath. A horizontal bar (see below) at the bottom of the display contains a tile for each document (similar to the Taskbar in Windows XP or Vista [2]). Clicking on a tile

249

Full Papers

Proceedings: NordiCHI 2010, October 16–20, 2010

The Piling Recent interface

The Piling Recent interface (shown in Figure 5) contains document windows piled in the order that they were last used, with the least recently used document at the bottom of the pile and the currently active document on top. As shown in Figure 6, documents are reduced in size and arranged vertically relative to their position in the pile so that the document at the bottom of the pile is smallest (at 30% size) and nearest the upper display border. Documents are reduced in size by zooming the view of the documents’ content. However, the document titles remain in a fixed, readable font size. When documents are opened, their windows are staggered so as to reduce overlap. Also, the staggering may help in remembering documents’ location. If the user clicks on a document in the pile, the document moves to the top of the pile and the other documents are rearranged, all with animated transitions. Rationale: Compared with Overlapping and Tabbed, the Piling Recent interface aims to provide better visibility of the documents, and titles of most documents are readable in full. The Piling Ordered interface Figure 4: The Tabbed interface. The document tabs below the window title bar contains the tabs for seven of the documents. Clicking the chevron near the right border opens a drop-down list showing all document titles.

The Piling Ordered interface (shown in Figure 7) is similar to Piling Recent, except document windows remain in the order they were opened. When clicked, a document is moved from its place to the top of the pile—the document returns to its place in the pile when another document is selected.

Rationale: This interface is included in the experiment because tabbed interfaces are common in widespread web browsers, spreadsheets, and programming environments. Common in many tabbed interfaces is that only a limited number of documents are visible as tabs and users either have to scroll in the tabs or otherwise get a view of the documents that are not shown.

Rationale: The spatial layout of document windows in Piling Ordered is more stable compared with Piling Recent, which can potentially help users in revisiting previously used documents.

Figure 6: The size and vertical location of a document at each position in a pile of five documents. At the top of the pile, the document is shown at full size.

Figure 5: The Piling Recent interface.

250

Full Papers

Proceedings: NordiCHI 2010, October 16–20, 2010

When document titles were not all shown in full, participants could not determine if the term occurred in a document title directly in the interface and titles were particularly truncated in the tiles in Overlapping and in tabs in Tabbed. In the Tabbed interface, however, participants could call up the list to read the titles in full. Navigate All tasks required participants to navigate to every document shown. An example task read: “Click the link to the first author of the document with the highest ‘Citation Count’” Each task used a different set of documents. We hypothesized that participants could traverse the documents quickly in the bar of Overlapping, and also in Tabbed in as far as the documents were visible in tabs. Compare References tasks asked participants to inspect the ‘References’ of three of the documents and determine the most referenced author. Each task of this type used a different set of ten documents. The three documents named in the task text had about 15 references each and at least one author referenced in all three documents.

Figure 7: The Piling Ordered interface. Participants

Eleven participants (eight male) between 25 and 51 years old (M = 34.6) were recruited by word of mouth.

We expected Piling Recent to be suitable for this task because participants could easily find the three documents close together at the top of the pile. We also were curious to see if participants arranged windows for comparison with the Overlapping interface.

Tasks

Ten web pages were opened in the interface before each task. Pages were selected from among 420 web pages from the ACM Digital Library and showed data on papers published 1994 to 2005. Documents in the experiment were thus all uniform in layout and appearance. Four types of task were used.

These four tasks were chosen to cover basic types of navigation among documents that occur in web browsing or programming. The tasks are similar to tasks used in earlier research [e.g., 20,29]. We are aware that not all user tasks that involve document navigation are among these four types of task. Also, tasks are limited in that they are taken out of context and involve already opened documents.

Navigate Title tasks required participants to do 25 tasks on finding a particular document, for instance “Click the link to the first author of the document: Pen computing for air traffic control”, where the document title varied between eight of the ten documents shown. We wanted to see how spatial layout of pages in the interfaces influenced performance in re-finding documents. Therefore, we used a Zipf-like distribution similar to the one used in Tak et al. [29] for selecting the target documents. The distribution was generated by randomly selecting eight targets. One target was then cued 10 times, one 5 times, then 3, 2, 2, 1, 1, and 1 for the others.

Materials

Participants used the 1440 x 900 pixel 15” display of a laptop computer. A mouse with scroll wheel was used for input. Tasks were presented in a view in the left side of the display. The interfaces used an area of 1040 x 878 pixels and document windows were 800 x 700 pixels. Design

The experiment employed a within-subjects design with the factors interface type (Overlapping, Tabbed, Piling Recent, Piling Ordered), and task type (Navigate Title, Navigate Term, Navigate All, Compare References). The order of interface type and task type was systematically varied and counter-balanced across participants. Participants completed 35 tasks with each interface: 25 Navigate Title tasks, four Navigate Term tasks, four Navigate All tasks, and two Compare References tasks.

We hypothesized that participants could find the cued document title quickly by scanning in the bar of Overlapping or the tabs in Tabbed. Navigate Term tasks required participants to find and compare documents with a given term in the title. An example task read: “Click the link to the first author of the document with ‘learning’ in the title that has the second most recent Year of Publication.” Each task of this type used a different set of documents, four of which contained the given term in the title.

Procedure

Participants were first given an introduction lasting about 15 minutes, which included instructions on how to use each interface and time for participants to practice tasks similar to the experimental tasks. Participants then completed tasks using each of the four interfaces. After completing the tasks

We hypothesized that participants could find the term in document titles faster and with less interaction using either Piling interface compared with Overlapping and Tabbed.

251

Full Papers

Proceedings: NordiCHI 2010, October 16–20, 2010

with each interface, participants received a questionnaire about satisfaction with the interface just used. The questionnaire contained nine questions from QUIS [12], two questions from NASA-TLX [17], and two additional questions asking how easy it was to find or re-find documents. Also, participants listed benefits and drawbacks of the interface. After completing all tasks, a final questionnaire asked participants to rank the four interfaces by order of preference. Last, participants were given opportunity to comment on the interfaces. The experiment lasted about an hour and 15 minutes for each participant. RESULTS Accuracy

We find no differences in the accuracy with which participants answer tasks, F(3, 8) = 0.25, ns. On average, participants answered 87% of the tasks correctly, ranging from 88% (with Overlapping) to 86% (with the two Piling variants). We had also expected no difference, as the tasks were relatively easy to solve with all interfaces.

Figure 8: Average task completion times (in seconds). Bars indicate the standard error of the mean.

reason, supported by our informal observations, is that participants sometimes lost track of the first document they viewed in sequentially traversing the documents. Hence, participants may have spent time looking at the same document twice.

Task completion time

We find an overall interaction between task and interface, F(9, 2) = 34.93, p < .05, and thus proceed to analyze the time differences on a per task basis (with statistics for significance omitted to save space, all differences are p < .05 with linear contrasts or post-hoc tests). Figure 8 shows the task completion times for each interface.

For the Compare-references task, we find that Piling Ordered (M = 131.9s) is significantly slower than the other three interfaces (Ms between 113.5s and 116.7s). One reason for this is that participants had trouble remembering the location of the three documents they had to compare references in. Compared to the other tasks involving revisiting documents, this task was cognitively more demanding. It might therefore have helped participants that the three documents could all be found in the order they were last used at the top of the pile in Piling Recent.

For the Navigate Title task, a post hoc comparison show no difference among interfaces, but to our surprise Tabbed was the fastest interface (M = 5.47s, SD = 2.84), followed by Overlapping (M = 5.66s, SD = 2.30), Piling Recent (M = 5.68s, SD = 2.71), and Piling Ordered (M = 6.21s, SD = 2.54). In particular, the Tabbed interface performed well even for documents that users had to access using the chevron and the drop down list.

Satisfaction and Preference!

A multivariate analysis of variance on the 13 satisfaction questions shows no overall differences in satisfaction between interfaces, Wilks’ != .274, F(3, 8) = 1.18, p > .3. Only for a question on whether tasks may be completed "with difficulty" or "with ease" do we find a significant difference, F(3, 40) = 3.20, p < .05, so that the tasks solved with the Overlapping interface are perceived to be easier than tasks with the other interfaces.

For the Navigate Term task, we find that the two Piling interfaces (Ordered: M = 53.3s, Recent: M = 53.1s) are significantly faster than the other interfaces (Overlapping: M = 60.6s; Tabbed: M = 63.8). This confirms our hypothesis that participants could find documents with the term faster using the Piling interfaces because titles of most documents are readable in full. Because the term is not always visible in truncated titles in Overlapping or Tabbed, participants may have spent more time switching between documents or calling up tool tips to look for the term.

In order of preference the interfaces are: Overlapping (M = 1.91, SD = 1.04), Piling Recent (M = 2.27, SD = 1.19), Tabbed (M = 2.82, SD = 1.08), and Piling Ordered (M = 3.00, SD = 1.00). However, an overall analysis of variance on the ranks suggest that these differences are only marginally significant, F(3, 40) = 2.361, p = .086.

For the Navigate All task, we find that Overlapping (M = 21.6s) is significantly faster than Tabbed (M = 35.1s). They are in turn significantly faster than Piling Ordered (M = 39.6s), which is significantly faster than Piling Recent (M = 47.3s). Looking at average task completion times, all participants were faster with Overlapping. This finding agrees with our expectation that participants could traverse the documents by clicking each tile in the bar. One reason participants were slower using Piling Recent compared with Piling Ordered is that the fixed document order in Piling Ordered helped participants revisit documents. Another

About Overlapping, six participants listed the taskbar as an advantage, three mentioning the arrangement of tiles in a row. Three participants liked that windows could be arranged for comparing their content. Two participants said it was hard to read or get overview of the titles. About Tabbed, participants described as advantages that it was “clear”, “simple”, and “familiar”. Also, three participants mentioned the alignment of tabs in a row. Six

252

Full Papers

Proceedings: NordiCHI 2010, October 16–20, 2010

participants expressed uncertainty about how the dropdown list in Tabbed worked, although two participants said it gave good overview of document titles. Overlapping Tabbed Piling Recent Piling Ordered

For both Piling interfaces, four participants mentioned overview of all documents as an advantage and three participants said the interfaces were good for finding a document. Five participants said about Piling Recent that it was confusing how documents were ordered, but three participants liked having recently used documents on top. Two participants said locations of documents were hard to remember in Piling Ordered, while two participants found it good for revisiting documents. Comments from three participants suggested problems in navigating through all documents using Piling Ordered.

Overlapping Tabbed Piling Recent Piling Ordered

10.6 8.3 7.2 5.7

11.4 12.3 13.9 11.9

5.7 5.0 5.2 5.9

75 109 111 105

32 53 192 121

123 84 99 94

For the Navigate All task, participants navigated in average to two more documents with Piling Recent than with Piling Ordered. This difference supports the explanation that participants sometimes lost track of the first document they viewed when using Piling Recent, resulting in slower performance. Looking at the mouse pointer travel distance, it seems Overlapping and Tabbed required less user efforts (32 and 53 in average distance) in navigating through all documents than the Piling interfaces (192 and 121 in average distance). This supports the explanation that the aligned tiles and tabs help scanning through the documents. For the Compare-references task, participants navigated to between 5 and 6 documents in average with all interfaces; which are surprisingly few switches between documents considering that participants had to compare about 15 references in each document. Piling Ordered was slower maybe because it was be difficult to remember the location of the three documents. Participants navigated only slightly more with Piling Ordered than with Piling Recent. It is unclear why the mouse pointer travel distance was larger with Overlapping compared with the other interfaces. It might be because some participants rearranged the documents to better compare them.

For the Navigate Term task, participants navigated 10.6 times in average using Overlapping and 8.3 times using Tabbed. In contrast, participants navigated 7.2 and 5.7 times in average using Piling Recent and Piling Ordered. This suggests that participants were faster with the Piling interfaces compared with Overlapping and Tabbed because they had to navigate to fewer documents to answer the task. Participants navigated to fewer documents with Tabbed than with Overlapping, which might be because some participants used the menu to find documents with the given term. It is not clear why participants navigated to more documents with Piling Recent compared with Piling Ordered. We wonder if the changing order of the documents made it difficult to revisit the correct document. The mouse pointer travelled relatively less between mouse events with Overlapping compared with the other Compare References

Compare References

interfaces, which indicates the benefit of traversing all the document tiles in the bar.

We analyzed data logged during the experiment to help uncover differences in how participants’ used the interfaces to complete the tasks. We summed the number of times that participants navigated to a document by clicking either in the document’s window, in a tile (only in Overlapping), or in a tab or pop-up menu (only in Tabbed). Table 2 shows the average number of such navigation actions for each interface. Also, we summed the distances that the mouse pointer traveled between mouse button events or tooltip call-ups (shown in Table 3). Distance was calculated as the diagonal between screen coordinates of the mouse pointer. Because the Navigate Title task involved only single-step navigation, we did not analyze the data for that task.

Navigate All

Navigate All

Table 3: Average distance that the mouse pointer traveled between mouse events for each interface (100=average distance for task across interfaces).

Interaction with the Interfaces

Navigate Term

Navigate Term

DISCUSSION

We emphasize three findings from the experiment, relate them to the literature, and elaborate on task effects. Visibility. The spatial arrangement in the Piling interfaces makes all documents visible and most titles readable in full. This might explain why participants completed Navigate Term tasks faster using the Piling interfaces. In contrast, document titles are truncated in Overlapping and Tabbed so participants had to inspect each tab or tile, either by using tooltips, by clicking the tab or tile to bring up the document, or by using the drop-down list in Tabbed. We note that the web pages used in the experiment were uniform in layout and appearance. However, it seems likely that the visibility in the Piling interfaces would improve navigation between visually distinct documents, for instance because users recognize content. Mander et al. studied piles of documents with varied content, but their piles were tightly arranged and used thumbnails [23].

Table 2: Average number of times that participants navigated to a document.

253

Full Papers

Proceedings: NordiCHI 2010, October 16–20, 2010

Layout. The aligned layout of tiles and tabs in Overlapping and Tabbed allow for scanning document titles and for clicking through each document in sequence, with little mouse movement. Participants thus completed Navigate All tasks faster using Overlapping and Tabbed compared with the Piling interfaces, which seemed to require more efforts in navigating through all documents. In Piling Recent, participants could continually click the document nearest the upper edge of the display to bring it on top. The staggered placement required moving mouse horizontally, however, and to some participants it seemed unclear when all documents had been viewed. In Piling Ordered, participants had to move the mouse to each document in the pile to bring it on top.

In an experiment with 11 participants, we have compared four interfaces that support navigation among documents: one using overlapping windows, one using a tabbed window, and two that arranged documents in a pile. Strong tasks effects were found in task completion times. In tasks that used visual features of documents, participants performed significantly better using piles, likely because visibility of documents was better compared to overlapping and tabbed windows. In contrast, participants performed better using overlapping and tabbed windows in tasks that required participants to scan through all documents. The aligned layout of document titles in the task bar and the tabbed windows’ tabs provided for scanning titles in sequence without much effort. Overall, participants preferred overlapping windows, commenting the taskbar as an advantage. In conclusion, results suggest that automatic spatial arrangement of documents in piles provides visibility of documents useful in some tasks, but suggest that techniques for aligning documents could be useful in tasks that involve serial scanning.

We were surprised that the Tabbed interface performed well, despite the efforts required in navigating to documents not visible in the tabs. Still, only ten documents were used in tasks, and we wonder how tabbed interfaces scale to more documents. Also, participants were effective at navigating many windows in sequence using the taskbar. However, in real life the taskbar often contains different application windows and documents are sometimes collapsed into a single tile.

For future work, several questions about the use of piles in interfaces for supporting navigation remain to be addressed. First, to provide benefits similar to those enjoyed from arranging and piling documents on desks, more work is required that looks beyond navigation among documents in a single pile. We have begun exploring interaction techniques that allow users to effortlessly arrange documents side by side, in multiple piles (see Figure 1). Another issue concerns how users will arrange and pile documents in focal and peripheral regions, for instance by arranging and ordering documents in piles in the periphery to structure their tasks and piling documents in the focus that are frequently used. This requires empirical studies of people working on more complex tasks than the simple navigation tasks used in the present study.

Further, while techniques for leafing through many documents in tabbed windows [6] and in piles [4] promise to support looking sequentially through documents, we have yet to see empirically studies of their use. Our data suggest that this might be particularly useful for piling interfaces. Stability. Piling Ordered was slower in Compare References tasks, which suggests that participants had difficulties remembering where the documents were. In contrast, Piling Recent seems more useful in this task because the three recent documents all were near the top of the pile. Tak et al. reach a different conclusion in a study of window switching interfaces: they found that a spatially stable layout allows faster switching to windows often switched to than a recency layout [29]. The importance of stability and recency in the spatial layout clearly depends on the task performed.

Also, participants in this study only navigated among 10 documents, with limited screen size. We wonder how piling interfaces scale compared to widespread techniques in navigating among many documents and with large displays. Last, it remains unclear how transient techniques can use spatial arrangement and representation of documents to support navigation among documents. We will address this in future work and explore how transient techniques can be used for leafing through document piles.

Moreover, in this study participants navigated only between documents that were unknown to them. The usefulness of piles in real use situations, where users open and arrange documents themselves, remains to be further researched. In particular, we are curious if spatially stable piles of documents, which were inferior in this experiment, might be useful for structuring work with multiple documents.

1. 2.

CONCLUSION AND FUTURE WORK

3.

REFERENCES

Overlapping windows and tabbed document interfaces are widely used in desktop environments, but they provide limited support for navigating among documents. Aimed at providing such support, this paper has investigated the use of automatically arranging documents in piles. Based on five dimensions, we have explored the design space of interfaces for navigation among documents using piles.

4.

254

Exposé. Apple Computer, Inc.: http://www.apple.com. Windows Taskbar. Microsoft: http://www.microsoft.com. Reinventing Tabs In the Browser, Mozilla Labs Design Challenge 2009. http://designchallenge.mozillalabs.com/summer09/. Retrieved 2010 May 9. Agarawala, A. and Balakrishnan, R. (2006), Keepin' it real: pushing the desktop metaphor with physics, piles

Full Papers

Proceedings: NordiCHI 2010, October 16–20, 2010

5.

6.

7.

8.

9.

10.

11.

12.

13.

14.

15.

16.

17.

18.

and the pen, in Proceedings of CHI '06. ACM: New York, NY, USA. p. 1283-1292. Aliakseyeu, D., Subramanian, S., Lucero, A., and Gutwin, C. (2006), Interacting with piles of artifacts on digital tables, in Proceedings of AVI '06. ACM: New York, NY, USA. p. 159-162. Beaudouin-Lafon, M. (2001), Novel interaction techniques for overlapping windows, in Proceedings of UIST '01. ACM: New York, NY, USA. p. 153-154. Bernstein, M. S., Shrager, J., and Winograd, T. (2008), Taskposé: exploring fluid boundaries in an associative window visualization, in Proceedings of UIST '08. ACM: New York, NY, USA. p. 231-234. Bi, X. and Balakrishnan, R. (2009), Comparing usage of a large high-resolution display to single or dual desktop displays for daily work, in Proceedings of CHI '09. ACM: New York, NY, USA. p. 1005-1014. Bly, S. A. and Rosenberg, J. K. (1986), A comparison of tiled and overlapping windows. SIGCHI Bull., 1986. 17(4): p. 101-106. Bury, K. F. and Darnell, M. J. (1986), Window Management in Interactive Computer Systems. SIGCHI Bull., 1986. 18(2): p. 65-66. Card, S. K., Robertson, G. G., and York, W. (1996), The WebBook and the Web Forager: an information workspace for the World-Wide Web, in Proceedings of CHI '96. ACM: New York, NY, USA. p. 111-117. Chin, J. P., Virginia, A., and Norman, K. L. (1988), Development of an instrument measuring user satisfaction of the human-computer interface, in Proceedings of CHI '88. ACM Press. p. 213-218. Czerwinski, M., Smith, G., Regan, T., Meyers, B., Robertson, G., and Starkweather, G. (2003), Toward characterizing the productivity benefits of very large displays, in Proceedings of Interact 2003. p. 9-16. Czerwinski, M., van Dantzich, M., Robertson, G., and Hoffman, H. (1999), The Contribution of Thumbnail Image, Mouse-over Text and Spatial Location Memory to Web Page Retrieval in 3D, in Proceedings of Interact '99: Edinburgh, UK. Dubroy, P. and Balakrishnan, R. (2010), A study of tabbed browsing among mozilla firefox users, in Proceedings of CHI '10. ACM: New York, NY, USA. p. 673-682. Grudin, J. (2001), Partitioning digital worlds: focal and peripheral awareness in multiple monitor use, in Proceedings of CHI '01. ACM: New York, NY, USA. p. 458-465. Hart, S. G. and Staveland, L. E. (1988), Development of NASA-TLX: Results of empirical and theoretical research, in Human Mental Workload, P.A. Hancock and M. Meshkati, Editors. Elsevier: Amsterdam, The Netherlands. p. 139-183. Henderson, J., D. Austin and Card, S. (1986), Rooms: the use of multiple virtual workspaces to reduce space contention in a window-based graphical user interface. ACM Trans. Graph., 1986. 5(3): p. 211-243.

19. Hutchings, D. R., Smith, G., Meyers, B., Czerwinski, M., and Robertson, G. (2004), Display space usage and window management operation comparisons between single monitor and multiple monitor users, in Proceedings of AVI '04. ACM Press: New York, NY, USA. p. 32-39. 20. Kandogan, E. and Shneiderman, B. (1997), Elastic Windows: evaluation of multi-window operations, in Proceedings of CHI '97. ACM: New York, NY, USA. p. 250-257. 21. Ko, A. J., Aung, H., and Myers, B. A. (2005), Eliciting design requirements for maintenance-oriented IDEs: a detailed study of corrective and perfective maintenance tasks, in Proceedings of ICSE '05. p. 126-135. 22. Malone, T. W. (1983), How do people organize their desks?: Implications for the design of office information systems. ACM Trans. Inf. Syst., 1983. 1(1): p. 99-112. 23. Mander, R., Salomon, G., and Wong, Y. Y. (1992), A ``pile'' metaphor for supporting casual organization of information, in Proceedings of CHI '92. ACM: New York, NY, USA. p. 627-634. 24. Miah, T. and Alty, J. L. (2000), Vanishing Windows--a technique for adaptive window management. Interacting with Computers, 2000. 12(4): p. 337-355. 25. O'Hara, K. and Sellen, A. (1997), A comparison of reading paper and on-line documents, in Proceedings of CHI '97. ACM: New York, NY, USA. p. 335-342. 26. Oliver, N., Czerwinski, M., Smith, G., and Roomp, K. (2008), RelAltTab: assisting users in switching windows, in Proceedings of IUI '08. ACM: New York, NY, USA. p. 385-388. 27. Robertson, G., Horvitz, E., Czerwinski, M., Baudisch, P., Hutchings, D. R., Meyers, B., Robbins, D., and Smith, G. (2004), Scalable Fabric: flexible task management, in Proceedings of AVI '04. ACM: New York, NY, USA. p. 85-89. 28. Robertson, G., van Dantzich, M., Robbins, D., Czerwinski, M., Hinckley, K., Risden, K., Thiel, D., and Gorokhovsky, V. (2000), The Task Gallery: a 3D window manager, in Proceedings of CHI '00. ACM: New York, NY, USA. p. 494-501. 29. Tak, S., Cockburn, A., Humm, K., Ahlström, D., Gutwin, C., and Scarr, J. (2009), Improving Window Switching Interfaces, in Proceedings of INTERACT '09. p. 187-200. 30. Voida, S., Mynatt, E. D., and Edwards, W. K. (2008), Re-framing the desktop interface around the activities of knowledge work, in Proceedings of UIST '08. ACM: New York, NY, USA. p. 211-220. 31. Weinreich, H., Obendorf, H., Herder, E., and Mayer, M. (2008), Not quite the average: An empirical study of Web use. ACM Trans. Web, 2008. 2(1): p. 1-31. 32. Zhang, X., Qu, Y., Giles, C. L., and Song, P. (2008), CiteSense: supporting sensemaking of research literature, in Proceedings of CHI '08. ACM: New York, NY, USA. p. 677-680.

255

Full Papers

Proceedings: NordiCHI 2010, October 16–20, 2010

Determining Usability Requirements into a Call-forTenders. A Case Study on the Development of a Healthcare System Timo Jokela Joticon Oy & University of Helsinki Finland timo.jokela@{joticon.fi, helsinki.fi} ABSTRACT

reported even in main newspapers. A room booking system of a city was not taken into use at all, due to its usability problems. Another public authority had a new website developed, where even the employees could not find basic information such as events or contact information.

Systems of public organizations, the development of which is acquired through public tendering, often suffer from poor usability. To resolve this issue, we explored how to determine usability requirements into a call-for-tenders. Our case is a usability-critical healthcare system to be developed for a city in Finland. We explored different options, and ended up with two measures: task completion success rate for defining effectiveness requirements, and a complementary measure that we call design solution success rate. We could not find appropriate ways for defining requirements that directly address efficiency and user satisfaction. Our case shows that the tendering context sets specific restrictions to the selection of usability measures, especially from the viewpoint of target setting and objective verification.

Why do these kinds of usability problems exist? It is reasonable to examine the specific development context: public tendering. A public authority needs issue a public call-for-tenders when acquiring the development of a new software system. The call-for-tenders includes the requirements for the system. The selected software development company is then committed to deliver what is required in the call-for-tenders.

Usability, user experience, measures, target levels, requirements, success rate, tendering, call-for-tenders

If usability is not among the requirements in the call-fortenders, it is understandable that software development companies do not consider usability when preparing their tenders. Vice versa, if usability is not required, inclusion of usability activities would lead to a more expensive tender and thereby make the tender less competitive.

ACM Classification Keywords

A Background Study

H5.m. Information interfaces and presentation (e.g., HCI): Miscellaneous.

We studied public call-for-tenders for software systems in Finland over a four months period [23]. Many call-fortenders did not include any kinds of usability requirements. Those that did, typical requirements were kind of:

Author Keywords

INTRODUCTION

Systems in public organizations widely suffer from poor usability. A travel management system, used in many public organizations, is a revealing example of consequences the usability problems may lead to. Users report that making a single travel expense report may take up to three hours, and requires contacting user support every time. User failures and negative feedback were

• "Error situations should be avoided" • "The user's memory load should be minimal" • "The software should have a modern and user-friendly interface." The substance of these requirements represents usability. The problem is, however, that fulfillment of these kinds of requirements is not objectively verifiable. It should be agreeable between the customer and the software developer whether the requirements are fulfilled or not.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. NordiCHI 2010, October 16–20, 2010, Reykjavik, Iceland. Copyright 2010 ACM ISBN: 978-1-60558-934-3...$5.00.

For example, how to verify whether an implemented system meets the requirement "Error situations should be avoided", or the requirement "The user's memory load should be minimal?" These kinds of "requirements" are not true

256

Full Papers

Proceedings: NordiCHI 2010, October 16–20, 2010

• We address a research problem – usability measures in the requirements context – that is not much studied. The focus of the research has been on measures in the evaluation context.

requirements; they can rather be regarded as wish lists that the development companies do not need to fulfill. An example of another type of requirement was: • "The software must be usability tested. A certificate of the test should be provided”.

• As we will show, the call-for-tenders context is a relevant aspect since it sets specific challenges to usability requirements. We found no empirical research on usability requirements in this context.

The fulfillment of this kind of requirement is verifiable (‘a certificate’). But in this case the validity is the problem. Validity means that the contents of the requirements should be right; i.e. the fulfillment of requirements truly means appropriate and good usability. A usability test – even multiple tests – does not guarantee good usability.

• The user base of systems of public organizations is remarkable; especially the growing number citizens who are becoming the users of e-governance applications. We find that the need for this research is most relevant from the pragmatic viewpoint, although in his remarkable study on usability measures, Hornbaek [11] does not include this topic among the research foci.

In one call-for-tenders, three user tasks were identified, with a requirement of the style: • "Log in. Success rate without any user guidance: 95%”. One problem here is the lack of coverage of the requirements. The requirements should cover widely enough the different areas of the system. It is not likely that there would only be three user tasks, and thereby the requirements are not comprehensive. (There also is a problem with the target setting; see discussion to follow).

LITERATURE REVIEW Usability Requirements Generally

Usability requirements at a general case are discussed rather widely in literature e.g. [8], [29], [26], [10], and [30]. The existing literature mainly focuses on describing and exploring the concepts and formats related to the definition of usability and the contents of usability requirements document, such as [15] and [13].

In all, our finding was that none of the call-for-tenders included usability requirements that had been verifiable, valid and/ or comprehensive.

Literature discussing target levels propose that one may include separate definitions for the target level (e.g. 90% of users can successfully complete a task) and the minimum acceptable level (e.g. 80% of users can successfully complete a task) [27]. Whiteside et al. [29] suggest that usability requirements be phrased at four levels: worst, planned, best and current.

Usability Measures vs. Usability Requirements

There exists substantial research on measuring usability. A large number of various usability and user experience measures can be found from the HCI literature [11]. Research has been carried out in understanding the correlations between the different usability measures [12] in an evaluation context. Lately, the research has expanded from usability to user experience measures [22].

There are not many empirical research reports on the use of quantitative usability requirements in practice. One of the few is one by Bevan et al. [5] who conducted case studies on quantitative usability evaluations following the Common Industry Format for usability testing, CIF [1] in a purchaser-supplier setting. Two of the case studies included the determination of quantitative usability requirements. In one case, five experts and four non-experts attempted to complete different sets of “typical tasks” in an existing system, and the goal of the new system was to “at least equal and if possible improve on these success rates”. Another case included a step “specification of the usability requirements”. It is reported that the cases were successful; however, the methodological aspects are not discussed in detail. - The author with his earlier colleagues report on measurable usability targets in product development [19].

Our research context represents another aspect of usability measures: rather than measuring usability in the evaluation context, our focus is how to determine measurable usability requirements. Defining measurable usability requirements induces a specific challenge that is not part of ‘only measuring usability’: one should be able to set target levels. For example, let’s say that a user satisfaction questionnaire SUS [6] is considered as a relevant usability measure. Measuring with SUS is quite straightforward. However, if one wants to use SUS in usability requirements, then one should determine the target level. Then we have a question: what should be the appropriate target level (e.g. 70, 80, or perhaps 90; SUS ratings are between 0 and 100)?

Usability Requirements in Software Contracting Research Problem and Relevance

Some authors have studied usability requirements specifically in the public software contracting context, e.g. [2], [24], [9], [3]. Their focus is on usability requirements after the selection of the software developer.

Our research problem is how to determine usability requirements for a call-for-tenders, so that they are verifiable, valid, and comprehensive. We find our research relevant for the following reasons:

There exists only limited research on the specific context of defining usability requirements in the call-for-tenders

257

Full Papers

Proceedings: NordiCHI 2010, October 16–20, 2010

• The use of the system by the citizens will be voluntary.

context. Lauesen [21] is one of the few who specifically addresses the tendering context. He proposes an approach where critical user tasks and the related critical usability factors – such as efficiency, learnability and understandability - are identified, and metrics and target levels are chosen. He, however, does not go to details or report empirical experience.

• The citizens should not end up with erroneous results, whatever task they are to perform. The erroneous results would lead to waste of healthcare resources, or even to negative consequences in the health of citizens (e.g. recording measurement results erroneously). • The systems should be very easy to use by the healthcare personnel, too. They also have a lot of experience of systems of poor usability, and have a very critical attitude towards new systems.

A major example of measurable usability requirements is Voluntary Voting System Guidelines (VVSG) recommendations [16] where ‘overall performance metrics’ are defined. Five usability metrics are identified:

THE PROCESS FOR EXPLORING THE USABILITY REQUIREMENTS

• Total Completion Score: the proportion of users who successfully cast a ballot;

Overview

• Perfect Ballot Index: the ratio of the number of cast ballots containing no erroneous votes to the number of cast ballots containing one or more errors;

The original setting of the work was not a research study. The case study is a normal commercial development project, with the specific feature that usability was seen truly critical for the success of the system. The author’s task was to determine 'usability requirements’. Defining usability requirements in the context of public tendering, however, proved to be challenging and got a nature of research work.

• Voter Inclusion Index: a measure of both voting accuracy and consistency; • Average Voting Session Time: mean time taken per voter to complete the process of activating, filling out, and casting the ballot; and

The requirements team consisted of a requirements analyst, two healthcare professionals and a usability consultant (the author). Experts on tendering software systems reviewed the requirements at several points of the requirements process.

• Average Voter Confidence: mean confidence level expressed by the voters that the system successfully recorded their votes. VVSG, however, is not a call-for-tenders setting in the sense that a contract would be made with single development companies, prior it is to develop the system. Further, VVSG is setting where big resources are expected (e.g. tests with 100 users).

The main source of data arose from the reflective thinking of the of the author, the author’s discussions and feedback with the representatives of the project team, raising the issue in a discussion list of usability professionals, in personal discussions with other usability professionals, and also – by coincidence – in an ISO usability standards meeting where the topics very close to the research problem were discussed. Overall, the requirements were composed iteratively: there are approximately 10 iterative versions of the requirements.

Summary

In summary, there exist substantial earlier work related to usability requirements but only limited earlier work on usability requirements in the context of public tendering. As the basis of our work, we utilize the various measures described in the literature, and the approaches for determining target levels. The idea of Lauesen, as well as the example of VVSG, is based on defining usability measures, and setting target levels. This is also our approach that we took in our project.

Phases

The determination of usability requirements consisted of the four main phases as illustrated in Figure 1: definition of strategic usability targets; identification of user groups; definition of the context of use; and definition of operational usability targets.

RESEARCH CONTEXT: THE HEALTHCARE SYSTEM

Our case study is a requirements determination project for a health-care system where the customer is Oulu, a city with 140 000 habitants in Finland. Healthcare costs getting higher all the time. It is hoped that the new system would decrease costs, through providing a platform through which the citizens can do part of their healthcare activities in their homes through Internet.

Defining Strategic Usability Targets

Strategic usability targets are business-driven high-level goals that address the benefits of usability, without going to the details of user tasks, usability measures, etc. A number of key stakeholders, managers and initiators were interviewed, and the strategic requirements for the system were determined. The usability strategic targets were determined as:

The usability of the system should be good for several reasons: • The financial success of the system requires that the system will be widely used, by majority of the citizens.

• The users (citizens, healthcare personnel) should be able to use the system without training

258

Full Papers

Proceedings: NordiCHI 2010, October 16–20, 2010

• All users (citizens of the target groups, healthcare personnel) should complete their tasks correctly, no matter which task they are to perform

• The user experience when carrying out routine tasks should be high as well, so that the users would use the system regularly (for example, citizens with chronic diseases have such routine tasks)

• The first trial of using the system should be such a positive experience, so that the user would continue using the system also in the future

Overall, usability was seen truly critical to the success of the system.

Figure 1. Usability requirements determination covers four steps (the reference: the JFunnel lifecycle model [17], [18])

identified dozens of user tasks, defined task goals to each of the tasks, and characterized the tasks based on whether they are ‘occasional’ or ‘routine’. An example of task definition is provided in Table 1.

Identification of User Groups

The basic categorization of users was obvious: the citizens and the healthcare personnel. • Citizens include the potential customers of the system (e.g. asthma or diabetes patients) who are accustomed to use basic Internet services, such as network banking and purchasing of tickets.

A specifically challenging job was to determine the task goals. The goals had to be defined precisely enough, so that they could be used as criteria for the correct completion of the tasks.

• The healthcare personnel consists of all those professionals (doctors, nurses, etc.) that need to use the system.

The determination work was based on: • The knowledge that the project team had on the applications and users

These two main groups, naturally, fall into sub categories. We, however, did not find necessary to define the sub categories at the requirements phase.

• Discussions and workshops with users, especially healthcare personnel • Interpretations, questions and comments imposed by the researcher

Definition of the Context of Use

Identification of user tasks, task goals and characteristics was a laborious task. The system contains several applications (appointment booking, weight control, recording of measures, etc.). For each application, we

Field studies – observations of users – were not carried out. First, we had resource limitations. Second, the healthcare personnel of the project team had a good knowledge on user

259

Full Papers

Proceedings: NordiCHI 2010, October 16–20, 2010

• User satisfaction can be measured with a questionnaire, for example SUMI [20] , SUS [6] and QUIS [7]

tasks and task goals – although the knowledge was rather tacit. Most of the users tasks of the system are carried out even today, with exception that they are currently carried out by healthcare personnel only (when citizens visiting hospitals or health centers). Task name

Reserve a time in the laboratory

User

Citizen

Task goal

The following will be correctly defined: date and time; the correct laboratory unit. The user becomes informed: what is the latest time for canceling or changing the time; how he/she should be prepared

Task characteristics

We started to formulate the requirements based on the 9241-11 attributes of usability. The task proved not to be simple, and several iterations were required. THE RESULT: THE USABILITY MEASURES, MEASURING INSTRUMENTS AND TARGET LEVELS

A measurable usability requirement consists of three elements: • Definition of the usability measure. The attributes effectiveness, efficiency, and satisfaction are of too high level of abstraction. One needs to define the precise measure(s) for the attributes. In some sources, e.g. VVSG [16], the measure is part of the definition of the measuring instrument. We, however, find it clearer to define these separately.

To be carried out occasionally

• Description of the measuring instruments. One needs to define exactly how the measuring is to be carried out.

Table 1. An example of the definition of a user task, task goal and task characteristics

• Target level. The target level defines ‘how good the system is to be’; it is the ultimate requirement for the usability.

Determination of Operational Usability Targets

The outcome of this phase is the ultimate goal of the requirements work: the usability requirements. As said, the aim was that the requirements be verifiable, valid, and comprehensive.

In the following, we discuss the measures from each of the usability attributes – effectiveness, efficiency, and satisfaction – separately. We justify our selections, and also describe why some potential measures were abandoned.

When examining the strategic usability goals, we concluded that the definition of usability of ISO 9241-11 is applicable in our case: “the extent to which a product can be used by specified users to achieve specified goals with effectiveness, efficiency and satisfaction in a specified context of use” [13].

Measure for Effectiveness Definition of the Measure

One of our strategic usability objectives was that “all users should be able to carry out their tasks correctly”. A natural basic choice for the measure is Nielsen’s task success rate: “the percentage of tasks that users complete correctly” [25].

• Effectiveness: users should be able to correctly perform their tasks • Efficiency: it should be easy to do the routine tasks

We, however, modified the Nielsen’s measure in two steps. First, we modified the measure as to “the percentage of users that complete a task correctly”. We call this task completion success rate. The reason for this modification was that we wanted to identify exactly those tasks that users have problems with. Thereby we can report the software developer exactly those parts of the system where improvements are required.

• Satisfaction: the user experience should be of high level, because the use of the system is voluntary and one should want to use the system after their first experience We are aware that there exist many other potential attributes of ‘user experience’ [22]. We, however, found the ‘traditional’ usability attributes adequate, because they very well match with the strategic usability objectives. In addition, this system is not meant for competitive consumer market, and we reasoned that no specific ‘gimmicks’ are needed, just a highly usable system.

The other modification was about the ‘percentage’. It is not possible to verify a requirement that is formulated in a style “90% of users should complete a task correctly” (unless one tests with the whole population). When one tests with a sample population, one only gets a certain confidence on the minimum and maximum number of users that complete a task correctly.

The literature reports a bunch of various usability measures for each attribute: • An example of an effectiveness measure is task success rate: the percentage of tasks that a user can successfully complete [25].

We also considered the style “9 out of 10 test users should complete a task correctly”. The problem in this style is that the result depends on the set of test persons, and thereby the results do not necessarily represent the whole target population.

• Efficiency can be measured by the mean time needed to successfully complete a task.

260

Full Papers

Proceedings: NordiCHI 2010, October 16–20, 2010

Thereby, our conclusion was to define the measure in a statistical form: 95% confidence that at least a predefined percentage of the target population completes a (specific) task correctly. We defined a task to be completed correctly when • The outcome of the task is correct

Number of test persons

Maximum number of unsuccessful attempts

10

0

20

1

26

2 Table 3. The maximum number of unsuccessful attempts 1

• The user believes that he/ she has accomplished the correct outcome

Measures for efficiency

In other cases, the task is not completed correctly.

Efficiency was found strategically relevant, especially because some users are expected to use the system routinely. We explored three alternatives for measuring efficiency

For comprehensive requirements, we had to define what is the ‘correct completion’ for each task. This led to a document of 70 pages that includes 20 – 50 definitions per application (an example is the definition: see Table 1).

• Task time • Adherence to design requirements

Measuring Instrument

• Task efficiency success rate

The measurement instrument should answer the question: How the task completion success rate is measured? The natural choice for the measuring instrument is usability testing.

Efficiency is typically measured through task time. We, however, did not find feasible to determine efficiency requirements using tasks time, because we did not have any measured benchmarks for task times. And we did not have resources to benchmark task times during the project.

A definition of the usability tests was produced, defining who are the test users, what are the tasks that the users are to do, what is the criterion for ‘task completion success’, etc. We defined, for example, that “the customer may select any set of user tasks, presuming that they represent natural order of work flow of the test person”. We provide the software developer the possibility to comment on selection and definition of the test tasks, prior the tests.

We also considered using design requirements, i.e. adherence to usability guidelines. We especially considered ISO 9241-110 [14] that includes appropriate guidelines, for example: “The steps required by the dialogue should be appropriate to the completion of the task, i.e. necessary steps should be included and unnecessary steps should be avoided.”

Target Level

We, however, decided to reject the measure because we could not understand how such requirements could be objectively verifiable. The verification would solely be based on expert reviews. We considered that this might lead to problems in agreeing on the results.

We set the target level as to: 95% confidence that at least 75% of target population completes the task correctly, as illustrated in Table 2. Users

Tasks

Target: the number of successful users

Citizens and healthcare personnel

Any task

95% confidence that at least a 75% of target population completes the task correctly

We further considered a subjective measure: the percentage of users that do not perceive the task unnecessary complex to do. Through this measure we hoped to addresses those design problems that lead to unnecessary user steps or actions. We, however, found difficult to measure this ‘perceived complexity’ in a valid way.

Table 2. The target level for task completion success rate

The percentage (75%) may seem quite modest - our strategic target was defined as to “all users should complete their tasks correctly”.

Our conclusion was we could not find an appropriate way for defining directly efficiency requirements.

We chose the target level because “at least 75% with the confidence of 95%” is statistically quite a high target anyway. For example, it we test with 10 users, all of them should be successful. See Table 3.

Satisfaction is relevant generally, and especially in this case because the strategic target was that users’ experience should be positive. The requirement was that after a first trial, a user should want to use the system again.

Measures for satisfaction

For measuring satisfaction, we considered using SUS, based on recent promising findings [28] [4]. We, however,

1

261

See http://www.measuringusability.com/comp_sample.php

Full Papers

Proceedings: NordiCHI 2010, October 16–20, 2010

abandoned SUS because we were not able to find any basis for defining the target level (i.e. should the target level be 70, 80, or perhaps 90?).

Overall, we found important to have a way capture other but the effectiveness aspect (task completion success rate); and design solution success rate is a complementary aspect.

Norms exist for the SUMI questionnaire [20]. Thinking afterwards, we could have considered more seriously the use of SUMI. Still setting the target level may have been a problem. In addition, SUMI is a ‘heavy weight’ commercial questionnaire, and its costs may have been a hinder.

We believe that the measure is objectively verifiable, although we see the risk that agreeing on a user problem may be more difficult that verifying the outcome of a task. Measuring Instrument

The measurement is usability testing. The tests can be the same ones as those for measuring the task success rate (see discussions above).

We also considered ISO 9241-110 guidelines. Our rationale was that designing the user interface following the guidelines (e.g. “If typical input values are required for a task, these values should be available to the user automatically as defaults.”) would probably have a positive impact on user satisfaction. We rejected the measure because of the same reasons as with efficiency: we considered that this might lead to problems in agreeing on the results.

Target Level

We set the target level: 95% confidence that at least a 50% of the target population do not have problems with a specific design solution, as illustrated in Table 2. We set the target level lower than the effectiveness target because we find that effectiveness is the key attribute.

Our conclusion was we could not find an appropriate way for defining directly satisfaction requirements. A mixed measure

We anyway found it important to have other usability requirements determined, in addition to effectiveness. We wanted to capture efficiency and satisfaction in some way. Our solution was a metric that we call design solution success rate.

Users

Tasks

Target: the number of successful users

Citizens and healthcare personnel

Any task

95% confidence that at least a 50% of the target population do not have problems with a design solution

Table 4. The target level for design solution success rate Definition of the Measure

We define design solution success rate as 95% confidence that at least a predefined percentage of target population do not have problems with a specific design solution.

Table 5 illustrates that if we test with 10 users, two users are allowed to have problems with a specific design solution.

A ‘design solution’ may represent different areas of design: terminology, presentation of information, error handling and recovery, dialogues, tables and forms, wizards, the steps required to accomplish a user task, etc.

Number of test persons

Maximum number of user problems

10

2

16

4

21

6

We defined that a user has a problem with a design solution when

Table 5. The maximum number of user problems

• When performing a task, a user gets into a trouble due to a specific design solution: makes a longer stop, is uncertain how to continue, makes errors, gets frustrated, etc. A user is not regarded to have a problem if an error was just a slip, or the problem is a minor one.

Summary

In summary, we ended up with two measures: task completion success rate and design solution success rate. We could not find a way to define direct measures for efficiency and user satisfaction (Figure 2).

• A post-task interview confirms that the user problem was due to the specific design solution. We are aware that user errors are typically considered as measures of effectiveness [13]. But our hypothesis is that design solution success rate contributes for the evaluation of efficiency and satisfaction: • Time spent recovering from problems would impact efficiency. • Frustration with problems with design solutions would lead to dissatisfaction. Frustration with unnecessary complex steps would also reveal efficiency problems.

262

Full Papers

Proceedings: NordiCHI 2010, October 16–20, 2010

• Validity. We deduced the usability requirements from the business needs of the system. We justified the selection of the measures and target levels. We believe that we reached at least reasonable valid requirements; but we advice the reader to critically go through our justifications. • Coverage. The coverage is based on the systematic determination of user tasks (with outcomes). We tried to capture all the relevant tasks; the document is approximately 70 pages.

Figure 2. The usability measures of the case project CURRENT STATUS OF THE PROJECT

The usability requirements were defined as obligatory requirements in the call-for-tenders. In other words, the development company is obliged to develop a system that meets the usability requirements.

When we contrast our results with work of others, we can find that • Unlike what is proposed in literature (e.g. [27]. [29]), we found not applicable to determine ‘acceptable’ and ‘minimum acceptable’, or ‘worst, planned, best and current’ levels of requirement. At least in our case our conclusion was that one needs only one target, and that is the minimum acceptable level.

The city received several tenders. We found this as a good result: the tough usability requirements did not scare the software suppliers from making tenders. At the time of writing this paper, the supplier is selected, and the implementation of the system is about to start.

• In our context, we found that there was no reason to categorize tasks into more or less critical ones. We just could not identify any task where user failure would be acceptable. Also this made the determination of requirements easier; we did not need to identify less and more important tasks.

DISCUSSION

We determined usability requirements – measures, measuring instruments and target levels - into a call-fortenders for the development of a usability-critical healthcare system. We explored different options, and ended up with two measures:

• When contrasting our measures with ones of VVSG, some differences can be found. For example, in contrast with our Task Completion Success Rate, VVSG has a measure Total Completion Score (TCS) that is the proportion of users who successfully cast a ballot, whether or not the ballot contains erroneous votes. In other words, TSC includes incorrectly completed tasks, and another measure for measuring the proportion of successful votes. In our context, we found that it was adequate to use one measure only for effectiveness.

(1) Task completion success rate: 95% confidence that at least a predefined percentage of the target population completes a (specific) task correctly (2) Design solution success rate: 95% confidence that at least a predefined percentage of target population do not have problems with a specific design solution call We could not find appropriate ways for defining requirements directly for efficiency and user satisfaction. Our study shows how the tendering context set specific restrictions to the selection of usability measures. The measures should be driven by the strategic usability goals of the system. One should select measures where one has a clear base, on which to define the target levels. The measuring instrument should objective, fair to the developer, and practical to use without extensive resources. We have presented reasoning on choosing the measures and setting the target levels, as proposed in [11].

The starting point of our research was solely practical. Still, we can find that we addressed some of the challenges posed in Hornbaek’s study [11]: • We interpret that the task completion success rate is about ‘macro tasks’, and the design solution success rate is about ‘micro tasks’. Thereby, we make the distinction proposed in [11], and even focus on the ‘macro tasks’, as suggested. • The design solution success rate can be regarded as one approach to extend measures of satisfaction beyond postuse questionnaires, although one needs to consider that it is not a direct user satisfaction measure.

We aimed for usability requirements that are verifiable, valid, and comprehensive. To which extent we were successful then? • Verifiability. Task success rate should be rather verifiable; probably the biggest risk is in the definitions of outcomes of the user tasks (there are so many user tasks). A risk may be in agreeing the problems when testing the design solution success rate. Naturally, we know more about this after the tests have been carried out.

Limitations

While we believe that the effectiveness measure - task completion success rate - is valid (a slight modification of an existing measure), we understand the potential problems of the design solution success measure. We do not have any evidence about how it correlates to user satisfaction. The idea of measuring design problems is very close to one of measuring user errors, which is often used as a measure for

263

Full Papers

Proceedings: NordiCHI 2010, October 16–20, 2010

effectiveness. Further, although we expect the measure be objective, we admit that there are risks, too.

usability’ with questionnaires? Or should there be developed other kinds of subjective measures for satisfaction for defining requirements?

At this stage, we do not yet have empirical results on the impact that the usability requirements have on the development project and the usability of the final system.

There is also need for research for how to deduce the appropriate measures and target levels from the business context of the system. We explain how we came up with the specific measures and target levels in the case of our healthcare system. But we find that there exists only limited guidance on how to do this.

Implications to Practice

We believe that this work is useful for usability practitioners who are involved in preparing public call-fortenders for software systems. This article provides lessons learnt, and also examples for how to determine usability requirements.

CONCLUSION

We determined usability requirements into a call-fortenders for the development of a usability-critical healthcare system for a European city. We determined effectiveness requirements using a variation of an existing measure. Direct measures for efficiency and satisfaction requirements could not be determined; we used an indirect measure that we call design solution success rate. Our experience shows that the tendering context sets specific considerations to the applicable types of usability measures.

The paper also shows that determining appropriate usability measures and setting target levels is a challenging task. One should aim to usability requirements that are verifiable, valid and comprehensive. Defining usability requirement requires quite remarkable resources and usability expertise. We believe that this work is interesting also for a larger audience of usability practitioners. The author with earlier colleagues reported a positive experience of setting measurable usability requirements in product development [19]. We proposed in that “it is not necessary to know how to measure exactly at the time of determining the requirements”. The public tendering context, however, is more demanding. Now we find that our earlier conclusion is not true: for objective verification, one absolutely needs to define the measuring instruments in the requirement phase.

ACKNOWLEDGEMENTS

I wish to thank the team and management of the ‘Omahoito’ team for co-operation and an interesting case: Riikka Hirvasniemi, Keijo Koski, Jarkko Polvi, Maarit Salmi and Seppo Voutilainen. I also wish to thank Jeff Sauro for the hint for the statistical style of defining usability measures. Further I thank the CloudWS program of Tivit and Tekes, which supported especially the theoretical part and writing of this work. Finally, I thank the anonymous reviewers for the valuable comments.

Topics for Future Research

Only few case studies are published on measurable usability requirements. This article is one of them, in the specific context of public tendering. More empirical studies are welcome, also in the wider context of defining measurable usability requirements.

REFERENCES

1.

Natural follow-up issues are to examine is to see how the development project progresses, and what will be the usability of the final system. One specific interest is to analyze how the design solution success rate measure works in practice: how easy it is to identify and agree on the user problems at the level of individual design solutions.

2.

3. 4.

Another viewpoint of measurable usability requirements is the impact on software development companies. They should be able to deliver ‘measured usability’ – not only e.g. number of usability tests or user studies. This probably is a new challenge. But as we see, verifiable, valid and comprehensive usability requirements this is the only way to assure good usability from the purchaser’s point of view.

5.

6.

There is also need for more theoretical work. We totally agree with the need for research to “validate and standardize the host of subjective satisfaction questionnaires”, proposed by Hornbaek [11]. But in addition to this, we find that research is needed also in how to set target levels using the satisfaction questionnaires. Is it feasible at all to determine target level levels for ‘good

7.

8.

264

ANSI, Common Industry Format for Usability Test Reports. 2001: NCITS 354-2001. Artman, H. Procurer Usability Requirements: Negotiations in Contract Development. in NordiCHI 2002. 2002. Århus. Artman, H. and S. Zällh, Finding a way to usability: procurement of a taxi dispatch system. 2005. Bangor, A., P. Kortum, and J. Miller, Determining What Individual SUS Scores Mean: Adding an Adjective Rating Scale. Journal of Usability Studies, 2009. 4(3): p. 114-123. Bevan, N., et al., Guide to specifying and evaluating usability as part of a contract, version1.0. PRUE project. 2002, Serco Usability Services: London. p. 47. Brooke, J., SUS - A "quick and dirty" usability scale. 1986, Digital Equipment Co. Ltd. Chin, J.P., V.A. Diehl, and K.L. Norman. Development of an instrument measuring user satisfaction of the human-computer interface. in Proceedings of SIGCHI '88. 1988. New York. Good, M., et al. User-derived impact analysis as a tool for usability engineering. in Conference Proceedings

Full Papers

Proceedings: NordiCHI 2010, October 16–20, 2010

9.

10.

11.

12.

13.

14.

15. 16.

17.

18. 19.

on Human Factors in Computing Systems. 1986. Boston. Gulliksen, J., et al., Key Principles of User-Centred Systems Design. Behaviour & Information Technology, 2003. 22(6): p. 397-409. Hix, D. and H.R. Hartson, Developing User Interfaces: Ensuring Usability Through Product & Process. 1993, New York: John Wiley & Sons. 416. Hornbaek, K., Current practice in measuring usability: Challenges to usability studies and research. Int. J. Human-Computer Studies, 2006. 64. Hornbaek, K. and E.L.-C. Law. Meta-Analysis of Correlations Among Usability Measures. in CHI 2007 Proceedings. 2007. San Jose. ISO/IEC, 9241-11 Ergonomic requirements for office work with visual display terminals (VDT)s - Part 11 Guidance on usability. 1998: ISO/IEC 9241-11: 1998 (E). ISO/IEC, 9241-110 Ergonomics of human-system interaction -- Part 110: Dialogue principles. 2006: ISO/IEC 9241-110: 2006 (E). ISO/IEC, 25062 Common Industry Format (CIF) for usability test reports. 2006: ISO/IEC 13407: 2006 (E). Jeffrey, W. and &al, Voluntary Voting System Guidelines Recommendations to the Election Assistance Commission. 2007, Technical Guidelines Development Committee, United States Election Assistance Commission. Jokela, T. A Two-Level Approach for Determining Measurable Usability Targets. in 5th COST294MAUSE Open Workshop on Valid Useful User Experience Measurement (VUUM). 2008. Reykjavik, Island. Jokela, T., Usability Guided Interaction Design (in Finnish). 2010: Väylä-Yhtiöt Oy. Jokela, T., et al., Quantitative Usability Requirements in the Development of the User Interface of a Mobile Phone. A Case Study. Personal and Ubiquitous Computing, 2006. 10(6): p. 345-355.

20. Kirakowski, J. and M. Corbett, SUMI: The software usability measurement inventory. British Journal of Educational Technology, 1993. 24(3): p. 210-212. 21. Lauesen, S. Usability requirements in a tender process. in OZCHI’98. 1998. Adelaide. 22. Law, E., et al. Towards a UX Manifesto. 2007. Lancaster, UK: http://www.cost294.org. 23. Lehtonen, T., et al., To Which Extent Usability Truly Matters? A Study on Usability Requirements in Callfor-Tenders of Software Systems, Issued by Public Authorities. Manuscript (in review for publication). 2010. 24. Markensten, E. and H. Artman. Procuring a Usable System Using Unemployed Personas in NordiCHI 2004. 2004. Tampere. 25. Nielsen, J., Success Rate: The Simplest Usability Metric. http://www.useit.com/alertbox/20010218.html. 2001. 26. Nielsen, J., Usability Engineering. 1993, San Diego: Academic Press, Inc. 358. 27. NIST, Proposed Industry Format for Usability Requirements. Draft version 0.62. 8-Aug-04. 2004. 28. Sauro, J. and J.S. Dumas. Comparison of Three OneQuestion, Post-Task Usability Questionnaires. in CHI 2009. 2009. Boston. 29. Whiteside, J., J. Bennett, and K. Holtzblatt, Usability Engineering: Our Experience and Evolution, in Handbook of human-computer interaction, M. Helander, Editor. 1988, North-Holland: Amsterdam. p. 791-817. 30. Wixon, D. and C. Wilson, The Usability Engineering Framework for Product Design and Evaluation, in Handbook of Human-Computer Interaction, M. Helander, T. Landauer, and P. Prabhu, Editors. 1997, Elsevier Science B.V: Amsterdam. p. 653-688.

265

Full Papers

Proceedings: NordiCHI 2010, October 16–20, 2010

Descriptive Quality of Experience for Mobile 3D Video Satu Jumisko-Pyykkö(1), Dominik Strohmeier(2), (1)Tampere University of Technology P.O.Box 589, 33101 Tampere, Finland satu.jumisko-pyykko, timo.j.utriainen{@tut.fi}

Timo Utriainen(1), Kristina Kunze(2) (2)Ilmenau University of Technology P.O.Box 100565, 98684 Ilmenau, Germany dominik.strohmeier, kristina.kunze{@tuilmenau.de}

ABSTRACT

context of use. To understand this construct of quality components, subjective quality evaluation experiments with prospective users are conducted.

Perceptual quality evaluation experiments are used to assess the excellence of multimedia quality. However, these studies disregard qualitative experiential descriptions, interpretations, and impressions of quality. The goal of this paper is to identify general descriptive characteristics of experienced quality of 3D video on mobile devices. We conducted five studies in which descriptive data was collected after the psychoperceptual quality evaluation experiment. Qualitative semi-structured interviews and written attribute description tasks were conducted with over 90 naïve participants. The experiments contained an extensive and heterogeneous set of produced quality by varying content, level of depth, compression and transmission parameters, and audio and display factors for 3D. The results showed that quality of experience is constructed from four main components, 1) visual quality, 2) viewing experience, 3) content, and 4) quality of other modalities and their interactions.

The term ‘quality’ has two dimensions: ‘the degree of excellence of something’ and ‘a distinctive attribute or characteristic possessed by -- something’ [24]. The excellence of multimedia quality is conventionally explored using quantitative quality evaluation experiments according to standardized procedures (e.g. by ITU [12,13]). Within these approaches less attention has been paid to understand the descriptive experiential dimensions of multimedia quality. According to previous descriptive studies, multimedia quality of experience is a more complex phenomenon than just a collection of impairment characteristics [16]. The active nature of human perceptual processes, in which material-driven characteristics as well as high-level knowledge, emotions, and attitudes are represented, is a necessary part of it [6,8,16,21,25]. For example, experienced audiovisual quality of mobile video is characterized by the impressions of audio, video, audiovisual, content, and usage factors [16]. In overall, current descriptive studies are rare. They target on 2D systems, explore mainly one medium at the time, or use settings irrelevant for 3D on mobile devices, and finally conclude quality of experience under the constraints of the study-dependent characteristics [1,10,16,22,23,27,28,32].

Author Keywords

Quality perception, quality of experience, multimedia, 3D ACM CLASSIFICATION KEYWORDS

H.5.1 Multimedia Evaluation/methodology

Information

Systems:

INTRODUCTION

Goal of this paper is to identify general descriptive characteristics of experienced quality of 3D video on mobile device. We conduct five qualitative studies where data-collection is based on interviews or written attribute descriptions of naive participants, and collected after a psychoperceptual quality evaluation experiment. The studies contained a broad set of heterogeneous, produced quality factors by varying content, level of depth, compression and transmission parameters, audio and display factors for 3D. We summarize the results as a model of descriptive quality of experience for 3D mobile video. Our work benefits academia to build up deeper understanding on the nature of perceptual quality and practitioners to develop descriptive (vocabulary-based) evaluation tools and apply the gained knowledge in the system development.

3D is expected to emerge from cinema to personal consumer electronics. For 3D on mobile devices, interest has been expressed towards video, television, and gaming [19]. In these applications, 3D can provide a greater immersion experience for users if this added value is not restrained by erroneous image quality [32]. The overall excellence of 3D is influenced by content production, required high-level of compression, requirements set by transmission channel, and finally, the quality of the display. However, the final quality is determined by user’s perception, influenced by his/her characteristics and the Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. NordiCHI 2010, October 16–20, 2010, Reykjavik, Iceland. Copyright 2010 ACM ISBN: 978-1-60558-934-3...$5.00.

266

Full Papers

Proceedings: NordiCHI 2010, October 16–20, 2010

QUALITY OF EXPERIENCE – 3D VIDEO ON MOBILE DEVICE AND ITS EVALUATION

pattern of low- and high-level factors can be identified in other related descriptive studies.

Quality of Experience (QoE) describes the relationship between perceived and produced quality and is also understood as a “multidimensional construct of user perceptions and behaviors [35]”. Produced quality represents provided quality of a certain system and is limited by its constraints. In mobile 3DTV systems, produced quality is characterized by various factors along the production chain on the levels of content, media and network [7]. Impairments like blocking artifacts or frame loss occur, propagate jointly over the chain and can influence the user’s experienced quality [2,7,32].

The majority of descriptions of quality are focused on lowlevel factors containing the descriptions of apparent features of uni- and multimodal material. Low-level factors in descriptive evaluation of audio and video often offer a description of either the excellence of stimuli or, with the use of antonymous expressions, their perceived impairments. In image and video quality research, low-level factors relate to the sharpness of the stimulus, its colors or brightness [1,10,27,28,32]. This is also valid for 3D videos (blurry, artifacts [33], brilliant [32]). In addition, low-level factors in 3D research often characterize an impairment of video quality through the display technology and crosstalk [32,33]. In audio research, assessors’ low-level descriptions lists timbral aspects (hissing, metallic [22]; “timbral balance between low-frequency and high-frequency” [22]) and distortions of the sound [22,23]. The combined audiovisual low-level factors emphasize motion, its errors and the synchronization between audio and video, and the balance between the qualities of the two media [16,26].

Perceived (or experienced) quality describes quality from the user’s point of view. It is characterized by the user’s active perceptual low- and high-level processing. Low-level processing extracts relevant features from the perceived sensory information. In vision, color, brightness, contrast, motion, and stereoscopic information are processed in the early stage of the perceptual process [8]. In audio, pitch, loudness, timbre or runtime differences for localization are relevant features [21]. High-level cognitive processing refers to the interpretation of stimuli through their personal meaning and relevance to human goal-oriented actions. This process is influenced by our knowledge, expectations, and emotions [6,25]. It affects the importance of each sensory attribute and more broadly enables human contextual behavior and active quality interpretation [14, 16,25-27]

Going beyond the apparent features of stimuli, the highlevel factors describe content, usage factors, and in 3DTV research, the added value of 3D. While only few were identified in descriptive audio quality research [22,23], high-level factors emphasized in visual quality evaluation, especially related to 3D videos. One of the key factors is object recognition [1,32]. In addition, the look of the stimuli is interpreted and assessors describe content as “professional” [27] or “amateurish” [27] or classify stimuli as “artistic” [27] or “life-like” [10]. A special aspect related to 3DTV is the added value which refers to an enhanced experience of 3D video in comparison to existing systems and an impression of increased reality or “emotional experience” [10] of the content [5,10,32].

To assess the impact of quality factors on perceived quality, different methods are applied in audiovisual and 3D-related research. Standardized psychoperceptual evaluation methods and their extensions study perceived quality quantitatively [9,12,13,18]. They are useful tools to examine a predefined set of technical quality parameters. However, their ability to discover new quality factors and quality structures is limited. To overcome this shortcoming, descriptive methods have been developed applying either interview-based approaches [16,27] or sensory profiling methods [29,32]. These descriptive methods extend quality evaluation with a hypothesis-free quality factor elicitation.

Taken together, currently existing descriptors construct audiovisual (3D) Quality of Experience from components of low-level and high-level factors as proposed by JumiskoPyykkö et al. [16]. Especially low-level factors show strong study-dependent results highlighting specific excellence of the system under evaluation. The high-level factors of 3D video were identified as content, usage factors, and the added value of 3D. Although the components of visual and audio descriptions can also be found in audiovisual descriptions, the results of multimodal descriptions show that audiovisual quality is more than just their combination. Following, we present our elicitation of experienced quality factors for audiovisual 3D quality perception.

DESCRIPTIVE QUALITY OF EXPERIENCE: FROM CHARACTERISTICS OF STIMULI TO BEYOND IT

In descriptive assessment methods for multimedia quality, assessors describe the factors that form their individual (multidimensional) QoE rationale. In contrast to quantitative approaches [12,13], quality is thereby expressed qualitatively in terms of open questions or sensory attributes and their definitions [16,27,32]. In general, Jumisko-Pyykkö et al. [16] showed that people derive two different kinds of quality descriptions. “Lowlevel factors” are directly derived from the characteristics of the presented stimuli, whereas “high-level factors” represent attributes that take into account users’ goals of using the system or their knowledge about the system. This

RESEARCH METHOD

The experienced quality descriptions were collected in five studies (following exp1 to exp5) into different research challenges for mobile 3D television and video. The experiment characteristics are summarized in Table 1.

267

Full Papers

Proceedings: NordiCHI 2010, October 16–20, 2010

Goal – We investigated audiovisual experienced quality under monoscopic and stereoscopic video presentation (exp1, exp4, exp5), studied the influence of video coding methods, coding settings, and transmission parameters on experienced quality of mobile 3D television (exp2 to exp5). All experiments were conducted in controlled laboratory environments. In addition, exp4 was conducted in indoor quasi-experimental (home-like) settings and exp5 in two different real-life-like settings.

positive/negative?” The interviewer used only the terms introduced by the participant. In exp 4, the interview was complemented with a stimuli-assisted description task in which a set of stimuli was shown one by one in randomized order and the test participant was shortly interviewed after each presented video. Stimuli - Heterogeneous stimuli material that represented the characteristics of potential contents for mobile 3D TV [19] were used in the experiments. The stimuli contained synthetic and natural video scenes, varying depth levels, and differing amounts of details (spatial) and movement (temporal) [Table 1].

Participants – Participants were aged 18-45 years and both genders were equally represented. The majority of them represented naïve participants (not studying, working or otherwise engaged in fields relating to video processing or television). The participation to only one of the experiments was accepted.

The variables of the experiments were selected to have impact on spatial, temporal, and depth quality of the test items. To study a quality comparison between 2D and 3D video, the presentation mode of the stereoscopic display was varied. Different coding methods, quality levels, bit rates, frame rates, and transmission error rates were varied for mobile 3D television and video. The related audio varied presentation mode (mono, stereo) and bit rates. All test stimuli were encoded using the recommended codecs: mp4v, H.264/AVC Reference Software JM 14.2 and H.264/MVC reference Software JMVC 5.0.5 and H.264/AVC (x264 b1077) with baseline profile for video. Audio was encoded with WMA9 and AAC-HEv2.

Procedure - All studies consisted of two parts: psychoperceptual and descriptive quality evaluation. Prior to the psychoperceptual evaluation sensorial tests, demographic data collection, and quality anchoring and training took a place. During the test session, the stimuli were presented one by one and rated independently and retrospectively using a bidimensional research method of acceptance [13,18]. The quantitative evaluation task was repeated in different contexts of use (exp4). The post-test session collected qualitative quality descriptions using two different methods. Open Profiling of Quality (OPQ) was used in exp1 to exp3 [32]. During the attribute elicitation task of OPQ, test participants watch a subset of test stimuli. While watching, they write down their own quality attributes. Hereby, assessors are not limited in the derivation of quality attributes. Each individual quality attribute should be unique, it should describe exactly one aspect of perceived quality, and the assessor must be able to define it. These definitions are collected at the end of the elicitation task for each attribute to better understand their meaning. Again, the test participants write down these definitions using their own words (e.g. movement - movements cannot be identified). In previous OPQ studies, these descriptions have only been used for interpretation of the individual attributes in the final analysis [32]. However, these definitions represent another qualitative view on experienced quality. In this paper, we took the descriptions of attributes as an independent dataset of elicited experienced quality factors.

Presentation - Two different devices equipped with twoview autostereoscopic displays were used in the experiments. These displays provide two separate views, one for each eye of the observer by selectively blocking the light when viewed from different directions [4]. The brain fuses the two different images into a three-dimensional image freely floating above the display. Autostereoscopic displays do not require the use of specialized glasses to experience the 3D effect as the light filter is built into the display [4]. Audio was played back using headphones (exp1, 3: AKG K450; exp4-5: Sony MDR-E818LP). Method of analysis - The analysis followed the principles of Grounded Theory framework through systematical steps of open coding, concept development and categorizing [31]. The analysis of components of quality of experience was conducted in two phases: Firstly, all studies were analyzed independently. Secondly, over the studies analysis was based on the results of independent analysis. 1. Analysis of components of quality of experience in independent studies - The analysis contained three main steps: 1) Open coding towards concepts: In the interview studies (exp4-5) all recorded material was transcribed to text, meaningful sentences were extracted and open-coded from all data for creating concepts. This phase was conducted by one researcher and reviewed by another researcher. In the attribute data (exp1-3), as the descriptions were short (one sentence), well-defined, and exact compared to the interview data they were treated directly as the concepts in the analysis.

In exp4 and exp5, the post-test session contained a semistructured interview about experienced quality. During the interview the main question with slight variations was presented several times and the supporting questions were asked to clarify further the answers from the main question. The main question was: “What kind of factors you paid attention to while evaluating quality?” and the supporting questions were: “Please could you clarify was X among the positive/negative factors/pleasant/unpleasant?” and “Which of the factors you mentioned was the most

268

Full Papers

Proceedings: NordiCHI 2010, October 16–20, 2010

Table 1. Characteristics of the experiments (VS=visual spatial details, VT=visual temporal motion, VD=amount of depth; L=low, M=medium, H=high) EXPERIMENT

DEVICE

EXPERIMENT VARIABLES

Exp1 [32] Sample size: 15

HDDP

Video: presentation mode (2D/3D); Audio: presentation mode (mono/stereo); Content: 6 contents

Exp2 [32] Sample size: 15

Exp3 [34] Sample size: 17

Exp4 [15,17] Sample size: 30

Exp5 [15,17] Sample size: 15

427x240px 3.5” headphones

HDDP 427x240px 3.5 “

HDDP 427x240px 3.5 “ headphones

3D LCD 400x480px 3.3 “ headphones

3D LCD 400x480px 3.3” headphones

STIMULI CHARACTERISTICS AND AUDIOVISUAL CODECS Length: ~ 18s Videos: Synthetic and Natural Presentation mode: 2D and 3D Quality level: Highly acceptable

Video: 4 coding schemes, 2 quality levels (low: 74-160kbps bit rate, high: 160452kbps); Content: 6 contents

Video: mp4v, 10-22Mbit/s, 25fps Audio: WMA 9, 48kHz 16bit Length: ~ 10s Videos: Synthetic and natural Presentation mode: 3D Quality level: Highly acceptable

Video: 3 coding schemes @ slice and nonslice mode, 2 MFER rates (10%, 20%); Audio: clean audio; Content: 4 contents

Video: H.264/AVC (JMVC 5.0.5) Audio: none Length: ~ 60s Videos: Synthetic and natural Presentation mode: 3D Quality level: Highly acceptable

Video: H.264/AVC (JM 14.2), MVC (JMVC 5.0.5) Audio: WMA 9 11/44.1 kHz Video: bit rate 160-1536kbps, frame rate 10- Length: ~ 30s 24fps, presentation mode (2D/3D); Videos: Synthetic and natural Content: 4 contents; Presentation mode: 2D and 3D Context of use: laboratory, home-like Quality level: Mainly unacceptable

Video: bit rate 320-768kbps, presentation mode (2D/3D); Audio: bit rate 18-48kbps; Content: 4 contents; Context of use: laboratory, bus-travel, station-wait

Video: H.264/AVC (x264 b1077) Audio: AAC-HEv2 (Nero 1.3.3.0), 44.1kHz 48kbps Length: ~30s Videos: Synthetic and natural Presentation mode: 2D and 3D Quality level: Mainly unacceptable

CONTENT CHARACTERISTICS VS VT VD Animation H H M Documentary H M H Videoconference M L M User-created H H H Music video M M M Documentary H M H Talking head Animation Feature film Horse Mountain Sports

M H H M H M

L M H L L H

L M M H H H

Documentary Animation Nature Roller

H H H H

M M M H

H H M H

Animation User-created Documentary Series

H M L L

H L M L

M M H M

Animation User-created Documentary Series

H M L L

H L M L

M M H M

Video: H.264/AVC (x264 b1077) Audio: AAC-HEv2 (Nero 1.3.3.0), 44.1kHz

2) All concepts were organized into subcategories and the subcategories were further organized under main categories. For interview data, the categorization was reviewed by another researcher. For the attribute data, the initial categorization was done independently by three independent researchers and final categories were constructed in the consensus between them. 3) Frequencies in each category were determined by counting the number of the participants who mentioned it. Several mentions of the same concept by the same participant were recorded only once. For 20% of randomly selected pieces of data (attribute descriptions or lettered interviews) inter-rater reliability is excellent (exp1-3: Cohen’s Kappa: 0.8, exp45: Cohen’s Kappa: 1.0).

the term ‘component’ to refer to any element of quality to combine the previously used terms of factors, components, dimensions, categories and aspects under the same umbrella. The identified concepts were categorized into initial subcategories and major categories by two researchers independently and reviewed by two other researchers. Final definitions for the categorizations were done in an agreement between two researchers. RESULTS

The results of components of quality of experience are presented in two parts. The main results of the independent studies are firstly described in two groups (based on datacollection method and display technique used). Secondly, components of quality of experience are summarized over the independent studies.

2. Analysis of components of quality of experience in all studies - The data was constructed from the sub-categories including their definitions from all experiments (1-5). These categories formed the concepts of analysis with equal importance between them. The equal importance was considered as the goal of the analysis to identify the general components for quality of experience. In this paper, we use

Quality of experience – study by study

Attribute based descriptions (exp1-3) - Experienced quality is constructed from components of visual quality (depth, spatial, temporal), viewing experience, content,

269

Full Papers

Proceedings: NordiCHI 2010, October 16–20, 2010

audio, and audiovisual quality (Table 2). The results showed similarities between studies, but also some studydependent characteristics. Visual quality, divided into depth, spatial, and temporal dimensions, was the most described component. In all experiments, visual depth quality is strongly characterized by the assessors’ ability to detect depth and its structure to separate the image clearly into foreground and background. Clarity of foreground and background, as well as a natural transition between them was underlined in the descriptions. Furthermore, the erroneous nature of depth was listed (exp1); artificial, strange or unnatural impression was created when the viewers felt that there was too much depth in the scenes, depth was somehow overdone, double images were visible, or when depth was constructed by multiple flat planes or cardboards. Assessor’s attribute illustrates well this failure in depth quality: ‘impression of an artificial world, one would not enter like this in reality’ (male, 26).

Visual spatial quality is expressed in the terms of clarity, color, brightness, impairments of different nature and the assessors’ ability to detect objects. In spatial visual quality, several negative categories were identified and they were all described with the same antonyms (clear, sharp, accurate, visible, error-free). In overall, spatial quality described the line from visibility of objects and overall image clarity to blurriness and finally to the structural imperfections such as blocking impairments and visible pixels. In cases of high video compression levels, the spatial impairments were highly announced in the results (exp2-3). Finally, visual temporal quality summarized the characteristics of motion from general mentions of motion and its fluency to impaired influent and blurry motion. The descriptions of temporal impairments (jerkiness, cut-offs) dominated when the transmission errors were introduced (exp3).

Table 2 Components of quality of experience, their definitions and percentage of participants’ attributes in this category in the experiments 1-3. COMPONENTS (major and sub) VISUAL DEPTH 3D effect in general Excellence of 3D effect Layered 3D Foreground Background VISUAL SPATIAL Clarity Color Brightness Blurry Visible pixels Detection of objects VISUAL TEMPORAL Motion in general Fluent motion Influent motion Blurry motion VIEWING EXPERIENCE Eye strain Ease of viewing Interest in content 3D Added value Overall quality CONTENT AUDIO AUDIOVISUAL

DEFINITION (examples) Descriptions of depth in video General descriptions of a perceived 3D effect and its delectability Artificial, strange, erroneous 3D descriptions(too much depth, flat planes) Depth is described having multiple layers or structure Foreground related descriptions Background related descriptions Descriptions of spatial video quality factors Good spatial quality (clarity, sharpness, accuracy, visibility, error-free) Colors in general, their intensity, hue, and contrast Brightness and contrast Blurry, inaccurate, not sharp Impairments with visible structure (e.g. blockiness, graininess, pixels). Ability to detect details, their edges, outlines Descriptions of temporal video quality factors General descriptions of motion in the content or camera movement Good temporal quality (fluency, dynamic, natural movements) Impairments in temporal quality (cut-offs, stops, jerky motion, judder) Experience of blurred motion under the fast motion User’s high level constructs of experienced quality Feeling of discomfort in the eyes Ease of concentration, focusing on viewing, free from interruptions Interests in viewing content Added value of the 3D effect (advantage over current system, fun, worth of seeing, touchable, involving) Experience of quality as a whole without emphasizing one certain factor Content and content dependent descriptions Mentions of audio and its excellence Audiovisual quality (synchronism and fitness between media). Total number attribute descriptions

270

Exp1 N=15 %

Exp2 N=15 %

Exp3 N=17 %

86.7 66.7 26.7 46.7 33.3

80.0 6.7 33.3 26.7 66.7

58.8 23.5 17.6 35.3

73.3 66.7 26.7 46.7 33.3 73.3

80.0 100.0 80.0 40.0 73.3 80.0

76.5 52.9 17.6 47.1 70.6 47.1

26.7 6.7 20.0

53.3 60.0 40.0 6.7

29.4 52.9 88.2 17.6

20.0 40.0 40.0 53.3

20.0 6.7 13.3 33.3

35.5 52.9 11.8 17.6

20.0 13.3 13.3

40.0 6.7

11.8 17.6 11.8 29.4

130

129

128

Full Papers

Proceedings: NordiCHI 2010, October 16–20, 2010

Table 3 Components of quality of experience, their definitions and percentage of participants’ attributes in this category in the experiments 4-5. COMPONENTS (major and sub, (+) = positive; (-) =negative)

DEFINITION (examples)

VISUAL DEPTH Feeling of 3D (+) Artificial look (-) Fore-background (-) Seeing in two (-) Shadows (-) Color errors (-) See-through objects (-) VISUAL SPATIAL Spatial accuracy (+) Spatial impairments (-) Details VISUAL TEMPORAL Fluent (+) Influent (-) Blurriness of motion (-) Other movement (-) VIEWING EXPERIENCE Followability (+) Pleasant to watch (+) Unpleasant to watch (-) Ease of viewing (+) Difficulty of viewing (-) Need to focus (-) Viewing angle (-) Learning curve Long term viewing (-) Eyestrain (-) No symptoms (+) New experience Comparison to existing Immersion (+) Added interest (+) Enhanced authenticity (+) GENERAL QUALITY Constant quality As a whole Error free (+) AUDIO AUDIOVISUAL CONTENT

Descriptions related to three-dimensional presentation 3D feeling (feeling of space, depth, surroundings) Artificial look of 3D (e.g. scene is too deep) Bad relation between foreground and background Seeing objects in two Shadows around objects Color errors around object borders Ability to see through objects (solid objects become transparent) Spatial factors of video presentation Positive spatial factors (accuracy, clarity, sharpness, good details, good colors) Negative spatial factors (inaccuracy, unclarity, blurriness, bad details, bad colors, blocking, pixels) Details of spatial video in general (e.g. individual details undecipherable) Temporal factors of video presentation Fluent movement Influent movement (jerkiness, stops) Blurriness during motion Other errors of motion (e.g. pixels in motion) Viewer's task, relation to content, and eye-strain Ability to follow the video content, fit to purpose of use Pleasurable to watch, strongly positive experience Unpleasant to watch Ease of viewing Difficulty of viewing Viewing 3D requires extra effort or concentration Limited or narrow viewing angle The existence of a learning curve for watching 3D Not comfortable for long-term viewing Symptoms of eyestrain, headache, sickness Not experiencing any physical symptoms New and previously unrealized experiences General mentions of comparison to existing technology Impression that participants become a part of the events Increased interest in viewing the video Enhanced authenticity (naturality, life-likeness, realism) General quality related descriptions Constant quality during a clip Entire video affects the feeling of quality Quality is expressed to lack the errors (no annoying features) Audio factors (Bad audio) Audiovisual factors (how the audio supports video presentation) Quality is described in relation to any of the contents of experiment Total number of descriptions

Viewing experience described the users’ high-level interpretations of system (media, content) or their influences on the users. These subcomponents are not direct descriptions of representations of stimuli (e.g. colors, visible errors). However, they emphasize the interpretation of stimuli including users’ knowledge, emotions, or attitudes as a part of quality experience. The viewingrelated aspects highlighted the ease of viewing task and interests in viewing. Added value of the 3D was conveyed through the expressions of benefits over current similar system (2D on small screen), increased impressions of

Exp4 N=30 %

Exp5 N=15 %

86.7 50.0 56.7 53.3 70.0 60.0 46.7

93.3 20.0 33.3 33.3 33.3

93.3 96.7 33.3

80.0 46.7

20.0 53.3 63.3 36.7 20.0 36.7 33.3 20.0 20.0 53.3 46.7 36.7 70.0 20.0 40.0 16.7 6.7 46.7

6.7 20.0

6.7 13.3 6.7 33.3 26.7 33.3 20.0 53.3 53.3 33.3 40.0 26.7 40.0

10.0 10.0 33.3 6.7 6.7 100

93.3

778

194

20.0 6.7

involvement and tangible video. The assessors conveyed the impressions of this category in the terms of ‘how strong feelings and situations are transferred to me’ and ‘recognizing the beauty an ordinary image could not communicate’ (male, 23). Furthermore, experienced discomfort was described as a negative part of viewing experience. Finally, the construct of overall quality contained illustrates the total impression of quality. The final three classes highlighted content, audio and audiovisual quality.

271

Full Papers

Proceedings: NordiCHI 2010, October 16–20, 2010

Interview based descriptions (exp4-5) - Quality of experience was formed from several visual quality components, viewing experience, quality factors in general, audio, audiovisual and content classes (Table 3). In overall, visual quality factors and viewing experience were the most commonly mentioned components containing several subcomponents and bipolar affective dimensions.

general quality component summarizes the overall impression of excellence of video and its error-freeness. Quality of Experience for 3D video on mobile device

The model of Descriptive Quality of Experience for 3D video on mobile device (DQoE- mobile 3D video), its components and subcomponents, their definitions and examples of subcomponent-dependent bipolar descriptive terms are described in Table 4.

Visual quality was defined by three subcomponents, depth, spatial and temporal quality similar to experiments 1-3. In the depth quality domain, feeling of 3D, artificiality and foreground-background relation were commonly announced. Such thoughts were described as: ‘[I liked] when it [image] was very clear, so that you didn’t have to put effort into seeing it properly, the whole system. For example, it really bothered me immediately when you couldn’t see the background properly’ (male, 39). In addition, several remarks about disparity errors between the two views were specified (seeing in two, shadows, colorerrors between views, see-through objects) underlining the common impairments of parallax barrier display technology [2]. Spatial quality was strongly defined by the factors of excellence (accuracy, inaccuracy). Finally, temporal quality especially referred to the fluency and blurriness of motion. Such thoughts were described as: ‘unpleasant was perhaps, especially scenes with fast motion were difficult to follow and they looked blurry at times’ (female, 43).

Quality of experience is composed of two main components, quality of visual modality, and viewing experience and two minor components, content, and audio and bimodal quality. Visual, audio and audiovisual quality underlines data-driven features of quality, i.e. directly detectable features and emphasizes characteristics of lowlevel perceptual processes as a part of quality perception. In contrast to the direct perceptual characteristics of media, viewing experience underlines the aspects of high-level perceptual processes where goal-oriented actions, knowledge, emotion and attitude are an essential part of quality interpretation. Visual quality is defined in detail with the aid of 16 subcomponents belonging to the groups of depth, spatial and motion quality. The excellence of depth is summarized as perceivable depth, its naturalness, composition of foreground and background layers, and balance between the qualities of these layers. Spatial quality is constructed from image clarity, block-freeness, colors, brightness, contrast and ability to detect objects and their edges. As a third component of quality factors of visual medium, motion is characterized by fluency, clarity and nature of motion.

Viewing experience described the factors of users’ viewing task and their relation to content as well as self-reported physical symptoms. Several commonly mentioned subcomponents defined the ease of viewing from several slightly different angles; the ability to reach the goals of viewing, a narrow viewing angle, and a need for extra concentration to learn to watch 3D were mentioned. Negative aspects of viewing 3D on a small display were described as eye-strain or by other self-reported uncomfortable feelings: ‘But when the picture quality is bad, then 3D doesn’t work. Then you can’t watch, because your eyes get tired. You really get a feeling that you have to focus and then you get these headaches and whatnot’ (female, 44). In contrast, positive experiences of watching videos in 3D presentation mode were announced as immersion (i.e. the feeling that participants become more mediated to the content) and enhanced authenticity. Finally, 3D on a portable device provided new experiences for the users, but its excellence was also evaluated against existing technology.

The second major component of quality of experience is formed from six subcomponents. Characteristics of viewing task are described by four components, ease and pleasantness of viewing, enhanced immersion and visual discomfort of viewing. Ease of viewing is defined as user’s ability to concentrate on the viewing task without extra effort or learning or interruptions by technology. Pleasantness of viewing represents the affective aspect of viewing in general. Enhanced immersion in viewing experience describes the viewers’ feelings of becoming a part of the events in the content or higher level of involvement, fun or tangibility. The enhanced immersion is also strongly related to positive impressions of 3D viewing. Visual discomfort summarizes the eye strain and other experienced discomfort that viewing possibly can cause on user. The comparisons between new and existing technology is based on user’s memory as defined by the subcomponent of impression of improved technology. Finally, the viewing experience is also characterized by an affective impression of quality as a whole (overall quality). The two minor classes summarize the quality of other related modality and interactions between modalities, and content-dependent quality.

In addition to visual quality and viewing experience, experienced quality was also constructed from quality of audio, audiovisual quality, content, and, finally, general quality components. Similar to the other experiments, audio and audiovisual quality was among the minor classes compared to visual quality. In the interviews, several quality descriptions were content-dependent resulting in a high number of mentions in this category. Furthermore,

272

Full Papers

Proceedings: NordiCHI 2010, October 16–20, 2010

Table 4 Quality of experience for 3D video on mobile device – Components and their definitions COMPONENTS (major and sub) - Bipolar impressions VISUAL QUALITY DEPTH Perceivable depth Perceivable/Not perceivable Impression of depth Natural/Artificial Foreground-background layers Smoothly combined layers/Separate layers Balance of foreground-background quality Balanced/Unbalanced SPATIAL Clarity of image Clear/Blur Block-free image Block-free/Visible blocks Color, brightness and contrast Good/Poor Objects and edges Accurate/Inaccurate MOTION Fluency of motion Fluent/Influent Clarity of motion Clear/Blurry Nature of motion Static/Dynamic VIEWING EXPERIENCE Ease of viewing Easy/Difficult Pleasantness of viewing Pleasant/Unpleasant Enhanced immersion Enhanced/Not enhanced Visual discomfort Experienced/Not experienced Comparison to existing technology Improved/Not improved Overall quality Good/Bad CONTENT OTHER MODALITIES INTERACTIONS Audio Audiovisual

DEFINITION (examples) Descriptions of quality of visual modality, divided into depth, spatial and motion quality Descriptions of depth quality in video, characterized by perceivable depth, its natural impression, composition of foreground and background layers, and balance of their quality Ability to detect depth or variable amount of depth as a part of presentation 3D effect creates a natural, realistic and error-free impression instead of an artificial and erroneous impression (e.g. too much depth, double objects, shadows, seeing through objects) Depth is composed of foreground and background layers and the impression of the transitions between these layers can vary from smooth to distinguishable separate layers Balance between the excellence of foreground and background of image quality (e.g. sharp foreground, blurry background or vice versa, or they are otherwise not in balance) Descriptions of spatial image quality of video, characterized by clarity, block-freeness, colors, brightness, contrast and ability to detect objects and edges Clarity of image in overall -- Clear (synonyms: sharpness, accuracy, visibility) vs. unclear (synonyms: blur, inaccurate, not sharp) Existence of impairments with visible structure in image (e.g. blockiness, graininess, pixels) Excellence of colors, brightness and contrast Ability to detect necessary objects and details, their edges and outlines Descriptions of motion of video, characterized by fluency, clarity and nature of motion Excellence of natural fluency of motion -- Fluent (dynamic, natural) vs. influent (cut-offs, stops, jerky) Excellence of clarity of motion (e.g. accuracy under fast movement or movement out of screen) -- Clear, sharp vs. blurred, pixilated Nature of motion in the content or camera movements - Static (synonym: slow) vs. dynamic (synonym: fast) Descriptions of viewing experience, characterized by ease and pleasantness of viewing, enhanced immersion in it, visual discomfort and impression of improved technology and overall quality Easy