Media in the Ubiquitous Era: Ambient, Social and Gaming Media (Premier Reference Source)

Media in the Ubiquitous Era: Ambient, Social and Gaming Media Artur Lugmayr Tampere University of Technology, Finland Helja Franssila Hypermedia Laboratory, Finland Pertti Näränen Tampere University of Applied Sciences, Finland Olli Sotamaa Tampere University of Technology, Finland Jukka Vanhala Tampere University of Technology, Finland Zhiwen Yu Northwestern Polytechnical Unviersity, China

Senior Editorial Director: Director of Book Publications: Editorial Director: Acquisitions Editor: Development Editor: Production Editor: Typesetters: Print Coordinator: Cover Design:

Kristin Klinger Julia Mosemann Lindsay Johnston Erika Carter Michael Killian Sean Woznicki Lisandro Gonzalez, Milan Vracarich Jamie Snavely Nick Newcomer

Published in the United States of America by Information Science Reference (an imprint of IGI Global) 701 E. Chocolate Avenue Hershey PA 17033 Tel: 717-533-8845 Fax: 717-533-8661 E-mail: [email protected] Web site: http://www.igi-global.com Copyright © 2012 by IGI Global. All rights reserved. No part of this publication may be reproduced, stored or distributed in any form or by any means, electronic or mechanical, including photocopying, without written permission from the publisher. Product or company names used in this set are for identification purposes only. Inclusion of the names of the products or companies does not indicate a claim of ownership by IGI Global of the trademark or registered trademark.

Library of Congress Cataloging-in-Publication Data

Media in the ubiquitous era: ambient, social and gaming media / Artur Lugmayr ... [et al.], editors. p. cm. Includes bibliographical references and index. Summary: “This book focuses on the definition of ambient and ubiquitous media from a cross-disciplinary viewpoint, covering the fields of commerce, science, research affecting citizens”--Provided by publisher. ISBN 978-1-60960-774-6 (hbk.) -- ISBN 978-1-60960-775-3 (ebook) -- ISBN 978-1-60960-776-0 (print & perpetual access) 1. Ubiquitous computing. 2. Ambient intelligence. 3. Communication--Technological innovations. I. Lugmayr, Artur. QA76.5915.M43 2012 004--dc23 2011031143

British Cataloguing in Publication Data A Cataloguing in Publication record for this book is available from the British Library. All work contributed to this book is new, previously-unpublished material. The views expressed in this book are those of the authors, but not necessarily of the publisher.

List of Reviewers Anders Drachen, IT University of Copenhagen, Denmark Janne Paavilainen, University of Tampere, Finland Christian Safran, Graz University of Technology, Austria Conor Linehan, University of Lincoln, UK Teresa Chambel, LaSIGE University of Lisbon, Portugal Thomas Schmieder, University of Applied Sciences Mittwaida, Germany Sanna Malinen, Tampere University of Technology, Finland Andrea Botero, Aalto University, School of Art & Design, Finland Sal Humphreys, University of Adelaide, Australia Hiroshi Tamura, University of Tokyo, Japan Alison Gazzard, University of Bedfordshire, UK Sari Vainikainen, VTT, Finland Stefan Uhlmann, Tampere University of Technology, Finland Ning Li, University of Surrey, UK Jiehan Zhou, University of Oulu, Finland Juan Quemada, Universidad Politecnica de Madrid, Spain Nan Jing, University of Southern California, USA

Table of Contents

Preface...................................................................................................................................................vii Acknowledgment................................................................................................................................... xi Section 1 Consumer Experience, Customer Research, and User Profiling Chapter 1 Analyzing User Behavior in Digital Games............................................................................................ 1 Anders Drachen, Copenhagen Business School, Denmark Alessandro Canossa, IT University of Copenhagen, Denmark Chapter 2 Comparing Two Playability Heuristic Sets with Expert Review Method: A Case Study of Mobile Game Evaluation............................................................................................ 29 Janne Paavilainen, University of Tampere, Finland Hannu Korhonen, Nokia Research Center, Finland Hannamari Saarenpää, University of Tampere, Finland Chapter 3 Lovely Place to Buy! Enhancing Grocery Shopping Experiences with a Human-Centric Approach...................................................................................................................... 53 Hiroshi Tamura, University of Tokyo, Japan Tamami Sugasaka, Fujitsu Laboratories Ltd., Japan Kazuhiro Ueda, University of Tokyo, Japan Chapter 4 Portable Personality and its Personalization Algorithms: An Overview and Directions....................... 66 Stefan Uhlmann, Tampere University of Technology, Finland Artur Lugmayr, Tampere University of Technology, Finland

Section 2 Learning, Training, and Knowledge Sharing Chapter 5 The Integration of Aspects of Geo-Tagging and Microblogging in m-Learning................................... 95 Christian Safran, Graz University of Technology, Austria Victor Manuel Garcia-Barrios, Carinthia University of Applied Sciences (CUAS), Austria Martin Ebner, Graz University of Technology, Austria Chapter 6 Teaching Group Decision Making Skills to Emergency Managers via Digital Games....................... 111 Conor Linehan, University of Lincoln, UK Shaun Lawson, University of Lincoln, UK Mark Doughty, University of Lincoln, UK Ben Kirman, University of Lincoln, UK Nina Haferkamp, University of Muenster, Germany Nicole C. Krämer, University of Duisburg-Essen, Germany Massimiliano Schembri, University of Naples & Institute of Cognitive Sciences and Technologies (ISTC-CNR), Italy Maria Luisa Nigrelli, University of Naples & Institute of Cognitive Sciences and Technologies (ISTC-CNR), Italy Chapter 7 Exploring Semantic Tagging with Tilkut............................................................................................. 130 Sari Vainikainen, VTT Technical Research Centre of Finland, Finland Pirjo Näkki, VTT Technical Research Centre of Finland, Finland Asta Bäck, VTT Technical Research Centre of Finland, Finland Chapter 8 A Knowledge-Based Multimedia Adaptation Management Framework for Ubiquitous Services............................................................................................................................. 149 Ning Li, The Open University, UK Abdelhak Attou, University of Surrey, UK Merat Shahadi, Kings College London, UK Klaus Moessner, University of Surrey, UK Section 3 Novel User-Interfaces, Emerging Forms of Interaction and Media Theories Chapter 9 Interactive Visualization and Exploration of Video Spaces through Colors in Motion....................... 171 Teresa Chambel, University of Lisbon, Portugal João Martinho, University of Lisbon, Portugal

Chapter 10 Issues on Acting in Digital Dramas..................................................................................................... 188 Thomas Schmieder, University of Applied Sciences Mittweida, Germany Robert J. Wierzbicki, University of Applied Sciences Mittweida, Germany Chapter 11 Re-Coding the Algorithm: Purposeful and Appropriated Play............................................................ 200 Alison Gazzard, University of Bedfordshire, UK Section 4 Rising Principles in Virtual Communities, Mediated Social Interaction, and Digital Community Networking Chapter 12 Exploring the Ecosystems and Principles of Community Innovation................................................. 216 Andrea Botero, Aalto University, Finland Kimmo Karhu, Aalto University, Finland Sami Vihavainen, Aalto University, Finland Chapter 13 Supporting Local Connections with Online Communities.................................................................. 235 Sanna Malinen, Tampere University of Technology, Finland Tytti Virjo, Tampere University of Technology, Finland Sari Kujala, Tampere University of Technology, Finland Chapter 14 P2P SCCM: Service-Oriented Community Coordinated Multimedia over P2P and Experience on Multimedia Annotation Service Development............................................................. 251 Jiehan Zhou, University of Oulu, Finland Mika Rautiainen, University of Oulu, Finland Zhonghong Ou , Aalto University, Finland Mika Ylianttila, University of Oulu, Finland Chapter 15 Unraveling Intellectual Property in a Specialist Social Networking Site............................................ 269 Sal Humphreys, University of Adelaide, Australia About the Contributors..................................................................................................................... 288 Index.................................................................................................................................................... 296

vii

Preface

Media in the ubiquitous area is undergoing a tremendous change. MindTrek (http://www.mindtrek.org), the yearly conference in Tampere, Finland, devotes its focus on the latest trends in the wider field of media. As part of its program, MindTrek organizes an academic conference – the Academic MindTrek Conference - attracting a worldwide academic audience. In the years 2008 and 2009, one major focus of the academic part was on research of media in the ubiquitous era. This edited book collects a selected set of extended contributions to both academic conferences discussing the latest trends of social media, ambient media, and digital games.

SOCIAL MEDIA Social media and Web 2.0 are applied in ever more diverse practices both in private and public communities. Traditional communication and expression modalities are challenged and totally new practices are constructed in the collaborative, interactive media space

AMBIENT AND UBIQUITOUS MEDIA “The medium is the message” - This conference track focuses on the definition of ambient and ubiquitous media from a cross-disciplinary viewpoint: ambient media between technology, art, and content. The focus of this track is on applications, theory, art-works, mixed-reality concepts, Web 3.0, and user experiences that make ubiquitous and ambient media tick.

DIGITAL GAMES Digital games and play are currently undergoing many transformations; gaming devices are becoming truly connected, players are finding more possibilities for collaboration, and simultaneously, games are being applied in novel ways and mobile usage contexts. This book is structured into four major sections, each one highlighting another aspect of the latest trends in the field of media:

viii

• • • •

Consumer Experience, Customer Research, and User Profiling; Learning, Training, Knowledge Sharing; Novel User-Interfaces, Emerging Forms of Interaction, and Media Theories; Communities, Mediated Social Interaction, and Digital Community Networking.

Each section compiles a set of chapters discussing issues, research, and cases contributing to this viewpoint. The first section “Consumer Experience, Customer Research, and User Profiling”, has a clear consumer oriented focus and contains the following four chapters: •

•

•

•

Andrea Botero et Al. (Exploring the Ecosystems and Principles of Community Innovation) discuss grassroots culture and the development of media to support and foster innovation within the scope of their chapter. They elaborate what communities drive to develop innovations and how tools can support this process with the help of practical examples. The focus is especially on community driven innovation processes and the implications on the innovation process as such. Janne Paavilainen et Al. (Comparing Two Playability Heuristic Sets with Expert Review Method A Case Study of Mobile Game Evaluation) focus on the evaluation of the user-interface design of digital games. Consumer experience is one crucial factor in the production of games. This chapter devotes in the development of a heuristics that help in user interface evaluation as well as in the playability of game designs. Hiroshi Tamura et Al. (Lovely Place to Buy! – Enhancing Grocery Shopping Experiences with a Human-Centric Approach) present ubiquitous services as a huge business potential for grocery stores, however, also for increasing the shopper’s experience. This chapter devotes especially the issue of exploiting the possibilities of ubiquitous services while shopping. It presents clear guidelines and implications for the development of systems aiding the consumer through their shopping activities. Stefan Uhlmann et Al. (Portable Personality and its Personalization Algorithms: An Overview and Directions) give insights in the increasing amounts of multimedia content requiring techniques to exchange, enrich, and gather information about consumers and their preferences. However, this chapter goes far beyond existing solutions for managing personal profiles. The described concept is based on a digital representation of a consumers’ personality and presents algorithms for advanced to associate personal profiles with multimedia services.

The second section “Learning, Training, and Knowledge Sharing” focuses on applications of digital media in the context of learning and sharing of knowledge. The following chapters contribute to this thematic: •

•

Christian Safran et Al. (The Integration of Aspects of Geo-Tagging and Microblogging in mLearning) emphasize location based services, social media as e.g. Wikis, mobility, and learning as major parts in today’s world of media. The chapter focuses on the development of a mobile micro-blogging platform for educational purposes. The application shall foster learning via geotagging services. Conor Linehan et Al. (Teaching Group Decision Making Skills to Emergency Managers via Digital Games) researches digital games, which can be played for fun, but also emerge in environments for training certain risk groups on specific disaster scenarios. This chapter focuses on

ix

•

•

the training of emergency managers in group decision skills in emergency situations as learning experience. The developed game emphasizes the learning experience in a simulated environment. Sari Vainikainen et Al. (Exploring Semantic Tagging with Tilkut) see collaborative bookmarking and adding metadata to tags as common services in social media. By adding semantic meaning and ontologies to these kind of services, social bookmarking becomes to a powerful tool for knowledge sharing. Within the scope of this chapter consumer studies for enterprise use for a social bookmarking service are presented to gain insights in the requirements for social bookmarking services on enterprise level. Ning Li et Al. (A Knowledge-Based Multimedia Adaptation Management Framework for Ubiquitous Services) discuss that the emergence of more and more multimedia services, devices, and delivery networks require smart mechanisms to adapt content to available resources. The suggested system provides a solution to perform this challenging tasks in a context aware environment to enable interoperability and smart media distribution.

The third section “Novel User-Interfaces, Emerging Forms of Interaction and Media Theories”, emphasize the development of new user experience based on user-interfaces, new forms of interaction, as well as the development of new forms of content types. The following chapters devote to this issue: •

•

•

Teresa Chambel et Al. (Interactive Visualization and Exploration of Video Spaces through Colors in Motion) discuss appropriate techniques for the visualization and exploration of digital spaces as one main problematic with the increasing amount of digital information. This chapter focuses on the description of an application that allows the exploration of videos with a novel designed user interface utilizing advanced visualization techniques for browsing and interacting with large scale video repositories Thomas Schieder et Al. (Issues on Acting in Digital Dramas) focus on creating a theory for the development of a plot for digital games based on theories in acting. Interactivity patters shall support consumers in the development of drama in newly emerging interactive environments such as e.g. iTV. The empathize is on the development of the ‘digital theatre’ based on commonly known theories coming from acting. Alison Gazzard (Re-Coding the Algorithm: Purposeful and Appropriated Play) presents a more media theoretical discussion is the content of this chapter. The chapter discusses various play types of videogames and how games can be explored outside of the intended rules of the actual games. It gives conclusions, theories, and research insights into reality games and their communities.

The last section of the book entitled “Rising Principles in Virtual Communities, Mediated Social Interaction, and Digital Community Networking”, prioritizes social developments that are emerging with the introduction of digital media. The following chapters devote to this thematic: •

Andrea Botero et Al. (Exploring the Ecosystems and Principles of Community Innovation) discuss grassroots culture and the development of media to support and foster innovation within the scope of this chapter. What communities drive to develop innovations and how tools can support this process is discussed on the example of practical examples. The focus is especially on community driven innovation processes and the implications on the innovation process as such.

x

•

•

•

Sanna Malinen et Al. (Supporting Local Connections with Online Communities) research online communities, and how they support social interaction through regional networking is within the scope of this chapter. A survey underlines the importance of locality in the forming process of online communities, maintaining friendships, and the connection of activities performed locally and digitally. A main focus is on the identity of residents and attachment to their local environments and the role this attachment plays in online community forming. Jiehan Zhou et Al. P2P SCCM: Service-oriented Community Coordinated Multimedia over P2P and Experience on Multimedia Annotation Service Development) see that collaborative consumption and tagging of media content became part of today’s world of digital media. Within the scope of this chapter, a technical solution based on P2P technology for annotating content is presented. The presented technical infrastructure allows the creation of multimedia intense web services via converging networks, platforms, and services. Sal Humphreys (Unravelling Intellectual Property in a Specialist Social Networking Site) emphasizes co-creation and cooperation on online social networking site and the impact on intellectual property. IPRs are playing a more and more important rule when various people contribute to each others’ digital works. This chapter discusses the problems around these issues from a legal aspect viewpoint.

With this great number of chapters and different viewpoints on the latest developments in the field of media, we compiled an interesting book that can act as future reference and teaching material. We advise the reader to follow up with the latest trends by following the Academic MindTrek series on www.mindtrek.org, or activities related to ambient media as undertaken by the Ambient Media Association (www.ambientmediaassociation.org) with the Semantic Ambient Media Workshop (SAME) series. Artur Lugmayr Tampere University of Technology, Finland Helja Franssila Hypermedia Laboratory, Finland Pertti Näränen Tampere University of Applied Sciences, Finland Olli Sotamaa Tampere University of Technology, Finland Jukka Vanhala Tampere University of Technology, Finland Zhiwen Yu Northwestern Polytechnical Unviersity, China Tampere, 2011

xi

Acknowledgment

We would like to thank our cooperation partners, sponsors, and local arrangers namely: MindTrek Ry, ACM, City of Tampere, Nokia Oyj, Ubiquitous Computing Cluster, Tampere University of Technology, Tampere University, TAMK University of Applied Sciences, Technology Centre Hermia, Neogames, Digibusiness cluster, Sombiz and COSS The Finnish Centre for Open Source Solutions, PIRAMK University of Applied Sciences, Gemilo, and the Ambient Media Association (www.ambientmediaassociation.org). Artur Lugmayr Tampere University of Technology, Finland Helja Franssila Hypermedia Laboratory, Finland Pertti Näränen Tampere University of Applied Sciences, Finland Olli Sotamaa Tampere University of Technology, Finland Jukka Vanhala Tampere University of Technology, Finland Zhiwen Yu Northwestern Polytechnical Unviersity, China

Section 1

Consumer Experience, Customer Research, and User Profiling

1

Chapter 1

Analyzing User Behavior in Digital Games Anders Drachen Copenhagen Business School, Denmark Alessandro Canossa IT University of Copenhagen, Denmark

ABSTRACT User research in digital game development has in recent years begun to expand from a previous existence on the sideline of development, to a central factor in game production, in recognition that the interaction between user and game is crucial to the perceived user experience. Paralleling this development, the methods and tools available for conducting user research in the industry and academia is changing, with modern methods being adopted from Human-Computer Interaction (HCI). Ubiquitous tracking of player behavior and player-game interaction forms one of the most recent additions to the arsenal of user-research testers in game development and game research. Player behavior instrumentation data can be recorded during all phases of game development, including post-launch, and forms a means for obtaining highly detailed, non-intrusive records of how people play games. Behavioral analysis is a relatively recent adoption to game development and research. However, it is central to understanding how games are being played. In this chapter, the current state-of-the-art of behavior analysis in digital games is reviewed, and a series of case studies are presented that showcase novel approaches of behavior analysis and how this can inform game development during production. The case studies focus on the major commercial game titles Kane & Lynch: Dog Days and Fragile Alliance, both developed by IO Interactive/Square Enix Europe. DOI: 10.4018/978-1-60960-774-6.ch001

Copyright © 2012, IGI Global. Copying or distributing in print or electronic forms without written permission of IGI Global is prohibited.

Analyzing User Behavior in Digital Games

INTRODUCTION Computer games have evolved from simple text-based adventures like Colossal Cave and Akalabeth to virtually photo-realistic renditions of virtual worlds with advanced mechanics, spreading across a dozen or more genres, offering an increasing number of entertainment opportunities (Bateman & Boon, 2005). This development is to no small degree driven by the evolution of gaming devices, the hardware platforms upon which games software is loaded, are also becoming more and more diverse, and thanks to the increasing connectedness of e.g. mobile networks, users are finding digital games accessible everywhere. The increased complexity of digital games – in terms of the amount of possible user actions and –behaviors that they afford, as well as the breath of interaction options between the user and the software/hardware –, the diversity and the distribution across different hardware devices (Lazzaro & Mellon, 2005; Mellon, 2009; Pagulayan, Keeker, Wixon, Romero & Fuller, 2003), are among the important factors driving an increased focus on the users, the players, of digital games, in the game development industry. Contemporaneously with the development in game design, user-research and user-oriented testing has become progressively more important to industrial development and quality assurance (Kim et al., 2008; Pagulayan, Keeker, Wixon, Romero & Fuller, 2003). The purpose of user-oriented game testing is to evaluate how specific components of, or the entirety of, a game is played by people; allowing designers to evaluate whether their ideas and work provides the experience they are designed for. User-oriented testing is useful in game production, because the perceived quality of a digital game product is generally related to the perceived user experience. Therefore, content testing is receiving increasing attention from industry and academia alike (e.g. Isbister & Schaffer, 2008; Jørgensen, 2004; Kim et al., 2008; Nørgaard & Rau, 2007).

2

Methods adopted from Human-Computer Interaction (HCI) (Hilbert & Redish, 1999; Kuniavsky, 2003) have begun to replace the traditional informal testing approaches used in game development and game research, with e.g. usability, playability and user behavior forming keywords in contemporary user-oriented testing and –research (Davis, Steury & Pagulayan, 2005; Isbister & Schaffer, 2008; Medlock, Wixon, Terrano, Romero & Fulton, 2002; Pagulayan, Keeker, Wixon, Romero, Fuller, 2003). Different methodological approaches have different weaknesses and strengths, with e.g. qualitative approaches being excellent for acquiring in-depth feedback from players (users) but requiring substantial resources. In comparison, quantitative approaches are generally better suited for larger participant groups, but less suited for in-depth analysis or study of user behavior and –experience. Given the limited resources of industrial testing, a considerable focus has therefore been aimed towards the quantitative methods. The automated collection and analysis of game metrics data forms one of the new quantitativelybased approaches that have in recent years been adopted from e.g. software development (Renaud & Gray, 2004) to serve in digital game development (Drachen & Canossa, 2009a; Kim et al., 2008; Swain, 2008). Game metrics covers not only player behavior (in-game behavior, player interaction with the different components of the game systems, community behavior, customer service evaluation), but also performance issues (e.g. server stability, monitoring changing features) and processes (turnaround times of new content, blocks to the development pipeline, etc.) (Mellon, 2009; Mellon & DuBose, 2008). Player metrics, a form of instrumentation data, are formed by logs or counts of users interacting with the game software, and is notable for being unobtrusive to collect (Blythe, Overbeeke, Monk & Wright, 2004; Dumas, 2003). The term metric should not be confused with the term heuristic (Desurvire, Caplan & Toth, 2004). Heuristics are

Analyzing User Behavior in Digital Games

design principles which assist in guiding game design, whereas metrics are instrumentation data logged from game software. The player metrics data recorded from digital games, and how they are data mined and analyzed (Kennerly, 2003) varies depending on the stakeholders involved. For examples, at the management level, it is of interest to know what install languages that customers (users) have used, for how long they have played a specific game, how many that completed the game or gave up partway there or for example activity levels on game servers. Community managers can be interested in information about how users interact with the game website, and in being able to provide e.g. heatmaps (Drachen & Canossa, 2009a; Thompson, 2007) or play statistics to the player community. Game researchers can be interested in any type of metrics data, depending on the specific purposes of the research project (Williams, Consalvo, Caplan & Yee, 2009; Williams, Yee & Caplan, 2008). User-research/Quality Assurance experts are conversely interested in the actual behaviors expressed by the players, either during game testing or post-launch. Within the context of user-oriented testing, instrumentation data related to player behavior (user-game interaction) are generally referred to as gameplay metrics (Swain, 2008; Tychsen & Canossa, 2008). Gameplay metrics form objective data on the player-game interaction. Any action the player takes while playing can potentially be measured, from low-level data such as button presses to in-game interaction data on movement, behavior etc. Gameplay metrics data can for example be used to locate design problems or evaluate player behavior patterns (Drachen, Canossa, & Yannakakis, 2009; Kim et al., 2008). Gameplay metrics can be considered similar to User-Initiated Events (UIEs) (Kim et al., 2008), i.e. actions taken by the user, for example moving their game avatar forward, picking up a virtual object, interacting with an AI-controlled entity, or similar. Importantly, UIEs can also be formed by low-level actions such as keystrokes, which

are indirectly relatable with user behavior inside digital game worlds. Since playing a computer game is formed as a series of action-reaction cycles, with the player/-s taking an action, and the game software responding, it is sometimes also necessary to consider Game-Initiated Events (GIEs), i.e. the actions taken by the game software, either independently of the user or as a response to an UIE. For example, if a player shoots at an AI-controlled agent in shooter-type games such as Deus Ex, Quake and Unreal Tournament, the AI-agent will initiate aggressive behavior towards the player. In this chapter, the current state-of-the-art of behavior analysis in digital games is reviewed, and three case studies from the major commercial titles Kane & Lynch: Dog Days [KL2] (IO Interactive) and Fragile Alliance [FA] (IO Interactive) are presented that showcase different novel forms of behavioral analysis performed via the application of gameplay metrics, for example, it is shown how to take advantage of the spatial dimension of behavior data. The case studies presented are specific to two games (covering single-player and multi-player environments), but are focused on generic behaviors such as player death, navigation and skill, which are common to many games or virtual environments featuring player-controlled characters/avatars (e.g. shooter-type games such as: Doom 3, Unreal Tournament, Bioshock and Crysis, and Role-Playing Games/Adventure games such as Oblivion, Neverwinter Nights and Dreamfall. Massively-multiplayer online games such as Age of Conan and World of Warcraft and online persistent worlds such as Second Life, have players taking control of a single avatar/character, and therefore also form potential targets for the behavioral analyses presented. The chapter is in part based on previous work, nominally: “Towards Gameplay Analysis via Gameplay Metrics”, in Proceedings of the 13th International MindTrek Conference © ACM, 2009; DOI: http://doi.acm. org/10.1145/1621841.1621878. This chapter presents new case study material, updated state-of-

3

Analyzing User Behavior in Digital Games

the-art, and new sections on e.g. metrics categories and combined methodological approaches. The case studies are all derived from research carried out in collaboration between the industry and academia, and this is reflected in the case studies being examples of user behavior analysis carried out in practice, with the purpose of evaluating gameplay. The case studies showcase behavior analysis in mid-production phases, expanding on previous work by considering new forms of behavioral data and including multivariate analysis, moving beyond the state-of-the-art. The case studies also focus at the detailed analysis of the behavior of few players, a subject that is virtually nonexistent in the current literature. The case studies are used to build an argument for the usefulness of detailed behavior analysis at the detailed level, in game design-, game production- and game research-oriented contexts. The methods can be directly extended to and applied in the context of, other forms of digital interactive entertainment.

STATE OF THE ART Compared to the extensive literature available on instrumentation-based user behavior analysis in general software development contexts (De Kort & Ijsselsteijn, 2007; Hilbert & Redish, 1999; Hurst, Hudson & Mankoff, 2007), it may be surprising that there is only limited knowledge available on the use of game metrics for development and research. Within the academia, publications are separated into those targeting the analysis of behavior in virtual environments in general (Börner & Penumarthy, 2003; Chittaro & Ieronutti, 2004; Chittaro, Ranon & Ieronutti, 2006), and those targeting digital games applications and user behavior inside the virtual worlds of games (Drachen & Canossa, 2008, 2009a, 2009b; Drachen et al., 2009; Ducheneaut & Moore, 2004; Ducheneaut, Yee, Nickell & Moore, 2006; Hoobler, Humphreys, & Agrawala, 2004; Kim et al., 2008; Southey,

4

Xiao, Holte, Trommelen & Buchanan, 2005; Thawonmas & Iizuka, 2008; Thawonmas, Kashifuji & Chen, 2008; Thurau, Kersting & Bauckhage, 2009). Some studies fall outside of these two categories, e.g. (Burney & Lock, 2007) who used simple player metrics to compare user performance in a planetarium dome with conventional flat-screen environments. Within location-aware games, for example those using mobile phones as the hardware platform, some work has been carried out tracking user location (e.g. Coulton, Bamford, Cheverst & Rashid, 2008), which in principle can be applied to virtual environments. Additionally, Thawonmas, Kashifuji & Chen (2008) evaluated behavior analysis in order to recognize AI-driven bots. Working from a sociological perspective, (Williams, Consalvo, Caplan & Yee, 2009; Williams, Yee & Caplan, 2008) have used game metrics to compare user-report data with actual usage data from the Massively Multi-Player Online RolePlaying Game (MMORPG) EverQuest 2, collaborating with Sony Online Entertainment. The focus of this research has however been focused across the virtual-world real-world divide, looking at e.g. gender roles or comparing reported number of hours per week played with actual number of hours per week played, rather than player behavior inside the game worlds. In work related to user behavior analysis, for example (Nørgaard & Rau, 2007) obtained a game metrics via eye tracking, applying the data to user-oriented testing. Gaze tracking also holds potential as an interaction tool (see e.g. (San Augustin, Mateo, Hansen & Villanueva, 2009). This leads into the field of psycho-physiological measures in game evaluation (Nacke, 2009), which also form high-resolution quantitative data about the users, but about the user experience, not user behavior. The potential merging of these to emergent research directions is discussed below. Within the field of adaptive gaming and AI, related work has been carried out using simple games and a mixture of psycho-physiological

Analyzing User Behavior in Digital Games

measures and user-data (Yannakakis & Hallam, 2007; Yannakakis & Hallam, 2008) or even using outright gameplay metrics in combination with neural network techniques (Drachen et al., 2009). (Southey et al., 2005) used data from the digital game FIFA 99´ to analyze various sweet and hard spots in terms of goal scoring, aiming to semi-automate gameplay analysis, collaborating with the major publisher EA Games. The use of behavioral data for adaptive gaming was highlighted by the major commercial title Left4Dead (2007, Valve), which features an AI-director that utilizes player behavior to control various challenge-features, e.g. the number of enemy zombies. It should be noted that the use of basic tracking of player actions to guide e.g. the responses of AI-agents is a stable of contemporary games (Redding, 2009). In general, a substantial amount of research relevant to behavior analysis in digital games exists in fields such as HCI, visualization and ethnography (Fua, Ward & Rundensteiner, 1999; Hilbert & Redish, 1999; Hurst et al., 2007; Kort, Steen, de Poot, ter Hofte & Mulder, 2005; Kuniavsky, 2003; Peterson, 2004), and it is also from here that most user-oriented methods applied in game development and game research are adopted and adapted (Davis, Steury & Pagaluayan, 2005; Jørgensen, 2004; Kim et al., 2008; Laitinen, 2005, 2006; Medlock, Wixon, Terrano, Romero & Fulton, 2002; Pagulayan & Keeker, 2007; Pagulayan, Keeker, Wixon, Romero & Fuller, 2003). From the industry, publicly available knowledge about the use of game metrics and gameplay metrics specifically, is rare because metrics data and the associated analyses are treated as confidential information, and therefore not publicly available. The information available about industry practices is therefore limited to a handful of conference presentations (King & Chen, 2009; Lazzaro & Mellon, 2005; Ludwig, 2007; Mellon, 2004; Mellon & DuBose, 2008; Mellon & Kazemi, 2008; Romero, 2008), industry whitepapers (Mellon, 2009), blogposts (Grosso, 2009); online popular articles for specialist magazines

and websites (DeRosa, 2007; Goetz, 2006; Kennerly, 2003; Sullivan, 2006; Thompson, 2007) and reports in research articles (Medler, 2009). Game metrics are also mentioned in several game design/ development books. For example, Byrne (2005) discusses player metrics in the sense of the abilities of the player characters, using these as design parameters in level design, for example the walk and run speed, height and width, jump distance and interaction distance of player characters. He discusses how different games feature different metrics, e.g. powerups, temporary modifiers, user-definable metrics, etc. According to Medler (2009), both Microsoft, Maxis and Sony track players using recording systems, and analyze the data using in-house developed analytic tools. However, where some details are known about the approaches of Microsoft (Kim et al., 2008), there is limited knowledge about Sony´s approach. Some development companies chose to partly share the collected data with the player community, for example in the form of diagrams and activity charts. The Steam service provides a macro-level overview of information recorded across different games offered via this service, providing an opportunity for the player community to gain a glimpse of some of the game metrics being collected (see: http://store.steampowered.com/stats/). Similarly, Maxis has allowed the player community access to data collected from Spore, providing the means for user-initiated data analysis (Moskowitz, Shodhan & Twardos, 2009). For games developed using the Flash platform, websites such as mochibot.com and nonoba.com provide analytics tool for developers. There exist a few specialist companies that perform game/gameplay metrics-based analysis for game developers, to greater or lesser degrees, e.g. Orbus Gameworks, Emergent Technologies and Nielsen Games, indicative of the increasing need for metrics-based analyses in the game industry (Mellon, 2009). A foundational aspect of game metrics work is data mining, which forms the core underlying methodology for recording,

5

Analyzing User Behavior in Digital Games

preprocessing, integrating, selecting and transforming data for analysis. Data mining techniques, visual analytics and statistical methods are used to analyze the data, which are ideally presented in a format suitable to the targeted stakeholder (which could be the player community). Importantly, while data mining and analytics can provide knowledge from the collected game metrics data, it cannot provide prioritization of results, e.g. deciding which problems to correct first. Essentially, data mining semi-automates the process a process of knowledge discovery (Han, Kamber & Pei, 2005; Hand, Heikki & Padhraic, 2001; Kennerly, 2003). Focusing on the work carried out on user behavior within virtual worlds (games or otherwise), a substantial part of the current published material stems from the work of Microsoft Game Labs, which perform game testing and user-oriented research for the various Microsoft-based game studios. Microsoft Game Labs developed e.g. the TRUE setup to capture gameplay metrics together with survey and video data, utilizing this during the user testing of e.g. Halo 3 and Shadowrun (Kim et al., 2008; Thompson, 2007). Metrics data have notably been applied in the context of Massively Multiplayer Online Games (MMOGs) and similar persistent, social virtual environments (Isbister & Schaffer, 2008; Lazzaro & Mellon, 2005; Mellon, 2009; Mellon & DuBose, 2008; Mellon & Kazemi, 2008), where they form a source of business intelligence to the development companies. These game forms are highlighted because they provide a continual service over a period of years (for example, the MMOG World of Warcraft has been running since 2003), continually during this period needing information about the users. When reviewing the available knowledge on behavior analysis in games it is evident that the majority is focused on character-based games, i.e. games where the player controls a single character/avatar, which forms the main vehicle for interaction between the player and the game

6

world. For example, the case studies presented by (Drachen & Canossa, 2008, 2009a, 2009b; Drachen, Canossa & Yannakakis, 2009; Hoobler, Humphreys & Agrawala, 2004; Kim et al., 2008) all focus on character-based games. Working with virtual worlds in general, (Chittaro & Ieronutti, 2004; Chittaro, Ranon & Ieronutti, 2006) similarly worked with avatar-based data in the VU-flow application for visualizing movement patterns in virtual environments. From the industry side, (DeRosa, 2007) reported from the use of time-spent reports developed at Bioware for the major commercial game Mass Effect, similarly a character-based game. Given that characterbased games make up if not the majority, then a substantial chunk of the major commercial game titles, this bias is perhaps not surprising. It means however that there is virtually no knowledge about how to perform player behavior analysis in game forms such as Real-Time Strategy (RTS) games, where players control multiple units (generally military); and Turn-Based Games (TBG) such as the Civilization-series. Within the work on character-based games, there is a bias towards multi-player or massively multi-player games forming the main focus of behavioral analysis (Ducheneaut & Moore, 2004; Ducheneaut, Yee, Nickell & Moore, 2006; Kim et al., 2008; Mellon, 2009; Thawonmas & Iizuka, 2008; Thawonmas, Kashifuji & Chen, 2008; Williams, Consalvo, Caplan & Yee, 2009; Williams, Yee & Caplan, 2008), in comparison to work on single-player titles (e.g. DeRosa, 2007; Drachen, Canossa & Yannakakis, 2009; Nørgaard & Rau, 2007; Tychsen & Canossa, 2008). Considering MMOGs specifically, it is evident from reports such as (L. Mellon, 2009) and conference presentations such as (King & Chen, 2009; Lazzaro & Mellon, 2005; Swain, 2008) that these and other forms of “social games” (e.g. Facebook games such as Farmville and Mafia Wars) form a basis for an emergent trend in Metrics-Driven Development, replacing traditional informal approaches; however, the specifics of behavior analysis and data-mining

Analyzing User Behavior in Digital Games

player activity in these games is unavailable beyond the cases mentioned in these reports and presentations. Data-mining player behavior and –activities in MMOGs is common in the industry, because these games require constant monitoring of the player base due to the persistent nature. The few examples of systems developed for capturing game metrics-like data developed outside of the academia are generally targeted at Virtual Environments (VEs) rather than games specifically (Börner & Penumarthy, 2003; Chittaro & Ieronutti, 2004; Chittaro, Ranon & Ieronutti, 2006). Additionally, these applications are often targeted at analyzing specific features, such as movement, or developed for use with specific applications/games [e.g. 14], and therefore not particularly flexible to accommodate the varied needs of behavior analyses across neither game genres, nor portable across application environments. Furthermore, the literature on the use of game metrics (in general, not just for behavior analysis) is largely based on data from in-house testing, with the exception of online/social games, where data are commonly derived from installed clients or game server (Williams, Consalvo, Caplan & Yee, 2009; Williams, Yee & Caplan, 2008) and from character-based games (Drachen, Canossa & Yannakakis, 2009). Published work is also somewhat biased towards high-level aggregate counts of specific metrics, although for example the publication of heatmaps (game level maps showing the locations of player character death aggregated for a number of players) and player statistics (e.g. for purposes of ranking) are becoming more and more common as community feedback tools, e.g. for the game World in Conflict, Half Life 2 and Team Fortress 2. Available behavioral evaluations appear to be generally oriented towards singlevariable analyses, e.g. level completion times, and spatial analyses showing behavior inside the virtual environment of games are rare metrics (e.g. the position of the player within the virtual

environment) (Drachen & Canossa, 2009a, 2009b; Hoobler, Humphreys & Agrawala, 2004). In summary, there is therefore a substantial room for development of novel approaches towards utilizing gameplay metrics for behavior analysis. There is a need to open up the discussion about how to utilize gameplay metrics analysis in game production and –research, and to broaden the available knowledge beyond the predominant MMOGs and MMORPGs. In this chapter, a beginning towards addressing some of these issues is attempted, presenting detailed multivariate case studies of behavioral analysis from two major commercial game titles, both single-player and multi-player.

LOGGING THE RIGHT BEHAVIORS As outlined above, the majority of the published work on behavior analysis in digital games, and the work on VWs, is focused on situations where the user controls a single avatar (or character), in a 3D environment. Typically, the digital games represented are First-Person Shooters (FPS) or Role-Playing Games (RPGs). The games can be either single- or multi-player. Despite this focus on character-based games, behavior analysis would appear to be useful to all forms of games: First of all, the practice of behavior analysis has a strong tradition within software development and website design, where, as noted above, the approach is applied in a variety of situations and contexts. It is therefore likely to expect the method to be equally valuable in a variety of games-related contexts. Secondly, behavioral analysis is already carried out in the game industry on a variety of games, even if the majority of the available published work is restricted to games where the player controls a single avatar or character. Nevertheless, behavior analysis is important to the industry, not just for the purpose of user-research/user-testing, but also in relation to e.g. community feedback. Within

7

Analyzing User Behavior in Digital Games

MMOGs, data-mining player behavior is vital to the industry because of the persistent nature of these games, requiring constant monitoring of the player base (Mellon, 2009). A general problem is that research work and practices carried out in the game industry rarely make it to any publication media, whether professional or academic. What little published knowledge there is emerges in industry conferences, events and –publications. The current published work is generally case-based and often fairly shallow in the analytic depth. The general immatureness of behavior analysis in digital game contexts, and the problems with freely publishing the results of these analyses, which would generally be considered proprietary information by the companies involved, means that it is difficult – at this stage - to provide a consensus about how behavior analysis should be carried out, and to provide guidelines or advice as to which gameplay metrics that it makes sense to track in different kinds of situations. Another barrier for this is that games vary in their design. It is therefore challenging to provide frameworks or guidelines for how to choose which metrics to track, applicable across all games. Digital games vary even within specific genres such as Role-Playing Games, First Person Shooters, Real-Time Strategy Games etc. Due to the variation, and the number of stakeholders that potentially are involved in an analysis (researchers in academia/scientific institutions; and marketing-, management-, community management-, design departments and user-research/game testing experts in the context of development/publishing companies), the questions asked will similarly vary on a case-by-case basis. If the focus is narrowed down to gameplay metrics (UIEs and GIEs) specifically, and thus behavior analysis, ignoring the broader range of game metrics that find use in business intelligence, it is possible to define three broad categories of metrics, as a function of their generality. These categories do not provide specific guidelines about

8

which metrics to track when, but provide an initial step in this direction. 1. Generic gameplay metrics: There are some features that are shared among all games, generally high-level user information such as total playing time, number of times a game has been played, the real-world time that has elapsed from a game was started until it finished, the ratio of players completing the game vs. those who gave up, playing time per level/segment of the game, total number of player actions, etc. These types of gameplay metrics are typically not useful for detailed gameplay analysis, but excellent for aggregated summaries of player activity. This would also be the kind of high-level gameplay metrics that is relevant to crossdepartmental reports. 2. Genre specific gameplay metrics: Although the term “genre” is nebulous at best within a digital game context, it is however widely used in both industry and academia, to describe sets of games that share specific features (Järvinen, 2008). Irrespective of how digital games with shared feature sets are grouped, the advantage in terms of behavior analysis is that these potentially can carry over between games within the group. For example, Drachen & Canossa (2008, future play paper), defined four categories of UIEs and one category of GIEs applicable in character-based games: 1. Navigation metrics: Covers navigation of the character in the game environment 2. Interaction metrics: Covers interactions with objects and entities of the game, initiated by the player via the character. 3. Narrative metrics: Covers navigation through a game storyline, for example quest completion data or navigation through story branches.

Analyzing User Behavior in Digital Games

4. Interface metrics: Covers all interactions with the game interface, either while playing the game or during periods spent interacting with other interfaces of the game (e.g. game setup). 5. Event metrics: Covers all Game Initiated Behaviors (GIEs), e.g. the actions of NPCs, activation of cut-scenes, or similar. 3. Game specific gameplay metrics: These are associated with the unique features of the specific game. In essence, the unique features lead to game-specific questions associated with user-testing. For example, in a game such as Batman: Arkham Asylum, the usage pattern of the main character´s abilities could be of interest, e.g. for evaluating if there are abilities that are over- or under-used. There exists – to the best knowledge of the authors – no published, generic process for identifying which metrics to track in order to address a specific research question or user-test. Which metrics to track generally flows from the question asked, however, how to log the information (as a frequency? Event-based?) can be difficult to determine, notably during the early phases of a game production or research project. Two examples highlight this difficulty: 1. During the development of Fragile Alliance (IO Interactive), the designers wanted to know how long players survived before being killed the first time. This situation shows the ideal situation where the question being asked provides the information necessary to build the analysis to answer it: Survival time can be done via a simple tracking of the timing (in seconds) between each kill event, as a function of a specific player or player tag. 2. The designers were also interested in knowing the ranges at which the different weapons implemented in the game were fired. This question poses a challenge as compared with the first example. Again, the required information is straight

forward (distance between shooter and target); however, how to log and analyze the information requires some consideration. First of all, Fragile Alliance is a team-based shooter game, meaning that literally thousands of shots can be fired in during a game session! If each shot being fired is to be tracked – in detail – a substantial amount of transmission bandwidth (for transmitting the logged data) and storage space would be required. Furthermore, not all shots hit the intended target. It would be relatively safe to assume that a shot hitting a legal target (e.g. one player shooting another on the opposing team), has a high chance of being intended – it could happen that a lucky player trying to hit a legal target by accident hits another legal target, however, it could be assumed (given the mechanics of Fragile Alliance) that the likelihood of this event is small enough to be of minimal influence on a statistical analysis. However, this leaves all shots that do not hit a legal target – for example, shots hitting a wall, car or other game-world object. Should these be included in the analysis? The problem with these shots is that the objects being hit are likely not the indented targets (unless the object in question is destructible and there is a point doing this). Including these in the analysis will therefore not inform about the ranges with which players intend to use specific weapons. Using probability analysis, it is possible to estimate the intended target of a spread of bullets from e.g. a submachine gun; however, this type of evaluation is computationally too cumbersome to employ in the context. In the current situation, the approach chosen was to record the position of the shooter and the legal target being hit, and the weapon type used, for each instance where the shot resulted in a death event only. This provides a compromise between obtaining all data and not overloading the bandwidth requirements. The approach works excellently for Fragile Alliance, where players generally use the same weapon to bring opponents down, but may be less appropriate for games where multiple different weapons are used to first wound and then kill opponents (e.g. using first a rocket launcher at long range, then a 9

Analyzing User Behavior in Digital Games

shotgun at close range, in Team Fortress 2). Using simple triangulation, it is based on the recorded data possible to calculate the mean range that each weapon is used, as well as useful parameters such as standard deviation. Using an associated time stamp, the temporal development in the selected variables can also be evaluated.

CASE STUDIES The case studies presented below form practical examples of behavior analysis in a digital game context. In the first case study, data from a series of playtesting sessions are included, in the second; the focus is on detailed analysis of the gameplay experience of just a few players. The case studies are focused on features such as death, navigation and environment interaction that are generic to character-based games. The approaches described should therefore be broadly applicable to these types of games, which arguably form the majority of major commercial titles along with games from the Real-Time Strategy (RTS) genre; as well as avatar-based social environments featuring 3D environments, e.g. Second Life. Data for the case studies presented are drawn from the Square Enix Europe (SEE) Metrics Suite, developed by the SEE Online Development Team hosted by IO Interactive. The Suite is engineered towards collecting game metrics from SEE produced games, both in-house as well as from various online services, such as the Xbox Live! Service. When performing spatial analyses on player behavior data, preprocessed data are imported into a geodatabase system. From this, data are extracted, plotted, analyzed and visualized using either a Geographical Information System (Longley, Goodchild, Macquire & Rhind, 2005) or a custom tool developed at IO Interactive, QuickMetrics. The GIS permits in-depth analysis of spatial metrics, where QuickMetrics is suited for rapid visualization of basic event-driven variables or player navigation data.

10

It is important to note that the gameplay metrics data used in the case studies are drawn from inhouse testing during mid-development of KL2 and FA. This causes two problems: 1) Some absolute numbers are confidential and cannot be reported (percentages are used instead); 2) The data are not obtained from user-research sessions where the experimental conditions are controlled. The data were recorded during playtesting sessions run at the user-research department of IO Interactive. The lack of controlled conditions means that there is a risk of bias in the data – i.e. that testers played differently than they normally would. However, given that a controlled laboratory setup would also present players with a different playing environment than that which they normally operate in, it is difficult to avoid this assumption. The exception is studies using remotely connected playdata from users playing in their native habitats (Drachen & Canossa, 2009b; Williams, Consalvo, Caplan & Yee, 2009; Williams, Yee & Caplan, 2008). However, this was not possible in the current context because KL2 and FA are not published games. It should be noted that studies using data from in-house playtesting are fairly common in the literature, as it forms one of two primary data sources for games-based user testing (Kim et al., 2008), with the other being remotely collected data from game servers, e.g. MMOG-servers (Drachen & Canossa, 2009b; Drachen, Canossa & Yannakakis, 2009; N. Ducheneaut & Moore, 2004; Ducheneaut, Yee & Moore, 2006; Williams, Consalvo, Caplan & Yee, 2009; Williams, Yee & Caplan, 2008).

Case Study 1: Level Analysis by Sub-Sector in Fragile Alliance In the first case study the focus is on showcasing how in-depth player behavior analysis operates in practice during mid-late development, when a vertical slice is playable but undergoing iterative user-testing in order to ensure the design works as intended (Isbister & Schaffer, 2008; Medlock,

Analyzing User Behavior in Digital Games

Figure 1. Screenshots from Fragile Alliance. (Top) Showing a traitor clearly marked to the remaining mercenary players. (Bottom) Teams of mercenaries and police engage (© 2008, IO Interactive. Used with permission).

Wion, Terrano, Romero and Fullerton, 2002; Pagulayan, Keeker, Wixon, Romero & Fuller, 2003). The setting is a development company but could just as easily be an academic research group. Fragile Alliance is a multi-player, online shooter-type game (Figure 1). The players either play as mercenaries trying to accomplish a specific mission, such as a bank robbery; or as police officers trying to prevent this. However, all players start as mercenaries, with the police team being comprised of AI agents. If a mercenary dies, they respawn (are reinstated in the game universe) as police officers, working alongside the AI agents. Apart from the risk of being killed by the police, mercenary players also face the risk of being betrayed by other mercenaries. In Fragile Alliance, mercenaries can betray each other, and steal each

other’s loot. If for example a mercenary player had managed to secure a sum of money from a bank vault, another mercenary could kill the first, and steal his/her money. If a mercenary kills another mercenary he becomes a “traitor” but is allowed to keep all the money collected without sharing (Figure 1). This game mechanics is designed to shift the balance of power from initially being on the side of mercenary team, towards the police (AI and players), as more and more mercenaries are eliminated. After the second death, the player will typically not respawn, but will have to wait for the game round to end (usually after a few hundred seconds depending on the map). A game session will typically consist of multiple rounds being placed on the same map (scenario) and/or different maps, similar to comparable games such

11

Analyzing User Behavior in Digital Games

Figure 2. Basic kill statistics for Fragile Alliance, based on 8943 kill events from a series of game sessions including 129 different players. (Top) Distribution of kills by AI-agents or players. (Bottom) Causes of death by killer role.

as Unreal Tournament and Counter-Strike. The winner of a round is the player who leaves with the most money, irrespective of how these are obtained. Police players earn money by securing money from mercenaries. During development of Fragile Alliance, the designers of the game were interested in evaluating if this shift in balance manifested in the actual behavior of the players. A series of playtests (over 100) were run using vertical slices of the game (a specific level map), including 129 players (in-house and external testers). Data from the game sessions were extracted from the SEE Metrics Suite. The dataset included 8943 kill events, with data about the role of the player being killed, who the killer was, the position of both players, the type of weapon used and a timestamp for the 12

event metric. A time-series approach was adopted, with data binned into 15 segment sections, and percentage distributions between the different killer roles calculated. The result showed that the majority of the kills in the game were caused by mercenaries, up to approximately 75-90 seconds of play. For example, from 30-45 seconds, 48% of all kills were caused by mercenaries, and only 35% by the police (of which 27% were by AIagents). After 90 seconds of play, the pattern changes. Collectedly after the 90 second mark, mercenaries account for 35% of the kills, while the police team caused 55% of the kills (8% of which were kills by AI-agents). The remaining percentages were taken up by specialist roles such as Undercover Cops and Dirty Cops, who are available in specific scenarios. Traitor mercenar-

Analyzing User Behavior in Digital Games

ies generally did not figure as a major influence in the kill distribution, generally causing 2-7% of the kills. Collectedly, players caused 70% of the kills, the AI-agents 26% - enough to be felt but not enough to wrest control from the players. The remaining 4% were suicides, e.g. players standing too close to exploding object such as cars (Figure 2). Analyzing kill statistics temporally does not address the spatial patterning of the events. Given the focus of Fragile Alliance on scenariobased levels, i.e. the player teams have specific missions beyond elimination of the opposing force, it is essential to ensure that players progress through the level, so that the right events occur in the right locations. For example, in the vertical slice used in the current case study (Figure 3), the objective for the mercenary players to reach the exit and complete the mission. In at least some games, it would therefore be desirable to have the mercenaries reach the exist area. Spatial analysis of gameplay metrics is a powerful tool for analyzing player behavior, as the can be plotted with pinpoint accuracy. This allows fine tuning of gameplay balance. In the vertical slice used here, a specific level from Fragile Alliance, the mercenaries spawn in the bottom of the map, the police AI agents to the top right (Figure 3). The objectives of the mercenaries is firstly to enter a vault, located to the left in the map, and thereafter to reach the level exist, in the top right corner, behind the spawning area of the police. The game level consists of four major sub-sectors: The spawning area, where the mercenary players enter the game (red in Figure 2). The vault area, where the money they need to steal are located (green in Figure 2), a subway station area approximately in the middle between the spawning areas of the two teams (yellow in Figure 2) and finally an area at street level (orange in Figure 2), through the rightmost side of which the mercenary players must go through if they want to escape (Figure 1, bottom). Combining visualization of the spatial behavior of players with statistics of their temporal (and

spatial) behavior permits a more in-depth analysis of the player behavior (Figure 4). Comparing the spatial and temporal behavior shows for example that mercenary players generally turn traitor either in the beginning of the game in the spawning area sub-sector, or later in the game in the road/exit area. Traitors are typically killed in the spawn area (61.25%), but rarely in the road/exit area (8.81%), which indicates that it is a much more risk-filled endeavor to turn traitor early in the game rather than later (it should be noted that further analysis showed that mercenaries turning traitor outside of the spawning area rarely move into the spawning area again – by this point the action has moved to the other segments of the map). For the mercenaries, the majority of the kills occur in the spawning area sub-sector, where mercenaries enter the game (Figure 4). The AIagent kills are spread across the entire map, indicating that their search & destroy behavior is working. Suicides occur in the vast majority of cases (76.04%) in the road/exit area, where a series of cars are placed which can explode if taking too much damage. A smaller part takes place in the metro station area, where players can be hit by metro trains while crossing the tracks coming from the vault to the exit/road area to the north in the map (Figure 3). The analysis resulted in designers adding warning noises to the subway trains and increasing the health of cars, in order to bring down the number of deaths caused by these two game features. In terms of the roles played by players when they are killed, the pattern is generally as was intended in the game design. Police (players and AI) are generally killed in the road/exit area where they spawn (69.32%), and very few are killed in the spawning- and vault areas, where instead the mercenaries are under pressure from the police (44.17%). A somewhat larger amount of death events occur in the spawn area than intended by the game design, which could indicate that mercenaries are perhaps a bit too eager to turn traitor early in the game. This could be a gameplay problem, how13

Analyzing User Behavior in Digital Games

Figure 3. The Fragile Alliance vertical slice (game level) divided into sub-sections: Bottom area = spawning area; Middle area = subway; Left area = vault area; Top area = road/exit area. (a) Locations where police officers were the cause of death events. A broad distribution is apparent indicating that police officers can reach the entire map. (b) Locations of suicides. (c) An example of feedback to the game designers of Fragile Alliance. The level map shows the distribution of about 250 player death occurrences overlain the level map, and has added explanations to guide the interpretation of the map. The map is developed using a GIS.

14

Analyzing User Behavior in Digital Games

Figure 4. (Top) Frequency distribution of the causes of player death as a function of the sub-sector. (Bottom) Frequency distribution of the player role at the time of death as a function of the sub-sector. Same data source as for Figure 1.

ever, it may not necessarily be a user experience problem – the players may find great enjoyment in following this behavior. In order to properly address this question, user experience measures need to be employed (see below). As the game level is iteratively refined and tested, different solutions can be attempted. The approach to analyzing player behavior in the above example is based on Fragile Alliance alone; however, the game is representative of a large number of multi-player shooter games, e.g.

Call of Duty, Team Fortress 2 and the Battlefieldseries, which have proliferated notably since the release of Counter-Strike. The methodology outlined is therefore directly transferrable to these games, with maybe minor modifications made to the measures depending on the “culture” of weapon use in the specific games, as noted above. In other game forms, e.g. single-player games (even non-shooters), causes of death may not be other players and AI-agents, but perhaps environmental effects. The principle of the analy-

15

Analyzing User Behavior in Digital Games

sis remains identical, however, and the interest in locating “trouble spots” in game levels/areas where the behavioral patterns are not as intended is also common in user-oriented testing in game development (Drachen & Canossa, 2009a; Isbister & Schaffer, 2008; Kim et al., 2008).

Case Study 2: Frustration in Kane & Lynch: Dog Days Kane & Lynch: Dog Days is a shooter-type game currently in development at IO Interactive. In terms of gameplay, the game follows previous shooters in that the player controls a single character and mainly has to worry about staying alive, eliminate enemies and solve specific tasks. In this case study, the game experience of a single player is investigated in detail with a focus on investigating the causes of frustration exhibited by the player. Frustration would normally be characterized as an unwanted component of user experience (Brave, 2003; Gilleade, 2004; Klein, 2002; Norman, 1988); however, frustration forms a recognized component of the experience people can obtain from playing digital games (Hazlett, 2006; Ijsselsteijn, 2007; Pagulayan, keeker, Wixon, Romero & Fuller, 2003; A. Tychsen, Newman, Brolund, & Hitchens, 2007; Vorderer, 2006). The case study showcases the potential of behavioural data to enable modelling of the navigation of players through a game environment. This is of key interest to game designers because it allows them to observe how their games are being played. User-oriented methods such as playability testing (Davis, Steury & pagulayan, 2005; Pagulayan, Keker, Wixon, Romero & Fuller, 2003) can also locate gameplay problems, however, when integrating gameplay metrics in a data collection suite, it becomes possible to model the second-by-second behaviour of one to – simultaneously - thousands of players. There are many ways to visualize navigational data. There exists various applications for handling data visualizations, which are flexible enough to handle a variety of contexts. In the current case,

16

Figure 5. Gameplay metrics data plotted directly in the game editor. The position of the player was tracked every second (points). Associated metrics tracked included the camera angle at the time (light grey cones), and whether the player character was taking damage (darker grey color). The green line represents the path of the player, calculated as vectors between the tracked positions.

a Geographic Information System (GIS), build using the package ArcGIS, was created to provide spatial analysis (Drachen & Canossa, 2009a; Longley, Goodchild, Macquire & Rhind, 2005). Another possibility is to visualize the data directly in the game editor (Figure 5). This form of visualization allows experimenters to see through the eyes of the player in a manner similar to a video

Analyzing User Behavior in Digital Games

Figure 6. Top-down view of the level showing start position, end position and all checkpoints (purple hexagons). The last checkpoint, that was malfunctioning, has been highlighted.

recording of a game session, but with the added benefit of having recorded metrics mapped within the game environment, and draw on quantitative results from these. Similar to the first case study, the type of analysis reported here is placed in mid-late development, where at least a vertical slice of a game in production is available. In the first case study, the research questions driving the analysis were pre-defined. This case study is an example of how questions can arise via user-testing in an industrial development (or empirical research) context: The study was made possible due to a

serendipitous series of events: During play-test sessions of the KL2, the game´s programmers delivered a version of the game in which a checkpoint malfunctioned forcing players to repeat a fairly long and challenging segment of play within a specific game level (Figure 6). During the play-test sessions, a user research expert at IO Interactive observed a test participant, who considered himself fairly proficient, become more and more angered as he failed to complete a level of the game, dying several times in the same area of the game level, and having to restart at an earlier point in the game as compared to where

17

Analyzing User Behavior in Digital Games

he died. The participant manifested frustration through body movements, facial expressions and verbalizations. Following the play-test, the user-research experts at IO Interactive wanted to discover if it was possible to recognize, in the patterns of player behaviour captured as gameplay metrics, feelings of frustration. Furthermore, which patterns of interaction and navigation in the game that point towards a state of frustration in the player, and whether these symptoms can be observed in different players. There are different theories of frustration (Amsel, 1990; Rosenzweig, 1944), outlining different types of frustration, for example failure to understand goals, failure to communicate means available to achieve goals and repeated failure to overcome challenges. For the case study, frustration was defined using the following definition: Repeated failure to overcome challenges. This definition formed a compromise between the nature of frustration and the limitations of userinstrumentation data (which cannot show what users feel, only provide indication based on defined guidelines). The malfunctioning checkpoint exasperated the situation because every failure was further punished with a lengthy navigation of the same environment and facing the same challenges without any sort of achievement or feeling of progression. In order to be able to confirm whether any frustration-detecting pattern was functional, the user research experts did not communicate to the gameplay metrics analysts where the participant manifested frustration. The gameplay metrics recorded during the play-test were given to the analysts with the mandate to individuate reoccurring patterns that could be symptomatic of frustration. The first instance of behavioural pattern that could indicate a point where the player felt frustration (Figure 7), included several indicators: 1) First of all the player died in the same location four consecutive times, actually regressing in the

18

second, third and fourth attempt (Figure 7). 2) Secondly, the number of enemies killed decreased considerably in each play trough. 3) Thirdly, the pace of the player becomes considerably faster in each play through, as displayed by the spacing of the small dots, and repeats the same route with no variation (third and fourth death). 4) Also lacking is the presence of special events such as triggering environment explosions or picking up weapons dropped by enemies. The fourth attempt proved to be the most unsuccessful, lasting only few seconds, showing the play-tester rushing into the enemies and failing almost instantly. After this failure the player appears to regain control, slowing the pace of movement, attempting a new route (leftward turn), killing a considerable amount of enemies and taking the time to pick up dropped weapons (Figure 7). The analysis of the behavioural data were then correlated with the video recording of the playtest, and discussed with the user-research expert who ran the play-test. At the time indicated by the behaviour analyst, the player was evidenced to display signs of frustration, such as irritability, vocalized discontent and a certain blushing. A second set of data were employed to see if the same behavioural pattern occurred later in the play-test (Figure 8). In this situation (Figure 8), the first attempt should have triggered the malfunctioning checkpoint, which was not working. Following the first instance of death, when the player attempted the challenge a second time, the player is also performing well, displaying proficiency and interest even in sub-tasks that are not vital to survival and game progress, such as searching fallen enemies for ammunition and weapons. In comparison, 3rd to 6th attempts to progress through the game level display a similar lack of progress, and series of death events happening more and more rapidly. The player increases pace without paying attention to secondary tasks and kills fewer and fewer

Analyzing User Behavior in Digital Games

Figure 7. Player paths and in-game events in Kane & Lynch 2: Dog Days, expressed via recorded behaviour data. The images show, from top to bottom, the path of a play-tester and specific events that occurred during the test (e.g. player getting wounded). Each image shows the time from one instance of player death to the next, showing decreasingly less progress in the game from death 1-4; indicative of a behavioral pattern pointing towards player frustration.

enemies with each attempt. The four elements individuated earlier are present, to an even stronger degree: 1) The player dies in the same location, sometimes actually regressing; 2) The number of enemies killed decreases considerably; 3) The pace of the player becomes considerably faster, repeating the same route with limited or no variation; 4) The player does not give attention to non-vital, secondary tasks such as triggering environment explosions or picking up weapons. Similarly to the first example, the video recordings from the playtest were examined with the user-research expert running the test. Similar vocal and body-language responses showing frustration were found in the play-tester from death event 3-6.

It should be noted that the case study represents a very small sample of participants and a specific game, and the results are therefore not generalizable. Also, the patterns identified are only applicable when frustration is defined as being failure to overcome challenges. Other forms of frustration were not considered. The case study serves to highlight the usefulness of behavioural data to solve problems that arise during user-testing games, as well as during empirical games research. Furthermore, it shows how behavioural analysis can support the practice of placing check points in games such as KL2 using the experience and gut-instinct of designers. The four elements of frustration individuated in this case study regard a single player in a single game level, it will be

19

Analyzing User Behavior in Digital Games

Figure 8. Player paths and in-game events in Kane & Lynch 2: Dog Days, expressed via recorded behaviour data. The images show, from top to bottom, the path of a play-tester and specific events that occurred during the test (e.g. player getting wounded). Each image show the time from one instance of player death to the next, showing progressively less progress in the game from death 3-6; indicative of a behavioural pattern pointing towards player frustration.

vital in the future to verify the presence of these patterns in data gathered from the same player but in different levels, from different players and maybe from different games. If the hypothesis can be confirmed, it could be possible to identify universal markers enabling the automatic detection

20

of frustration problems during instrumentationbased evaluation of play experience.

Analyzing User Behavior in Digital Games

BEHAVIOR AND USER EXPERIENCE While behavioral analysis addresses specific questions in game development and –research, such as game-space usage analytics, player interaction and player navigation etc., there is an second usage of behavioral data, namely in the combination with user experience data to provide a linkage between game design and play experience (Isbister & Schaffer, 2008; Nacke, 2009; Romero, 2008). In essence, gameplay metrics provide opportunity to link fine-grained behavioral information (finer than any other method, baring detailed video analysis) with user experience data. It should also be noted that order to enable metrics-based analysis, an infrastructure is needed to capture the data, which can mean substantial storage needs in the case of large commercial titles. Gameplay metrics provide information only regarding actions undertaken by players, it is usually not possible to assess reasons and motivations behind the action, unless additional user data are captured (Drachen & Canossa, 2008, 2009b; Isbister & Schaffer, 2008; Lazzaro & Mellon, 2005). Gameplay metrics do not inform whether the player is male or female, or what the player thinks of the game experience. In short, gameplay metrics cannot provide any contextual data, although drawing inferences about the reasons for observed behaviors may be possible. Towards this end, Lazzaro & Mellon (2005) proposed the use of “fun meters” in games, essentially the collection of metrics of user behavior that are indicative of whether or not the players are enjoying the gaming experience. The essence of their argument is that behavior indicates the kind of experience the player is having (with the additional social factor). For example, looking at what people spend their money on in The Sims Online as an indicator of what entertains them in the game. Lazzaro & Mellon (2005) noted that many features in games affect enjoyment, and that each of these needs a meter (measure). In turn, these meters require a data source, which relates to the overarching

question being asked. Extracting the data for the measures can be difficult, with basically two ways possible: Indirect (asking players) and direct (observing players). The authors highlight the added strength that correlation between data sources, whether all quantitative or mixed quantitative/ qualitative, brings to an analysis. User experience data in game testing is generally obtained using qualitative or semi-quantitative methods, such as user feedback (e.g. attitudinal data) via interviews or surveys, or potentially in combination with usability-oriented testing (Isbister & Schaffer, 2008; Laitinen, 2005, 2006; Romero, 2008). Usability testing generally focuses on measuring the ease of operation of a game, while playability testing explores is users have a good playing experience. In comparison, gameplay metrics analysis offers however insights into how the users are actually playing the games being tested. Kim et al. (2008) and Romero (2008) presented the TRUE-solution of Microsoft Game User Research Labs, which in brief is a system capable of recording screen capture, video footage, behavioral and survey-data in one coherent framework. The TRUE system uses e.g. small pop-up surveys that activate during timed intervals to quickly assess the user experience of the player, recording simultaneously the position of the player character in the game environment (Kim et al., 2008; Romero, 2008). The problem with this approach is that the interaction flow between player and game is interrupted, and furthermore that the evaluation of the user experience is limited to one or a few dimensions, as surveys need to be kept short to keep the interruption to interaction flow to a minimum. A promising approach, combining metrics with psycho-physiological methods (Cacioppo, Tassinary, & Berntson, 2007), has not been attempted yet – at least no published studies are known to the authors. It is however not unfeasible that this will occur in the near future, given the development of commercially viable EEG and EMG devices,

21

Analyzing User Behavior in Digital Games

and the proliferation of psycho-physiological studies of game experience (Mandryk & Inkpen, 2004; Nacke, 2009; Ravaja, Saari, Laarni, & Kallinen, 2006). Projects are already under way, e.g. in Canada and Scandinavia, both places as collaborations between companies and research institutions. The results remain to be seen, but in theory combining these two high-detail methods should be useful to correlate specific behaviors with the perceived user experience.

CONCLUSION AND PERSPECTIVES In this chapter, the state-of-the-art of player (user) behavior analysis in digital games has been reviewed, and the current trends in industry and academia outlined. Case studies from the two major commercial titles Kane & Lynch: Dog Days and Fragile Alliance, have been presented which showcase how gameplay metrics can be employed to inform behavior analysis in practice (whether in a research or development context). The focus has been on detailed, multivariate work on data from just a few players, operating inside the virtual worlds themselves, taking advantage of the spatial dimension of play. This is a kind of analysis on which there exists virtually no published material. The case studies are based on common features of shooter-type games: Spatial navigation, environment interaction and death events; and are therefore cross-applicable across character-based games. The case studies indicate the usefulness of behavior analysis via gameplay metrics to recreate the play experience and thereby evaluate game design. For example, evaluating challenge and reward systems, locate areas that are over-/under-used, check for areas where players find navigation difficult, and importantly if the players operate as intended by the game design. Behavior analysis via gameplay metrics addresses one of the major challenges to gamesoriented user research, namely that of tracking and analyzing user behavior when interacting with

22

contemporary computer games. As a user-oriented approach, it complements existing methods, providing detailed, quantitative data from – potentially – very large player samples, to supplement qualitative or semi-quantitative data from e.g. playability- and usability testing (Isbister & Schaffer, 2008; Kim et al., 2008; Lazzaro & Mellon, 2005). Additionally, Mellon (2009) highlighted Metrics-Driven Development as a method for utilizing instrumentation data to drive development and design of computer games, focusing on quantifying problems, thus rendering them measureable based on user-instrumentation data. The field of player behavior analysis remains in its relative infancy, with methodologies lagging behind the general software industry, despite an emergent interest in the industry, and e.g. online games having access to a comparatively broader variety of user measures. One of the primary barriers for the uptake of game metrics in the industry appears to be cultural. Firstly, because metricsbased analysis is a relatively new addition to the industry, developers are reluctant to invest funds in the area. Secondly, metrics do not add to the features of a game. As noted by Mellon (2009): “The biggest roadblock to our industry reaping the rewards of metrics is in fact our own business practices: if a task does not directly “put pixels on the screen”, it is not a game feature and thus it is at the bottom of the funding list.” This leads to situations where the tools that do get build for tracking and logging game metrics are often build by individual systems engineers for their own needs, and thus do not get passed on within companies. A similar affect is present in the academia, where e.g. tools build by PhD-students do not get applied following their graduation. The exceptions to this rule is companies such Square Enix Europe, who has invested the time and resources to build team-wide tools for capturing business intelligence data, including behavioral metrics. Similar tools have been reported for companies developing and running MMOGs, e.g. Lord of the Rings Online (Mellon & Kazemi, 2008). In

Analyzing User Behavior in Digital Games

the academia, the resources necessary to obtain game metrics data has acted as a barrier for research, although inroads are being made thanks to collaborations with game companies (Drachen & Canossa, 2009a, 2009b; Thawonmas & Iizuka, 2008; Thawonmas, Kashifuji & Chen, 2008).

Brave, S., & Nass, C. (2003). Emotion in HumanComputer Interaction. In The Human-Computer Interaction Handbook: fundamentals, evolving technologies and emerging applications (pp. 82–93). Mahwah: Lawrence Erlbaum Associates, Inc.

ACKNOWLEDGMENT

Burney, T., & Lock, P. (2007). Measuring GamePLay Performance and Perceived Immersion in a Domed Planetarium Projection Environment. Paper presented at the ICEC

This would not have been possible without the stellar work of the Square Enix Europe Online Development Team. Also sincere thanks to the many other colleagues at IO Interactive, Square Enix Europe and Crystal Dynamics. This work is based on an earlier work: Towards Gameplay analysis via gameplay metrics, in Proceedings of the 13th MindTrek Conference (ISBN: 978-1-60558-633-5) © ACM, 2009. http:// doi.acm.org/10.1145/1621841.1621878

REFERENCES Amsel, A. (1990). Arousal, suppression, and persistence: Frustration theory, attention, and its disorders. Cognition and Emotion, 43(3), 239–268. doi:10.1080/02699939008410798 Bateman, C., & Boon, R. (2005). 21st Century Game Design. Charles River Media. Blythe, M. A., Overbeeke, K., Monk, A. F., & Wright, P. C. (2004). Funology - from usability to enjoyment (1st ed., Vol. 3). Dordrecht: Kluwer Academic Publishers. Börner, K., & Penumarthy, S. (2003). Social Diffusion Patterns in Three-Dimensional Virtual Worlds. Information Visualization, 2, 182–198. doi:10.1057/palgrave.ivs.9500050

Byrne, E. (2005). Game Level Design. Charles River Media/Cengage Learning. Cacioppo, J. T., Tassinary, L. G., & Berntson, G. G. (2007). Handbook of Psychophysiology (3rd ed.). Cambridge University Press. Chittaro, L., & Ieronutti, L. (2004). A visual tool for tracing users´ behavior in virtual environments. Paper presented at the Working Conference on Advanced Visual Interfaces. Chittaro, L., Ranon, R., & Ieronutti, L. (2006). VU-Flow: A Visualization Tool for Analyzing Navigation in Virtual Environments. IEEE Transactions on Visualization and Computer Graphics, 12(6), 1475–1485. doi:10.1109/TVCG.2006.109 Coulton, P., Bamford, W., Cheverst, K., & Rashid, O. (2008). 3D Space-Time Visualization of Player Behavior in Pervasive Location-Based Games. International Journal of Computer Games Technology, 2008 Article ID 192153, 5. Davis, J., Steury, K., & Pagulayan, R. (2005). A survey method for assessing perceptions of a game: The consumer playtest in game design. Game Studies; The International Journal of Computer Game Research, 5. De Kort, Y., & Ijsselsteijn, W. (2007). People, places and play: Player experience in a sociospatial context. ACM Computers in Entertainment, 6(2).

23

Analyzing User Behavior in Digital Games

DeRosa, P. (2007). Tracking Player Feedback to Improve Game Design. Gamasutra. Desurvire, H., Caplan, M., & Toth, J. A. (2004). Using heuristics to evaluate the playability of games. Paper presented at the Computer-Human Interaction (CHI) Extended Abstracts. Drachen, A., & Canossa, A. (2008). Defining Personas in Games Using Metrics. Paper presented at the Future Play. Drachen, A., & Canossa, A. (2009a). Analyzing Spatial User Behavior in Computer Games using Geographic Information Systems. Paper presented at the 13th MindTrek. Drachen, A., & Canossa, A. (2009b). Towards Gameplay Analysis via Gameplay Metrics. Paper presented at the 13th MindTrek Drachen, A., Canossa, A., & Yannakakis, G. (2009). Player Modeling using Self-Organization in Tomb Raider: Underworld. Paper presented at the IEEE Computational Intelligence in Games. Ducheneaut, N., & Moore, R. J. (2004). The social side of gaming: a study of interaction patterns in a massively multiplayer online game. Paper presented at the ACM Conference on Computer Supported Cooperative Work. Ducheneaut, N., Yee, N., Nickell, E., & Moore, R. J. (2006). Building an MMO With Mass Appearl. Games and Culture, 1(4), 281–317. doi:10.1177/1555412006292613 Dumas, J. S. (2003). User-Based Evaluations. In The HCI Handbook (pp. 1093–1117). Lawrence Erlbaum Associates. Fua, Y., Ward, M. O., & Rundensteiner, E. A. (1999). Hierarchical Parallel Coordinates for Visualizing Large Multi-variate Data Sets. Paper presented at the IEEE Visualization.

24

Gilleade, K. M., & Dix, A. (2004). Using frustration in the design of adaptive videogames. Paper presented at the ACM SIGCHI International Conference on Advances in Computer Entertainment Technology. Goetz, P. (2006). Too Many Clicks! Unit-Based Interfaces Considered Harmful. Gamasutra. Grosso, B. (2009). Refining the “Metrics Driven” Approach. from http://www.allfacebook.com/ 2009/06/ refining-metrics-driven-approach/ Han, J., Kamber, M., & Pei, J. (2005). Data Mining: Concepts and Techniques. Morgan Kaufmann. Hand, D., Heikki, M., & Padhraic, S. (2001). Principles of Data Mining. Cambridge: MIT Press. Hazlett, R. L. (2006). Measuring Emotional Valence during INteractive Experiences: Boys at video Game Play. Paper presented at the Computer-Human Interaction (CHI). Hilbert, D. M., & Redish, J. C. (1999). A practical guide to usability testing: Intellect books. Hoobler, N., Humphreys, G., & Agrawala, M. (2004). Visualizing Competitive Behaviors in Multi-User Virtual Environments. Paper presented at the IEEE Visualization Conference. Hurst, A., Hudson, S. E., & Mankoff, J. (2007). Dynamic detection of novice vs. skilled use without a task model. Paper presented at the ComputerHuman Interaction (CHI). Ijsselsteijn, W., de Kort, Y., Poels, K., Jugelionis, A., & Bellotti, F. (2007). Characterizing and Measuring User Experiences in Digital Games. Paper presented at the International Conference on Advances in Computer Entertainment (ACE). Isbister, K., & Schaffer, N. (2008). Game Usability: Advancing the Player Experience. Morgan Kaufman Publishers.

Analyzing User Behavior in Digital Games

Järvinen, A. (2008). Games without Frontiers: Theories and Methods for Game Studies and Design. PhD Dissertation, Acta Electronica Universitatis Tamperensis; 701; Tampere University Press, University of Tampere, Finland. Jørgensen, A. H. (2004). Marrying HCI/Usability and Computer Games: A Preliminary Look. Paper presented at the NordiCHI. Kennerly, D. (2003). Better Game Design Through Data Mining. Gamasutra. Kim, J. H., & Gunn, D. V. E., S., Phillips, B. C., Pagulayan, R. J., & Wixon, D. (2008). Tracking Real-Time User Experience (TRUE): A comprehensive instrumentation solution for complex systems. Paper presented at the Computer-Human Interaction (CHI). King, D., & Chen, S. (2009). Metrics for Social Games. Paper presented at the Social Games Summit. Klein, J., Moon, Y., & Picard, R. W. (2002). This computer responds to user frustration: Theory, design and results. Interacting with Computers, 14(2), 119–140. Kort, J., Steen, M. G. D., de Poot, H., ter Hofte, H., & Mulder, I. (2005). Studying usage of complex applications. Paper presented at the Measuring Behavior. Kuniavsky, M. (2003). Observing the User Experience: A Practitioner’s Guide to User Research. Morgan Kaufman Publishers. Laitinen, S. (2005, June 23). Better Games Through Usability Evaluation and Testing. Gamasutra. Laitinen, S. (2006). Do usability expert evaluation and test provide novel and useful data for game development? Journal of Usability Studies, 1(2). Lazzaro, N., & Mellon, L. (2005). Fun Meters for Games. Austin Game Developers Conference.

Longley, P., Goodchild, M. F., Macquire, D., & Rhind, D. (2005). Geographic Information Systems and Science. John Wiley & Sons Publishers. Ludwig, J. (2007). Flogging: Data Collection on the High Seas. Austin Game Developers Conference. Mandryk, R., & Inkpen, K. (2004). Physiological indicators for the evaluation of co-located collaborative play. Paper presented at the ACM Conference on Computer Supported Cooperative Work (CSCW). Medler, B. (2009). Generations of Game Analytics, Achievements and High Scores. Eludamos Journal for Computer Game Culture, 3(2), 177–194. Medlock, M. C., Wixon, D., Terrano, M., Romero, R. L., & Fulton, B. (2002). Using the RITE method to improve products: A definition and a case study. Paper presented at the Usability Professionals Association. Mellon, L. (2004). Metrics in MMP Development and Operations. Game Developers Conference. Mellon, L. (2009). Applying Metrics Driven Development to MMO Costs and Risks. Versant Corporation. Mellon, L., & DuBose, B. (2008). Acellerating the QA Test Cycle Via Metrics and Automation. Mellon, L., & Kazemi, D. (2008). Wake up and smell the metrics! Austin Game Developers Conference. Moskowitz, D., Shodhan, S., & Twardos, M. (2009). Spore API: Accessing a Unique Database of Player Creativity. Paper presented at the SIGGRAPH. Nacke, L. E. (2009). Affective Ludology - Scientific Measurement of User Experience in Interactive Entertainment. Blekinge Institute of Technology, Blekinge.

25

Analyzing User Behavior in Digital Games

Nørgaard, M., & Rau, J. (2007). User Testing in the Combat Zone. Paper presented at the International Conference on Advances in Computer Entertainment Technology (ACE).

Rosenzweig, S. (1944). An outline of frustration theory. In Hunt, J. M. (Ed.), Personality and the behavior disorders (pp. 379–388). Oxford, England: Ronald Press.

Norman, D. A. (1988). The Psychology of everyday things. New York: Basic Books.

San Augustin, J., Mateo, J. C., Hansen, J. P., & Villanueva, A. (2009). Evaluation of the potential of gaze input for game interaction. PsychNology Journal, 7(2), 213–236.

Pagulayan, R. J., & Keeker, K. (2007). Measuring Pleasure and Fun: Playtesting. In Wilson, C. (Ed.), Handbook of Formal and Informal Interaction Design Methods. Morgan Kaufmann Publishers. Pagulayan, R. J., Keeker, K., Wixon, D., Romero, R. L., & Fuller, T. (2003). User-centered design in games. In The HCI Handbook (pp. 883–906). Lawrence Erlbaum Associates. Pagulayan, R. J., Keeker, K., Wixon, D., Romero, R. L., & Fuller, T. (2003). User-centered design in games. In The HCI Handbook (pp. 883–906). Lawrence Erlbaum Associates. Peterson, E. T. (2004). Web Analytics Demystified: A Marketer’s Guide to Understanding how your Web Site Affects your Business. Celilo Group Media. Ravaja, N., Saari, T., Laarni, J., & Kallinen, K. (2006). Phasic emotional reactions to video game events: A psychophysiological investigation. Media Psychology, 8, 343–367. doi:10.1207/ s1532785xmep0804_2 Redding, P. (2009). Aarf! Arf arf arf: Talking to the player with barks. Renaud, K., & Gray, P. (2004). Making sense of low-level usage data to understand user activities. Paper presented at the SAICSIT. Romero, J. (2008). Successful Instrumentation: Tracking Attitudes and Behaviors to Improve Games. San Jose, CA.

26

Southey, F., Xiao, G., Holte, R. C., Trommelen, M., & Buchanan, J. (2005). Semi-Automated Gameplay Analysis by Machine Learning. Paper presented at the AIIDE Sullivan, L. (2006, September 8). Video-Game Analytics Track Players´ Behavior. TechWeb Technology News, from http://www.techweb.com/ wire/192700568 Swain, C. (2008). Master Metrics: The Science Behind the Art of Game Design. NLGD Conference, Utrecht, Netherlands. Thawonmas, R., & Iizuka, K. (2008). Visualization of online-Game Players Based on Their Action Behaviors. International Journal of Computer Games Technology. Thawonmas, R., Kashifuji, Y., & Chen, K.-T. (2008). Design of MMORPG Bots Based on Behavior Analysis. Paper presented at the Advances in Computer Entertainment Technology. Thompson, C. (2007). Halo 3: How Microsoft Labs Invented a New Science of Play. Wired Magazine, 15. Thurau, C., Kersting, K., & Bauckhage, C. (2009). Convex Non-Negative Matrix Factorization in the Wild. Paper presented at the ICDM. Tychsen, A., & Canossa, A. (2008). Defining Personas in Games Using Metrics. Paper presented at the Future Play.

Analyzing User Behavior in Digital Games

Tychsen, A., Newman, K., Brolund, T., & Hitchens, M. (2007). Cross-format analysis of the gaming experience in multi-player role playing games. Paper presented at the DIGRA. Vorderer, P., & Bryant, J. (Eds.). (2006). Playing Video Games - Motives, Responses and Consequences. Mahwah: Lawrence Erlbaum Associates Publishers. Williams, D., Consalvo, M., Caplan, S., & Yee, N. (2009). Looking for Gender (LFG): Gender roles and behaviors among online gamers. The Journal of Communication, 59, 700–725. doi:10.1111/j.1460-2466.2009.01453.x Williams, D., Yee, N., & Caplan, S. E. (2008). Who plays, how much, and why? Debunking the stereotypical gamer profile. Journal of Computer-Mediated Communication, 13, 993–1018. doi:10.1111/j.1083-6101.2008.00428.x Yannakakis, N., & Hallam, J. (2007). Capturing Player Enjoyment in computer Games. Paper presented at the Advanced Intelligent Paradigms in Computer Games. Yannakakis, N., & Hallam, J. (2008). Real-time Game Adaptation of Augmented-Reality Games for Optimizing Player Satisfaction. Paper presented at the IEEE Symposium on Computational Intelligence and Games.

ADDITIONAL READING Chittaro, L., & Ieronutti, L. (2004). A visual tool for tracing users´ behavior in virtual environments. Paper presented at the Working Conference on Advanced Visual Interfaces. Chittaro, L., Ranon, R., & Ieronutti, L. (2006). VU-Flow: A Visualization Tool for Analyzing Navigation in Virtual Environments. IEEE Transactions on Visualization and Computer Graphics, 12(6), 1475–1485. doi:10.1109/TVCG.2006.109

Drachen, A., & Canossa, A. (2008). Defining Personas in Games Using Metrics. Paper presented at the Future Play. Drachen, A., & Canossa, A. (2009a). Analyzing Spatial User Behavior in Computer Games using Geographic Information Systems. Paper presented at the 13th MindTrek. Drachen, A., & Canossa, A. (2009b). Towards Gameplay Analysis via Gameplay Metrics. Paper presented at the 13th MindTrek Drachen, A., Canossa, A., & Yannakakis, G. (2009). Player Modeling using Self-Organization in Tomb Raider: Underworld. Paper presented at the IEEE Computational Intelligence in Games. Ducheneaut, N., & Moore, R. J. (2004). The social side of gaming: a study of interaction patterns in a massively multiplayer online game. Paper presented at the ACM Conference on Computer Supported Cooperative Work. Ducheneaut, N., Yee, N., Nickell, E., & Moore, R. J. (2006). Building an MMO With Mass Appearl. Games and Culture, 1(4), 281–317. doi:10.1177/1555412006292613 Han, J., Kamber, M., & Pei, J. (2005). Data Mining: Concepts and Techniques. Morgan Kaufmann. Hand, D., Heikki, M., & Padhraic, S. (2001). Principles of Data Mining. Cambridge: MIT Press. Kim, J. H., & Gunn, D. V. E., S., Phillips, B. C., Pagulayan, R. J., & Wixon, D. (2008). Tracking Real-Time User Experience (TRUE): A comprehensive instrumentation solution for complex systems. Paper presented at the Computer-Human Interaction (CHI). Mellon, L. (2009). Applying Metrics Driven Development to MMO Costs and Risks. Versant Corporation.

27

Analyzing User Behavior in Digital Games

Pagulayan, R. J., & Keeker, K. (2007). Measuring Pleasure and Fun: Playtesting. In Wilson, C. (Ed.), Handbook of Formal and Informal Interaction Design Methods. Morgan Kaufmann Publishers. Pagulayan, R. J., Keeker, K., Wixon, D., Romero, R. L., & Fuller, T. (2003). User-centered design in games. In The HCI Handbook (pp. 883–906). Lawrence Erlbaum Associates. Williams, D., Consalvo, M., Caplan, S., & Yee, N. (2009). Looking for Gender (LFG): Gender roles and behaviors among online gamers. The Journal of Communication, 59, 700–725. doi:10.1111/j.1460-2466.2009.01453.x Williams, D., Yee, N., & Caplan, S. E. (2008). Who plays, how much, and why? Debunking the stereotypical gamer profile. Journal of Computer-Mediated Communication, 13, 993–1018. doi:10.1111/j.1083-6101.2008.00428.x

KEY TERMS AND DEFINITIONS Game Metric: Any quantitative measures used during or following game development.

28

Game metrics generally relate to measures of performance, process or players. Gameplay Metric: A specific type of player metric. Any quantitative measure obtained from players of computer games, as pertaining to their actions inside the game environment or during interaction with game menus/interface. Player Metric: Any quantitative measure obtained from players of computer games. User Behavior: The behavior expressed by users of a specific product, notably in terms of how the user interact with the product. User behavior takes place in a given spatio-temporal and social context. User Experience (UX): The subjectively perceived experience of using/interacting with a product, for example the experience of playing a computer game. User-Initiated Event (UIE): Any action initiated by a user of digital software, for example, pressing the mouse button or repositioning a digital avatar in a virtual world environment.

29

Chapter 2

Comparing Two Playability Heuristic Sets with Expert Review Method:

A Case Study of Mobile Game Evaluation Janne Paavilainen University of Tampere, Finland Hannu Korhonen Nokia Research Center, Finland Hannamari Saarenpää University of Tampere, Finland

ABSTRACT The expert review method is a widely adopted usability inspection method for evaluating productivity software. Recently, there has been increasing interest to apply this method for the evaluation of video games, as well. In order to use the method effectively, there need to be playability heuristics that take into account the characteristics of video games. There are several playability heuristic sets available, but they are substantially different, and they have not been compared to discover their strengths and weaknesses in game evaluations. In this chapter, we report on a study comparing two playability heuristic sets in evaluating the playability of a video game. The results indicate that the heuristics can assist inspectors in evaluating both the user interface and the gameplay aspects of the game. However, playability heuristics need to be developed further before they can be utilized by the practitioners. Especially, the clarity and comprehensibility of the heuristics need to be improved, and the optimal number of heuristics is still open. DOI: 10.4018/978-1-60960-774-6.ch002

Copyright © 2012, IGI Global. Copying or distributing in print or electronic forms without written permission of IGI Global is prohibited.

Comparing Two Playability Heuristic Sets with Expert Review Method

INTRODUCTION Competition in the game industry is hard and the gaming experience has become a crucial factor in differentiating similar kinds of game titles. If a game is not enjoyable to play, players can easily switch to another game. Typically, gaming experience can be evaluated after there is a working prototype implemented and it is ready for beta testing. At this point, correcting any playability problems (e.g. UI navigation is complex, goals are not clear, or the challenge level or pace is set incorrectly) is often too expensive, or the project schedule does not allow any delays due to marketing reasons. As a result, there is a need for an evaluation method that can identify these playability problems before beta testing starts and thus provide time for corrections. Productivity software has been evaluated for years with the expert review method to find usability problems in the design and implementation (Nielsen and Molich, 1990). In an expert review method, a small group of experts evaluate a product based on a set of heuristics. Heuristics are guidelines, rule of thumb statements, which reflect the desirable aspects of a given product. The method is cost-efficient and effective, and the design can be evaluated already in early project stages. A skillful and knowledgeable usability expert can identify usability problems as accurately as in user testing (Molich and Dumas, 2008). Evaluating games with this method is a tempting idea, but traditional usability heuristics cannot be applied directly (Federoff, 2002; Desurvire et al., 2004; Korhonen and Koivisto, 2006). The design objectives between productivity software and games are different, and the evaluation methods need to recognize this divergence as well before they can be effectively applied to the domain of games. Pagulayan et al. (2008) describe these differences, and according to them, productivity software is a tool and the design intention is to make tasks easier, more efficient, less error-prone, and increase the quality of the results.

30

Games, instead, are intended to be pleasurable to play and sufficiently challenging (Pagulayan et al., 2008). Because of these differences, a set of specifically designed heuristics are needed when video games are evaluated with the expert review method. Playability has been studied very little by game researchers and HCI researchers. The research community is lacking a commonly agreed upon definition for playability, which would describe important issues influencing the game experience and guiding the research work. Egenfield-Nielsen et al. (2008) state that a game has good playability when it is easy to use, fun and challenging. Järvinen et al. (2002) have defined playability as an evaluation tool which consists of four components: 1) functional, 2) structural, 3) audiovisual, and 4) social playability. These components can be used to evaluate both the formal and the informal aspects of a game. Fabricatore et al. (2002) have defined playability in action games as the possibility of understanding and controlling gameplay. In addition, they state that poor playability cannot be balanced or replaced by non-functional aspects of the design. According to usability glossary1, playability is affected by the quality of different aspects, including storyline, controls, pace, and usability. Along with the academia, the game industry has also approached the issue of playability from the practical perspective. For example, Games User Research at Microsoft Game Studios has published several empirical papers considering usability, playability and user experience in video games2. For our work we have defined playability as follows. Playability is related to intuitiveness, unobtrusiveness, fun, and challenge. In addition, it is a combination of user interface and the gameplay, i.e. game content aspects of the game. In multiplayer games, players’ social interaction also affects playability. The user interface consists of game menus, controls and an interface through which a player interacts with game items, nonplayer characters (NPCs), and other players. A game has good playability when the user interface

Comparing Two Playability Heuristic Sets with Expert Review Method

is intuitive and unobtrusive, so that the player can concentrate on playing the game. Gameplay includes, for example, game mechanics, narrative, and goals that the player tries to achieve. Fun and challenge are created by the gameplay; a game has good playability especially when the gameplay is understandable, balanced, suitably difficult, and engaging. Despite the lack of a commonly agreed upon definition, researchers have defined playability heuristic sets that could be used to evaluate video games and their playability. However, the development work is still ongoing and there is very little knowledge about the usefulness and clarity of these heuristic sets. In addition, there are no previously published studies that would use these heuristic sets to evaluate a video game and compare the results. In this chapter, we report an experiment in which two playability heuristic sets are used in a video game evaluation to discover their weaknesses and strengths in identifying playability problems, as well as whether they are helpful to inspectors in conducting the evaluation. The results indicate that heuristic sets should be improved before they are usable for the practitioners. The rest of this chapter is structured as follows. First, we review relevant related work regarding the expert review method and introduce playability heuristics that have been developed. Next, we describe an experiment we arranged to compare two playability heuristic sets in game evaluation and report the results of the experiment followed by discussion and conclusions.

RELATED WORK In this section, we present the expert review method and look at playability heuristics that have been developed for evaluating video games.

The Expert Review Method Inspection methods are well known and widely used to evaluate the usability of products. According to Nielsen (2005a) heuristic evaluation is the most popular usability inspection method. The popularity of the method is due to its costeffectiveness and ease of implementation in discovering usability problems even by novice inspectors. The method was developed by Nielsen and Molich (1990) and it is also known as the expert review method, since the inspectors’ experience and knowledge affect the results of the evaluation (Jacobsen et al., 1998). The first version of usability heuristics was published together with the method, but the revised and currently used version of the heuristics was published in 1994 (Nielsen, 1994a). When conducting an expert review, inspectors go through the evaluated software individually and write down usability problems that they notice. The problems are assigned with the appropriate heuristic, and later the findings from all inspectors are synthesized into a single usability report with suggestions on how to fix the problems. The heuristics act as a guideline for the inspectors to focus on typical usability issues that cause problems. Moreover, the problems found are also rated with a severity rating (e.g. three-scale rating minor, major, and critical) to emphasize its severity for the usability of the product. Nielsen (2005b) has suggested that three to five expert inspectors (preferably with domain expertise) are enough, since adding more inspectors would not increase the amount of problems found significantly. Nielsen’s heuristics were made for evaluating productive software interface design. However, several researchers have extended these heuristics or developed new ones for different application domains. Ling and Salvendy (2005) present a summary of some of these studies. The summary contains domains such as websites, e-learning systems, groupware, notification systems, and games.

31

Comparing Two Playability Heuristic Sets with Expert Review Method

The applicability of traditional usability heuristics in game evaluations has been questioned by game researchers (Federoff, 2002; Desurvire et al., 2004; Korhonen and Koivisto, 2006). The most important reason for this is that usability heuristics concentrate primarily on the user interface and disregard other aspects of a product. For example, in video games it is equally important to evaluate the gameplay as well. In their study, Johnson and Wiles show how games contravene the traditional usability heuristics to achieve a good game experience (Johnson and Wiles, 2003). Hence, game researchers have started to develop heuristics which would include both usability and gameplay issues, to assist game developers in discovering playability problems in the game design.

Development of Playability Heuristics In the early 1980s, Malone (1982) studied video games and what makes the user interface enjoyable. He identified three principles (challenge, fantasy, and curiosity) that are needed for designing enjoyable user interfaces. Malone also calls these principles heuristics in a design framework. Although the list is very limited and it concentrates only on high level issues in games, it highlights the importance of the game content in the evaluation. Clanton argued that human-computer interaction in games can be divided into three levels: game interface, game mechanics and game play (Clanton, 1998). Furthermore, Clanton describes 15 principles which can be used to gain and keep the interest of a player. Although these principles are not called heuristics, they can be understood as such. Federoff (2002) defined the first playability heuristics that are similar to usability heuristics. These heuristics were a result of a case study in a game company, but they lack validation, or at least such results have not been published. Fabricatore et al. (2002) have studied players and their preferences that will affect the playability

32

of action video games. Even though these are not described as heuristics, they could be converted into heuristics to evaluate games belonging to this specific genre. Desurvire et al. (2004) published Heuristic Evaluation for Playability (HEP) in 2004. The heuristics were validated in a study in which heuristic evaluation was compared to user testing. The results indicated that the HEP heuristics were very good at identifying playability problems in a game prototype. Korhonen and Koivisto (2006) have published a playability heuristic set for mobile games. However, the heuristics are applicable for evaluating games in other platforms as well because of their modular structure. The playability heuristic set can be extended or limited based on the needs of the evaluation. Later on, the core heuristics were extended with multiplayer heuristics (Korhonen and Koivisto, 2007). In addition, the number of heuristics is smaller than in two previous sets from Federoff (2002) and Desurvire et al. (2004). The heuristics of Korhonen and Koivisto have been validated in several game evaluations. Schaffer (2007) presented a white paper introducing heuristics for usability in games. According to Schaffer, earlier heuristics lacked concrete examples, making them less clear for practitioners. Schaffer’s heuristics are based on literature and on his own expertise from the field of human-computer interaction. Pinelle et al. published ten game usability heuristics that are based on game reviews (Pinelle et al. 2008a) and they have been validated in a preliminary study. These heuristics are used to evaluate game usability and there are no heuristics concerning gameplay issues. Later Pinelle et al. (2009) also published ten additional heuristics focusing on multiplayer usability. Desurvire and Wiberg (2009) have presented PLAY heuristics, which are based on the earlier HEP heuristics (Desurvire et al., 2004). The PLAY heuristics feature 19 top level headings, each containing one to six heuristics (50 heuristics in

Comparing Two Playability Heuristic Sets with Expert Review Method

total). The PLAY heuristics are aimed toward action adventure, first person shooter and real-time strategy games. More recently, Köffel et al. (2010) have presented a synthesis of earlier heuristics. The authors handpicked 29 heuristics from the earlier models and added ten more heuristics focusing on advanced electronic tabletop games. There are also other guidelines that are targeted for game developers in order to make games more engaging and usable for players (e.g. Falstein and Barwood, 2001; Snow, 2007). In addition to articles and websites, edited books have also been published recently on the topic (e.g. Isbister and Schaffer, 2008; Bernhaupt, 2010). These cover various methods for evaluating usability, playability and user experience in general. Based on the literature review, expert review could be an appropriate method for evaluating the playability of video games, but there should be specific playability heuristics accompanying the method. Several researchers have started to develop these heuristics, and currently, there are multiple heuristic sets available. However, the work is still ongoing and the heuristic sets are quite different, even though there are some common issues included. This raises the question of which heuristic set should be used in a game evaluation, and if one heuristic set is easier to use than another one from the inspectors’ point of view. In our work, we aim to achieve some clarity about the different playability heuristic sets and their usefulness in game evaluations. We compare two playability heuristic sets in a game evaluation to see what their strengths and weaknesses are and how inspectors perceive the heuristic sets.

Validation of Domain Specific Heuristics Traditional usability heuristics (Nielsen, 1994a) are widely used in usability evaluations. One limitation of these heuristics is that they are advisedly general and they do not cover specific

characteristics of systems (Ling and Salvendy, 2005: Sim et al., 2009). Nielsen has noted that there could be domain-specific heuristics for a specific class of products as a supplement to the general heuristics. Several researchers have developed heuristics for different domains. Baker et al. (2001) have developed heuristics to identify usability problems in real-time collaboration within a shared visual workspace. In their validation study, two groups of inspectors evaluated two groupware applications by using these heuristics. The evaluation results were compared to previously published studies by replicating Nielsen and Molich’s analysis methodology (Nielsen, 1990; Nielsen, 1992; Nielsen, 1994a). Berry (2003) has developed heuristics for notification systems and compared them to traditional usability heuristics. In the study, inspectors were divided into two groups and they evaluated three versions of the system’s user interface. The results indicate that both heuristic sets performed quite similarly in identifying usability problems. Mankoff et al. (2003) compared the performance of heuristics developed for evaluation of ambient displays to traditional usability heuristics. The results indicate that with the help of modified heuristics inspectors were able to identify more usability problems than inspectors using traditional usability heuristics. However, the best result was achieved by combining both sets of heuristics. Bertini et al. (2009) have also developed mobile usability heuristics that not only take into account applications, but also the device and context in which it is used. This heuristic set was also evaluated against the traditional usability heuristics in the evaluation of two mobile applications. Previously presented studies compared domain-specific heuristics to traditional usability heuristics. Although this kind of comparison is useful in determining the usefulness of a new heuristic set, it still lacks a critical analysis of the heuristics to determine their usefulness in that specific domain. One of the problems related

33

Comparing Two Playability Heuristic Sets with Expert Review Method

to such comparison studies is that there are not many domains that have multiple domain-specific heuristic sets. Ling and Salvendy (2005) have presented a summary of some studies in which domain-specific heuristic sets have been developed. In their review, each domain contained only one heuristic set. Zuk et al. (2006) used three sets of domainspecific heuristics that are targeted to information visualization systems. The goal of the study was not to compare heuristic sets as such, but to identify a common set of heuristics derived from these three heuristic sets to find common visualization problems. Video games are one of the few application domains that have multiple heuristic sets developed for them (e.g. Federoff, 2002; Desurvire et al., 2004; Korhonen and Koivisto, 2006; Pinelle, 2008a). However, studies empirically evaluating different heuristic sets in this domain and comparing the applicability of the heuristic sets have not been published.

THE EXPERIMENT We arranged an evaluation session with eight persons who are working in the game industry or in the academia as game researchers to explore how the expert review method and two playability heuristic sets operate in a game evaluation. First, the participants were briefly introduced to the expert review method and the heuristics that are commonly used in productivity software evaluations to give an idea of how usability specialists usually conduct evaluations. Three participants had previous experience in conducting an expert review of a product. The participants were divided into 4 teams (two persons in each team) forming two groups based on the playability heuristic sets that were given to them. In the evaluation session, the teams played a game for one hour. The inspectors observed the game and wrote down short descriptions of pos-

34

sible playability problems they encountered in the game. After that, the teams went through their own playability problems and assigned violated playability heuristics to these problems. Finally, the observations were talked through with other teams and the participants discussed playability problems, the evaluation method, and the playability heuristics they used. The results section describes the main observations from the discussion, which was recorded with a video camera. The game evaluated was EA Mobile’s The Simpsons: Minutes to Meltdown3. We selected this game for the evaluation because it was short enough to be evaluated in a single day workshop as it can be completed in less than 30 minutes in real time. The game did not receive favorable reviews (e.g. Buchanan 2007; Dredge, 2007; PurestProdigy, 2008), which made us to believe there were many playability problems to be found. We also wanted to use a mobile game, because it would be easy to obtain for every inspector and the evaluation session would be easy to arrange. In this game, the player controls Homer who has 30 minutes to save Springfield from a nuclear disaster. The game features slightly tilted pseudo3D top-down perspective and Homer’s movement is controlled with a rocker key, or keys 2, 4, 6 and 8 (up, left, right, down respectively) on the keypad. Left soft key brings up the pause menu and right soft key shows the timer. Context sensitive action is executed by pressing the rocker key or key 5 on the keypad. The game features three levels and locations. The game starts at the Simpson’s apartment where Homer must find his car keys so he can drive to the power plant. Homer must interact with various characters and avoid furious citizens to complete the first level. The second level is the Springfield city centre. Homer has crashed his car and must continue on foot. There are various obstacles and hazards on the way as he tries to reach the plant. The last level is the power plant, where Homer must find the override valve to cancel the meltdown.

Comparing Two Playability Heuristic Sets with Expert Review Method

In this level, depending on the mobile phone, the player also controls Bart in certain key locations.

Playability Heuristics As there are multiple heuristic sets available, it is important to choose heuristic sets that can be compared. As described in the chapter on related work, some heuristics are proposals which have not been validated, others are targeted to a specific game genre or they do not consider all aspects of playability. For this study, we selected playability heuristic sets from Desurvire et al. (2004) and Korhonen and Koivisto (2006) because they resembled each other, they were probably the most advanced at the time of the study and they have both been validated by their authors. Although there were other heuristics available, they were either not validated by their authors (e.g. Federoff, 2002; Schaffer, 2007) or they only focused on usability issues in games (e.g. Pinelle et al., 2008a). The heuristic sets selected are based on literature reviews and the initial heuristics were reviewed by game researchers and game designers. The playability heuristics were developed further in game development projects and they were both validated by their authors. Although the sets have some similarities in their content, there are major differences in how the heuristics are organized and described.

Heuristic Evaluation of Playability (HEP) This playability heuristic set contains 43 heuristics and the authors have defined four categories for organizing them (Desurvire et al., 2004; see Appendix 1). Game Play is related to challenges and problems that the player must face to win a game. Game Story includes heuristics for story and character development. Game Mechanics involves the structure which defines how the game units interact with the environment. Game Usability addresses the interface and the controls the player utilizes when interacting with the game.

Most heuristics are presented as one sentence descriptions and they have been validated in a user study. Teams that used this playability heuristic set during the evaluation are referred to as Violet 1 and Violet 2 in the results section.

Playability Heuristics for Mobile Games This playability heuristic set contains 29 heuristics which have been organized into three modules (Korhonen and Koivisto, 2006; see Appendix 2). Each module can be included or excluded depending on the needs of the evaluation. Two core modules, Gameplay and Game Usability, are common to all games. The Mobility module contains heuristics that are specific to mobile games. Each heuristic is described in detail on a separate document including examples of use (Koivisto and Korhonen, 2006). The heuristics were validated in several mobile game evaluations conducted by playability experts. Teams that used this playability heuristic set during the evaluation are referred to as Orange 1 and Orange 2 in the results section.

RESULTS In this section, we present the main results of the study, which are based on the comments from the group interview as well as the analysis of the data collected from the evaluation reports.

Heuristics Provide Guidance The participants commented that the expert review method seemed to be an appropriate method for evaluating video games, because it helped them to focus on the different aspects of the game during the evaluation. One game industry participant stated that they use similar kind of evaluation approach on a weekly basis to manage game production processes.

35

Comparing Two Playability Heuristic Sets with Expert Review Method

Participants commented that heuristics could also be useful in the design and implementation phases in game development to identify possible playability problems that might exist in the design. The participants would not use heuristics in the very early phases of game development, as they considered that the heuristics might restrict creativity at that point. One double expert participant also stated that she would not use heuristics in the very late phases either, as actual playtesting would be more beneficial. According to her, heuristics are good for finding basic problems and playtesting is used to refine the details.

Defining a Proper Abstraction Level Although the participants appreciated the efficiency of the expert review method, they stated that there are certain challenges when the method is applied to game evaluations. Their biggest concern was related to the heuristics and their descriptions. The variety of video games is enormous and defining playability heuristics that are suitable for evaluating all kinds of games can be a challenge. “It is a laborious and challenging task to define heuristics that can capture those aspects that are considered to be essential from the point of view of game experience. In addition, game environments are changing constantly as they adopt new kinds of technical enablers”, Violet 1 inspector. Therefore, it is important that the playability heuristics are on the right abstraction level. Too specific heuristics restrict their applicability to a large number of games, but in contrast, heuristics that are on a very general level lose their power to guide and assist inspectors during the evaluation. The participants stated that both heuristic sets had problems in this respect. Playability heuristics defined by Desurvire et al. had both detailed heuristics and very broad heuristics, which were difficult to use during the evaluation. For example, there is the Game Play

36

heuristic number 10 (“The game is fun for the Player first, the designer second and the computer third. That is, if the non-expert player’s experience isn’t put first, excellent game mechanics and graphics programming triumphs are meaningless.”) This heuristic was considered to be very difficult to apply during an evaluation. Playability heuristics defined by Korhonen and Koivisto also had some heuristics which were considered to be quite specific, and they could be combined to provide a more concise list. For example, heuristics GP9 (“The players can express themselves”) and GP10 (“The game supports different playing styles”) describe similar kinds of issues on heading level, which are related to the player’s behavior and playing style in the game world.

Usability and Gameplay Problems The participants suggested that it is generally easier to find usability problems than it is to find gameplay problems from games. However, in this case, the participants found more gameplay related problems. One suggested reason for this was that the inspectors were experienced gamers and might thus be blind to some usability problems that would annoy novice gamers. On the other hand, in this case the user interface was generally liked as it was considered to reflect the industry standards well. From this perspective, it might be easier to spot usability problems, which are few but obvious, than playability problems, which are more numerous but also harder to point out. Korhonen and Koivisto (2006) reported similar findings in their earlier study when they developed their heuristics. One participant questioned the difference between usability and gameplay problems. Sometimes, it was hard to decide if the problem found was related to usability or gameplay. “There were a lot of things which were on the borderline of being in the game’s interaction structure or in the user

Comparing Two Playability Heuristic Sets with Expert Review Method

interface structure. In the end, it was not possible to separate them clearly”, Orange 2 inspector. In addition, it was discussed that it is not always obvious whether gameplay problems are actually problems or just hard challenges. For example, if a player has to go through the same part of the game over and over again, but is highly immersed and enjoys the game, calling out a gameplay problem might not be necessary. In our case, it was noted by one inspector that the game evaluated featured so many problems that it was not possible to get immersed.

Evaluation Process The participants commented that the evaluation task influenced their gaming experience, and for that reason, playing the game was different than what it would be normally. The objective of the game design is to immerse players on different levels (Ermi and Mäyrä, 2005). The evaluation task, however, prevents immersion because the inspectors need to be alert all the time and inspect the game for problems in playability. In addition, the inspectors found it difficult to play the game like any other player would, and for that reason, the evaluation session cannot be considered equal to a normal play session. “There are two dimensions that make the evaluation difficult. First, you should be able to describe the problem that you have identified and it affects your gaming experience negatively. On the other hand, you should play the game as players would play and get a positive gaming experience”, Violet 1 inspector. Another issue the participants pointed out was that it is very important for the inspectors to familiarize themselves with the heuristics beforehand. In our study, playability heuristics sets contained 43 or 29 heuristics. When considering Miller’s golden rule of 7±2 (Miller, 1956), the number of heuristics might have been overwhelming and

there was too much information about the heuristics to keep in mind. During the evaluation, it was time-consuming to browse through the whole list and find a proper heuristic for each playability problem. Due to time constraints, the participants did not study the heuristics beforehand, but there was a playability expert present in case they had any questions concerning the heuristics. The large amount of heuristics brought up an idea in the end discussion that inspectors could use the heuristics in a more systematic fashion. First, the game would be played for some time, and then the inspector would go through the heuristics in a checklist manner. This approach has also been suggested earlier by Nielsen (1994b). However, it was noted that in this case the tool is using the inspector and not vice versa and therefore there is a possibility that the inspector does not recognize possible problems that are outside of the scope of the heuristics. It was also considered that the heuristics in general are problem-oriented and do not support positive findings very well. It was suggested that instead of formal statements, the heuristics could be in the form of questions, which might intuitively help to discover positive features from the game. One participant stated that due to his expertise in game development, he was able to find positive features easily. Finding positive features was considered important. One inspector stated that, from a psychological perspective, no one likes to read an evaluation report which is full of negativity. Especially if a report is delivered to a person who has not seen the game, it might give a false impression of it. Reporting positive features of the game also enhances the possibility that those features are left intact and are not accidentally removed, changed or “fixed” by the designers (Nielsen 1994a).

37

Comparing Two Playability Heuristic Sets with Expert Review Method

Revision for Playability Heuristic Sets The participants found several issues troublesome with the playability heuristics defined by Desurvire et al. These issues made utilization of the heuristics difficult during the evaluation. There is a total of 43 heuristics in the set and the participants thought this is too much. The heuristics are organized into four categories, but the participants did not find them helpful because some heuristics were in a different category from what they expected. For example, some Game Story heuristics were located in the Game Play category and vice versa. The Violet team inspectors pointed out that Game Play heuristic number 8 (“Players discover the story as part of game play”) would belong to the Game Story category rather than the Game Play category, and that Game Story heuristic number 6 (“Player experiences fairness of outcomes”) sounds more like a heuristic belonging to the Game Play category. There were also some overlapping heuristics in the set. Another problem that the participants noticed was the descriptions of the heuristics, as they were presumably influenced by the game that was used as a basis during the development work. Some heuristics were seen to be too specific to apply in practice. In addition, the descriptions were not consistent in terms of wording and the level of generalization. Some heuristics clearly set requirements for the game design and state explicitly how the game should be designed. An example of this kind of a heuristic is Game Play heuristic number 3 (“Provide clear goals, present overriding goal clearly as well as short-term goals throughout the play”), whereas some heuristics are more like recommendations for designers. For example, Game Play heuristic number 5 (“The game is enjoyable to replay”) is a too general and subjective issue to evaluate with the expertbased method. There were also some heuristics which were difficult to understand and apply during the evaluation. The participants pointed

38

out Game Play heuristic number 10 (“The game is fun for the Player first, the designer second and the computer third. That is, if the non-expert player’s experience isn’t put first, excellent game mechanics and graphics programming triumphs are meaningless”) to be an example of such a heuristic. Finally, the participants commented that the current writing style and format makes understanding the heuristics more difficult because they are not consistent and are missing either a heading or a description. Playability heuristics developed by Korhonen and Koivisto were not optimal either. Even though each heuristic clearly had the heading and the description, they were presented in two documents which made using them difficult. The first document described the heuristics on a heading level, in a similar fashion to the other heuristic set. There was a separate document available that contained the descriptions and practical examples (Koivisto and Korhonen, 2006). Some descriptions were also long, and reading the entire description and examples was time-consuming. The participants commented that this playability heuristic set was in a better shape and the wording of the heuristics was more consistent and on a more generic level than in the other heuristic set. However, there were still some heuristics such as GP8 (“There are no repetitive or boring tasks”) and GP11 (“The game does not stagnate”) that sounded similar on the heading level and they could possibly be combined. One suggestion was that heuristics could be organized on different levels inside one category. For example, high level usability problems would consist of more abstract heuristics which are applicable to a large number of games. Low level usability heuristics would be more focused on certain game genres as it has been noted that different genres have different problems (Pinelle, 2008b).

Comparing Two Playability Heuristic Sets with Expert Review Method

Table 1. Playability problems concerning different heuristic categories Orange

Categories

Teams

O1

O2

Game Usability

3

Gameplay

8

Mobility

Violet

Total

%

V1

4

7

28%

6

5

13

52%

3

1

0

1

4%

-

Game Story

-

-

-

-

Game Mechanics

-

-

-

-

V2

Total

%

3

9

20%

11

14

32%

-

-

-

1

2

3

7%

5

0

5

11%

Unassigned

1

3

4

16%

5

8

13

30%

Total

13

12

25

100%

20

24

44

100%

Evaluation Statistics Surprisingly, there was very little consistency in reporting playability problems between the four teams. Only a few playability problems were identified by more than one team. Even teams with the same heuristic lists assigned different heuristics to a playability problem. The teams reported 69 playability problems in total. 13 playability problems were reported by two or more teams and 52 playability problems were uniquely reported by a single team. In addition, there were 13 duplicate playability problems (i.e. reported multiple times by a single team), but these problems have been excluded from the analysis. There was a difference between groups in how many playability problems they reported. Teams Orange 1 (O1) and Orange 2 (O2) identified 13 and 12 playability problems respectively. Teams Violet 1 (V1) and Violet 2 (V2) identified a substantially larger number of playability problems, 20 and 24 playability problems respectively. Most problems reported by both teams were related to gameplay issues. Teams O1 and O2 reported more than a half of the problems belonging to this category. The second most common problem category was game usability. Playability problem distribution in the heuristic categories is illustrated in Table 1. It should be noted that the heuristic categories are not comparable because they contain different heuristics. In addition, some

categories exist only in one playability heuristic set and those categories are left empty on the table in the other set. Some user interface problems were due to the mobile phones the participants used. The game looked and sounded different on their devices, and there were some minor changes in the game content because of the smaller screen resolution and the memory capacity of the device. The teams seemed to have difficulties in assigning violated heuristics to the identified playability problems, and the participants commented that they could not always find a proper playability heuristic from the set. Especially for teams V1 and V2, assigning a violated playability heuristic was difficult, and they left 30% of the playability problems open (Table 1). Teams O1 and O2 were able to do it more accurately, and they left only 16% of reported playability problems open. Usually the teams assigned only one violated heuristic per problem, but there were a few cases when they assigned several heuristics (Table 2). The teams reported nine playability problems to which they assigned several heuristics from the same category that the problem violated. Three of them were related to Game Usability and the rest were Gameplay problems. There were also three playability problems to which the teams assigned playability problems from different categories. These problems were combinations of Gameplay, Game Usability, and Game Story related issues.

39

Comparing Two Playability Heuristic Sets with Expert Review Method

Table 2. Assigning heuristics to playability problems

Heuristics

Orange Teams Count

%

Count

%

Single

12

48%

28

64%

Many Same Category

8

32%

1

2%

Many Different Category

1

4%

2

5%

Unassigned

4

16%

13

30%

Total

25

100%

44

1005

Finding the same playability problems seemed to be difficult, and the majority of the playability problems (75%) are reported only by a single team (Figure 1). However, there was one playability problem which all teams reported. The playability problem concerns player progression in the game. If Homer dies in the game, the player has to start all over from the beginning. Teams were also consistent when assigning the violated heuristic for this problem. Teams O1 and O2 assigned the gameplay heuristic GP14 (“The player does not lose any hard-won possessions”). In addition, Team O1 marked that the problem violated Gameplay heuristic GP8 (“There are no repetitive or boring tasks”). Teams V1 and V2 also had a consensus on the violated heuristic. They assigned Game Story heuristic GS6 (“Player experiences fairness of outcomes”) to describe the problem. In addition, team V1 assigned Game Play heuristic GP5 (“The game is enjoyable to replay”). There were two problems that were identified by three teams. The first problem concerned navigation in the game world, due to the fact that the player gets lost very easily on the second level. The second problem was related to the game menu design. Even though the three teams identified the same problem, each team assigned a different heuristic to describe the problem or left the problem open. For the playability problems identified by two teams, there was hardly any consistency in the heuristics assigned. One explanation for different evaluation results between

40

Violet Teams

Figure 1. Playability problems reported by teams

teams O1 and O2 might be that team O1 reported very specific playability problems such as “catching the pig is hard and it is not clear how it should be done”, whereas team O2 reported more general level problems like “the game is too linear and prone to stagnate” or “game features boring repetition without optional ways to advance”. Similarly, teams V1 and V2 used quite different heuristics to describe the playability problems. Team V2 did not assign any playability problems to the Mechanics category, even though team V1

Comparing Two Playability Heuristic Sets with Expert Review Method

used Mechanics quite extensively. They found three playability problems that violated heuristic ME1 (“game should react in a consistent, challenging, and exciting way to the player’s actions (e.g., appropriate music with the action)”). In addition, they assigned two other heuristics from the Mechanics category to describe identified problems. Correspondingly, team V2 concluded that five playability problems violated Game Play heuristic GP2 (“provide consistency between the game elements and the overarching setting and story to suspend disbelief”), while team V1 thought that none of their playability problems violated this heuristic. Both teams reported playability problems with different abstraction levels. Team V1 identified both specific and general level problems, whereas team V2 concentrated on criticizing the illogical gameplay. Examples of such playability problems in the gameplay were “Barney opens up a gate when you bring him coffee” and “The player can only go through certain bushes”.

DISCUSSION The inspectors’ comments indicate that the expert review method is applicable to game evaluations. This supports earlier claims made by several researchers (e.g. Desurvire et al., 2004; Korhonen and Koivisto, 2006; Laitinen 2006; Pinelle et al., 2008a). The inspectors liked the method as it is not too time-consuming or laborious to execute. They thought that the method could also be used at earlier development phases, when there are only design sketches or low fidelity prototypes available. Playability heuristics, however, need to be developed further before the method can be widely adopted by the practitioners. Playability heuristics should be presented in a similar manner to how Nielsen (1994a) has presented traditional playability heuristics. Ling and Salvendy (2005) have also concluded that domain-specific heuris-

tic sets should be structured and they should not contain too many heuristics. In this study, we used playability heuristic sets developed by Desurvire et al. (2004) and by Korhonen and Koivisto (2006). The study revealed that both heuristic sets need to be improved in order for them to be usable and easily understandable. The inspectors considered that there were too many heuristics in the set developed by Desurvire et al (2004). In addition, their organization into categories, as well as their descriptions, need to be developed further as they were inconsistent and overlapping. This was visible in the evaluation data, as the teams who used this heuristic set did not assign any violated heuristic to 30% of the identified playability problems. The playability heuristic set developed by Korhonen and Koivisto (2006) was more consistent in wording and organization, but the inspectors thought that the heuristics should be accompanied by short and compact descriptions since the descriptions were presented in a separate document. In the study design, it is important to think about the hardware that will be used, since it can have remarkable influence on what kinds of playability problems are reported. Especially mobile phones can be very different in their technical capabilities and there are many device generations on the market. We did not anticipate that the game would vary so much on different devices. In our study, the inspectors used their personal mobile phones in the evaluation and therefore, we did not have sufficient control over the hardware. Some teams reported playability problems which were somewhat specific for the device they used. These problems were related to the audio and the amount of content on the screen. Gray and Salzman (1998) call this as an internal validity problem. In game evaluations, the inspectors seem to face similar challenges in identifying the same playability problems. This result is consistent with comparison studies conducted with productivity software. However, the results of this study are slightly better than those reported by Molich and

41

Comparing Two Playability Heuristic Sets with Expert Review Method

Dumas (2008). The majority of the playability problems (75%) were reported only by a single team. However, one playability problem was commonly reported by all four teams, and the violated heuristic was assigned consistently within the teams, as well. Furthermore, there were 12 playability problems which were reported by at least two teams. It is an interesting question for future work why the inspectors do not identify the same problems in the game. Unlike productivity software, video games in general are quite linear at the beginning and the players are guided through first missions or levels by the game design (Adams and Rollings, 2007). Therefore, the inspectors should have gone through the same aspects of the game and presumably identified the same problems. The problem that all teams identified in this study was critical for the game experience, and this is probably the reason why it was reported. Further research is required to understand why the teams did not find the same playability problems. There are several possible reasons for the inconsistency of the problems reported. One obvious explanation is the inspector effect (Jacobsen et al., 1998) and its influence on the results. It has been concluded in many previous evaluation studies of productivity software that evaluation results differ quite a lot because of this factor (e.g. Jacobsen et al., 1998; Molich and Dumas, 2008). In this study, our inspectors had different backgrounds, game design, and evaluation experience. Although we tried to balance teams in their evaluation experience and game design experience, it did not seem to be enough. Another possible explanation for the inconsistency might be the heuristic sets that the inspectors used in the study. The purpose of the heuristic sets is to guide the evaluation and remind the inspectors to pay attention to important aspects of playability. The results indicate that using the heuristic sets was not straightforward and the inspectors had some problems with them, which might also explain the difference in reported playability problems.

42

However, one interesting observation from the data is that most of the playability problems that were reported by two or three teams included teams from both groups. There were only a few problems which were reported only by one group. Unfortunately, there is not sufficient data from this study to make any deeper analysis on how a playability heuristic set influences finding playability problems in a video game. Third possible explanation for this inconsistency might be that the inspectors had a different baseline for reporting. Some teams mainly reported general problems, focusing on certain aspects of the game, while others reported very specific problems. In the Violet group, team V2 did not report any playability problems which would violate heuristics from the Mechanics category, whereas team V1 assigned five problems to this category. Correspondingly, team V1 assigned five playability problems to one Game Play heuristic which was not used by team V2 at all. Otherwise, the teams reported problems that violated a large number of playability heuristics from different categories. This difference is probably due to the evaluation experience that the teams had. In addition, we probably did not instruct the teams clearly enough on what kinds of issues they should pay attention to and how to report those findings. In future studies, there should be greater emphasis on the instructions for creating a problem reporting baseline as equal as possible. One characteristic of the game evaluations is to think about the origin of the playability problem, and whether the problem is in the user interface or in the game content. This problem does not usually exist in productivity software evaluations, as the evaluation concerns only user interface aspects of the product. Evaluating the content and the user interface together has been studied on other domains (e.g Galagher et al., 2001). In our study, the inspectors identified 12 playability problems to which they assigned multiple heuristics, and in three cases they were from different categories. We do not know for sure why the inspectors did

Comparing Two Playability Heuristic Sets with Expert Review Method

it this way. They possibly did not have time to analyze the problem thoroughly to find the origin of the problem. It was also noted that the inspectors should always be aware of the creative vision that the designers have and what is used as a design principle when designing a game. Typically, it also guides the experience that the designers want to create for the players (Pagulayan and Steury, 2004). The evaluation should always be relative to this vision, because otherwise the inspectors might focus the evaluation incorrectly and point out issues which are contradicting to the vision. This further emphasizes the need for the designers and the inspectors to work in close cooperation. In the future, we are planning to continue these comparison studies to find out the optimal set of playability heuristics. The shortcoming of this study was that we could not compare which playability heuristics are used to describe the same playability problems because there was too little data for this. In the next study, we should also eliminate internal validity errors, which were related to the inspectors’ experience in using the evaluation method, their familiarity with the playability heuristics, and the devices that the inspectors used in the evaluation. In addition, there is a need to think about a new presentation format for the heuristics, which would better support the evaluation. In the discussion, it became obvious that presenting heuristics as a list is not easily utilized during the evaluation. The heuristics could be improved by using keywords, color coding for the categories and presenting them in a compact format, such as cards, for example.

CONCLUSION In this chapter, we have explored two different playability heuristic sets to discover their strengths and weaknesses when they are used to evaluate a mobile game using the expert review method. This kind of a comparison study has not been reported

previously, although there are several playability heuristic sets available. The results indicate that both heuristic sets should be improved as there were problems in clarity and comprehensibility. This study is the first attempt to develop playability heuristics that would help inspectors to conduct game evaluations, and to provide precise and relevant evaluation results when evaluating video games with an analytical evaluation method.

ACKNOWLEDGMENT The GameSpace project was funded by Tekes (Finnish Funding Agency for Technology and Innovation), Veikkaus, TeliaSonera Finland, Nokia, Sulake Corporation, and Digital Chocolate. We thank all project members and the inspectors in this study.

REFERENCES Adams, E., & Rollings, A. (2007). Game Design and Development: Fundamentals of Game Design. Upper Saddle River, NJ: Prentice Hall. Baker, K., Greenberg, S., & Gutwin, C. (2001). Heuristic Evaluation of Groupware Based on the Mechanics of Collaboration. In Proceedings of the 8th IFIP International Conference on Engineering for Human-Computer Interaction (pp. 123-140). Springer-Verlag. Retrieved September 15, 2010, from http://portal.acm.org/ citation. cfm?id=645350.650731 Bernhaupt, R. (Ed.). (2010). Evaluating User Experience in Games: Concepts and Methods. London, UK: Springer-Verlag. Berry, B. (2003). Adapting Heuristics for Notification Systems. Paper presented at the ACM SE Conference.

43

Comparing Two Playability Heuristic Sets with Expert Review Method

Bertini, E., Catarci, T., Dix, A., Gabrielli, S., Kimani, S., & Santucci, G. (2009). Appropriating Heuristic Evaluation for Mobile Computing. International Journal of Mobile Human Computer Interaction, 1(1), 20–41. doi:10.4018/ jmhci.2009010102

Ermi, L., & Mäyrä, F. (2005). Fundamental components of the gameplay experience: Analysing immersion. In DIGRA 2005 Conference: Changing Views – Worlds in Play. Retrieved September 15, 2010, from http://citeseerx.ist.psu.edu /viewdoc/ summary?

Buchanan, L. (2007). The Simpsons: Minutes to Meltdown Review. IGN game review. Retrieved September 10, 2010, from http://wireless.ign.com/ articles /807/807589p1.html

Fabricatore, C., Nussbaum, M., & Rosas, R. (2002). Playability in Action Videogames: A Qualitative Design Model. Human-Computer Interaction, 17(4), 311–368. doi:10.1207/ S15327051HCI1704_1

Clanton, C. (1998). An interpreted demonstration of computer game design. In CHI 98 conference summary on Human factors in computing systems (pp. 1-2). Los Angeles, CA: ACM. Desurvire, H., Caplan, M., & Toth, J. A. (2004). Using heuristics to evaluate the playability of games. In CHI ‘04 extended abstracts on Human factors in computing systems (pp. 1509-1512). Vienna, Austria: ACM. Desurvire, H., & Wiberg, C. (2009). Game Usability Heuristics (PLAY) for Evaluating and Designing Better Games: The Next Iteration. In A. Ozok & P. Zaphiris (Eds.), Online Communities and Social Computing, Lecture Notes in Computer Science (Vol. 5621, pp. 557-566). Springer Berlin / Heidelberg. Retrieved September 15, 2010, from http://dx.doi.org/10.1007/ 978-3-642-02774-1_60 Dredge, S. (2007). The Simpsons: Minutes to Meltdown. PocketGamer game review. Retrieved September 10, 2010, from http://www.pocketgamer.co.uk/r/ Mobile/The+Simpsons:+Minutes+ To+Meltdown/review.asp?c=3683 Egenfield-Nielsen, S., Smith, J. H., & Tosca, S. P. (2008). Understanding Video Games: The Essential Introduction. London, UK: Routledge.

44

Falstein, N., & Barwood, H. The 400 Project. Retrieved September 10, 2010, from http://theinspiracy.com/ 400_project.htm. Federoff, M. A. (2002). Heuristics and Usability Guidelines for the Creation and Evaluation of Fun in Video Games. Master’s Thesis, Department of Telecommunications, Indiana University, Indiana. Galagher, K., Foster, D., & Parsons, J. (2001). The Medium Is Not the Message: Advertising Effectiveness and Content Evaluation in Print and on the Web. Journal of Advertising Research, 41(4), 57–70. Gray, W. D., & Salzman, M. C. (1998). Damaged Merchandise? A Review of Experiments That Compare Usability Evaluation Methods. Human-Computer Interaction, 13(3), 203–261. doi:10.1207/s15327051hci1303_2 Isbister, K., & Schaffer, N. (Eds.). (2008). Game Usability: Advice From The Experts For Advancing The Player Experience. San Francisco, CA: Morgan Kauffman. Jacobsen, N. E., Hertzum, M., & John, B. E. (1998). The evaluator effect in usability tests. In CHI 98 conference summary on Human factors in computing systems (pp. 255-256). Los Angeles, CA: ACM.

Comparing Two Playability Heuristic Sets with Expert Review Method

Järvinen, A., Heliö, S., & Mäyrä, F. (2002). Communication and Community in Digital Entertainment Services. University of Tampere. Johnson, D., & Wiles, J. (2003). Effective affective user interface design in games. Ergonomics, 46(13), 1332–1345. doi:10.1080/00140130310 001610865 Köffel, C., Hochleitner, W., Leitner, J., Haller, M., Geven, A., & Tscheligi, M. (2010). Using Heuristics to Evaluate the Overall User Experience of Video Games and Advanced Interaction Games. In Bernhaupt, R. (Ed.), Evaluating User Experience in Games: Concepts and Methods. Springer. doi:10.1007/978-1-84882-963-3_13 Koivisto, E. M. I., & Korhonen, H. (2006). Mobile Game Playability Heuristics. Retrieved September 10, 2010, from http://www.forum.nokia.com. Korhonen, H., & Koivisto, E. M. I. (2006). Playability heuristics for mobile games. In Proceedings of the 8th conference on Humancomputer interaction with mobile devices and services (pp. 9-16). Helsinki, Finland: ACM. doi:10.1145/1152215.1152218 Korhonen, H., & Koivisto, E. M. I. (2007). Playability heuristics for mobile multi-player games. In Proceedings of the 2nd international conference on Digital interactive media in entertainment and arts (pp. 28-35). Perth, Australia: ACM. Laitinen, S. (2006). Do Usability Expert Evaluation and Testing Provide Novel and Useful Data For Game Development? Journal of Usability Studies, 2(1), 64–75. Ling, C., & Salvendy, G. (2005). Extension of Heuristic Evaluation Method: a Review and Reappraisal. Ergonomia. An International Journal of Ergonomics and Human Factors, 27(3), 179–197.

Malone, T. W. (1982). Heuristics for designing enjoyable user interfaces: Lessons from computer games. In Proceedings of the 1982 conference on Human factors in computing systems (pp. 63-68). Gaithersburg, MD: ACM. Mankoff, J., Dey, A. K., Hsieh, G., Kientz, J., Lederer, S., & Ames, M. (2003). Heuristic evaluation of ambient displays. In Proceedings of the SIGCHI conference on Human factors in computing systems (pp. 169-176). Ft. Lauderdale, FL: ACM. Mäyrä, F., & Ermi, L. (2005). Fundamental Components of the Gameplay Experience: Analysing Immersion. Retrieved September 15, 2010, from http://www.digra.org/dl/db/ 06276.41516.pdf Miller, G. A. (1956). The magical number seven plus or minus two: some limits on our capacity for processing information. Psychological Review, 63(2), 81–97. doi:10.1037/h0043158 Molich, R., & Dumas, J. S. (2008). Comparative Usability Evaluation (CUE-4). Behaviour & Information Technology, 27, 263–281. doi:10.1080/01449290600959062 Nielsen, J. (1992). Finding usability problems through heuristic evaluation. In Proceedings of the SIGCHI conference on Human factors in computing systems (pp. 373-380). Monterey, CA: ACM. Nielsen, J. (1994a). Heuristic Evaluation. In Nielsen, J., & Mack, R. L. (Eds.), Usability Inspection Methods. New York: John Wiley & Sons. Nielsen, J. (1994b). Usability Engineering. San Francisco, CA: Morgan Kaufmann. Nielsen, J. (2005a). Heuristic Evaluation. UseIt website. Retrieved September 10, 2010, from http://www.useit.com/ papers/heuristic/ Nielsen, J. (2005b). How to Conduct a Heuristic Evaluation. UseIt website. Retrieved September 10, 2010, from http://www.useit.com/papers/ heuristic/heuristic_evaluation.html

45

Comparing Two Playability Heuristic Sets with Expert Review Method

Nielsen, J., & Molich, R. (1990). Heuristic evaluation of user interfaces. In Proceedings of the SIGCHI conference on Human factors in computing systems: Empowering people (pp. 249-256). Seattle, WA: ACM. Pagulayan, R., & Steury, K. (2004). Beyond usability in games. interactions, 11(5), 70-71. Pagulayan, R. J., Keeker, K., Wixon, D., Romero, R. L., & Fuller, T. (2008). User-centered Design in Games. In Jacko. J Sears A., (Eds.). Handbook for Human-Computer Interaction in Interactive Systems, Second Edition, Lawrence Erlbaum Associates, Inc., 741-759. Pinelle, D., Wong, N., & Stach, T. (2008a). Heuristic evaluation for games: usability principles for video game design. In Proceeding of the twenty-sixth annual SIGCHI conference on Human factors in computing systems (pp. 1453-1462). Florence, Italy: ACM. Pinelle, D., Wong, N., & Stach, T. (2008b). Using genres to customize usability evaluations of video games. In Proceedings of the 2008 Conference on Future Play: Research, Play, Share (pp. 129-136). Toronto, Ontario, Canada: ACM. Pinelle, D., Wong, N., Stach, T., & Gutwin, C. (2009). Usability heuristics for networked multiplayer games. In Proceedings of the ACM 2009 international conference on Supporting group work (pp. 169-178). Sanibel Island, FL: ACM. PurestProdigy. (2007). The Simpsons: Minutes to Meltdown. GameFaqs user review. Retrieved September 10, 2010, from http://www.gamefaqs. com/mobile /mobile/review/R125801.html Schaffer, N. (2007). Heuristics for Usability in Games. White Paper. Retrieved September 10, 2010, from http://www.playerfriendly.com/ files/ heuristics.pdf

46

Sim, G., Read, J., & Cockton, G. (2009). Evidence Based Design of Heuristics for Computer Assisted Assessment. In T. Gross, J. Gulliksen, P. Kotzé, L. Oestreicher, P. Palanque, R. Prates, & M. Winckler (Eds.), Human-Computer Interaction – INTERACT 2009, Lecture Notes in Computer Science (Vol. 5726, pp. 204-216). Springer Berlin / Heidelberg. Retrieved September 15, 2010, from http://dx.doi.org/10.1007/ 978-3-642-03655-2_25 Snow, B. (2007). Game Usability 101. BusinessWeek. Retrieved September 10, 2010, from http://www.businessweek.com/ innovate/content/ oct2007 /id20071012_041625.htm. Zuk, T., Schlesier, L., Neumann, P., Hancock, M. S., & Carpendale, S. (2006). Heuristics for information visualization evaluation. In Proceedings of the 2006 AVI workshop on BEyond time and errors: novel evaluation methods for information visualization (pp. 1-6). Venice, Italy: ACM.

ADDITIONAL READING Adams, E. (2009). Fundamentals of Game Design (2nd ed.). Berkeley, CA: New Riders Press. Barr, P., Noble, J., & Biddle, R. (2007). Video game values: Human-computer interaction and games. Interacting with Computers, 19(2), 180–195. doi:10.1016/j.intcom.2006.08.008 Bekker, M. M., Baauw, E., & Barendregt, W. (2008). A comparison of two analytical evaluation methods for educational computer games for young children. Cognition Technology and Work, 10(2), 129–140. doi:10.1007/s10111-007-0068-x Benford, S., Crabtree, A., Reeves, S., Sheridan, J., Dix, A., Flintham, M., & Drozd, A. (2006). The Frame of the Game: Blurring the Boundary between Fiction and Reality in Mobile Experiences. In Proceedings of the SIGCHI conference on Human Factors in computing systems (pp. 427-436). Montréal, Québec, Canada: ACM.

Comparing Two Playability Heuristic Sets with Expert Review Method

Bertini, E., Gabrielli, S., & Kimani, S. (2006). Appropriating and assessing heuristics for mobile computing. In Proceedings of the working conference on Advanced visual interfaces (pp. 119-126). Venezia, Italy: ACM.

Falstein, N. (2005). Understanding Fun - The Theory of Natural Funativity. In S. Rabin (Ed.), Introduction to Game Development (Game Developement Series, pp. 71-97). Hingham, MA: Charles River Media, Inc.

Bond, M., & Beale, R. (2009). What makes a good game?: using reviews to inform design. In Proceedings of the 23rd British HCI Group Annual Conference on People and Computers: Celebrating People and Technology (pp. 418-422). Cambridge, United Kingdom: British Computer Society. Retrieved September 15, 2010, from http:// portal.acm.org/ citation.cfm?id=1671065&dl= GUIDE&coll=GUIDE&CFID= 104687055&CFTOKEN=98301315

Febretti, A., & Garzotto, F. (2009). Usability, playability, and long-term engagement in computer games. In Proceedings of the 27th international conference extended abstracts on Human factors in computing systems (pp. 4063-4068). Boston, MA: ACM.

Bosser, A., & Nakatsu, R. (2006). Hardcore Gamers and Casual Gamers Playing Online Together. In R. Harper, M. Rauterberg, & M. Combetto (Eds.), Entertainment Computing - ICEC 2006, Lecture Notes in Computer Science (Vol. 4161, pp. 374-377). Springer Berlin / Heidelberg. Retrieved September 15, 2010, from http://dx.doi. org/10.1007/ 11872320_53 Crawford, C. (2003). Chris Crawford on Game Design. Berkeley, CA: New Riders Games. De Kort, Y. A. W., & Ijsselsteijn, W. A. (2008). People, places, and play: player experience in a socio-spatial context. Computers in Entertainment, 6(2), 1–11. doi:10.1145/1371216.1371221 Dongseong, C., Kim, H., & Kim, J. (1999). Toward the Construction of Fun Computer Games: Relative Importance of Design Factors between Developers versus Players. Personal and Ubiquitous Computing, 3(3), 92–104. Dumas, J. S., & Loring, B. A. (2008). Moderating Usability Tests: Principles and Practices for Interacting. San Francisco, CA: Morgan Kaufmann. Dumas, J. S., Molich, R., & Jeffries, R. (2004). Describing usability problems: are we sending the right message? Interaction, 11(4), 24–29. doi:10.1145/1005261.1005274

Hornbæk, K., & Frøkjær, E. (2008). Comparison of techniques for matching of usability problem descriptions. Interacting with Computers, 20(6), 505–514. doi:10.1016/j.intcom.2008.08.005 Hvannberg, E. T., Law, E. L., & Lárusdóttir, M. K. (2007). Heuristic evaluation: Comparing ways of finding and reporting usability problems. Interacting with Computers, 19(2), 225–240. doi:10.1016/j.intcom.2006.10.001 Jegers, K. (2008). Investigating the Applicability of Usability and Playability Heuristics for Evaluation of Pervasive Games. In 2008 Third International Conference on Internet and Web Applications and Services (pp. 656-661). Presented at the 2008 3rd International Conference on internet and Web Applications and Services (ICIW), Athens, Greece. Kampmann, W. (2003). Playing and Gaming Reflections and Classifications. Game Studies, 3(1). Retrieved September 15, 2010, from http:// www.gamestudies.org/ 0301/walther/ Korhonen, H. (2010). Comparison of Playtesting and Expert Review Methods in Mobile Game Evaluation. Paper presented at the International Conference on Fun and Games, 2010. Korhonen, H., Montola, M., & Arrasvuori, J. (2009). Understanding Playful User Experience through Digital Games. Paper presented at the Designing Pleasurable Products and Interfaces.

47

Comparing Two Playability Heuristic Sets with Expert Review Method

Ling, C., & Salvendy, G. (2009). Effect of evaluators’ cognitive style on heuristic evaluation: Field dependent and field independent evaluators. International Journal of Human-Computer Studies, 67(4), 382–393. doi:10.1016/j.ijhcs.2008.11.002 Nielsen, J. (1992). Reliability of severity estimates for usability problems found by heuristic evaluation. In Posters and short talks of the 1992 SIGCHI conference on Human factors in computing systems (pp. 129-130). Monterey, California: ACM. Omar, H. M., & Jaafar, A. (2008). Playability Heuristics Evaluation (PHE) approach for Malaysian educational games. In 2008 International Symposium on Information Technology (pp. 1-7). Presented at the 2008 International Symposium on Information Technology, Kuala Lumpur, Malaysia. Orvis, K. A., Horn, D. B., & Belanich, J. (2008). The roles of task difficulty and prior videogame experience on performance and motivation in instructional videogames. Computers in Human Behavior, 24(5), 2415–2433. doi:10.1016/j. chb.2008.02.016 Paavilainen, J. (2010). Critical Review on Video Game Evaluation Heuristics: Social Games Perspective. Paper presented at the international conference on Future Play: Research, Play, Share. Papaloukas, S., Patriarcheas, K., & Xenos, M. (2009). Usability Assessment Heuristics in New Genre Videogames. In Proceedings of the 2009 13th Panhellenic Conference on Informatics (pp. 202-206). IEEE Computer Society. Retrieved September 15, 2010, from http://portal.acm.org/ citation.cfm?id=1684685 Pinelle, D., Wong, N., Stach, T., & Gutwin, C. (2009). Usability heuristics for networked multiplayer games. In Proceedings of the ACM 2009 international conference on Supporting group work (pp. 169-178). Sanibel Island, FL: ACM.

48

Qin, H., Rau, P. P., & Salvendy, G. (2010). Effects of different scenarios of game difficulty on player immersion. Interacting with Computers, 22(3), 230–239. doi:10.1016/j.intcom.2009.12.004 Rollings, A., & Adams, E. (2003). Andrew Rollings and Ernest Adams on Game Design (Ltd Rmst.). Berkeley, CA: New Riders Games. Rouse, R. (2001). Game Design: Theory and Practice. Plano, TX: Wordware Publishing. Rubin, J. (1994). Handbook of Usability Testing: How to plan, design and conduct effective test. John Wiley & Sons. Salen, K., & Zimmerman, E. (2003). Rules of Play: Game Design Fundamentals. Cambridge, MA: The MIT Press. Schell, J. (2008). The Art of Game Design. San Francisco, CA: Morgan Kaufmann. Sutcliffe, A., & Gault, B. (2004). Heuristic evaluation of virtual reality applications. Interacting with Computers, 16(4), 831–849. doi:10.1016/j. intcom.2004.05.001 Sweetser, P., & Wyeth, P. (2005). GameFlow: a model for evaluating player enjoyment in games. Computers in Entertainment, 3(3). Wiberg, C. (2005). Fun in the Home: Guidelines for Evaluating Interactive Entertainment on the Web. Paper presented at the Conference on HumanComputer Interaction International. Wiberg, C., Jegers, K., & Desurvire, H. (2009). How Applicable is Your Evaluation Methods Really? Analysis and Re-design of Evaluation Methods for Fun and Entertainment. In Proceedings of the 2009 Second International Conferences on Advances in Computer-Human Interactions (pp. 324-328). IEEE Computer Society. Retrieved September 15, 2010, from http://portal. acm.org/ citation.cfm?id=1509869.1509931 &coll=Portal&dl=GUIDE&CFID =104691490&CFTOKEN =32024841

Comparing Two Playability Heuristic Sets with Expert Review Method

KEY TERMS AND DEFINITIONS Expert Review: An analytical evaluation method in which experts conduct an evaluation. The experts have good knowledge of usability/ playability principles and they are preferably also experts in the domain. Game Evaluation: Game evaluations assess playability or game experience of a game by using different evaluation methods. The methods can be either analytical evaluation methods or user testing. Game Experience: An experience enabled by the game for the player. Usability and playability are game centric terms where as game experience is related to the experiential engagement between the player and the game. Heuristics: Heuristics are guidelines, rule of thumb statements, which reflect the desirable aspects of a given product. Playability: Playability is defined as aspects that relate to desirable aspects of a good game. The game has good playability when the user interface is intuitive and unobtrusive, so that the player can concentrate on playing the game. Gameplay includes, for example, game mechanics, narrative,

and goals that the player tries to achieve. Fun and challenge are created by the gameplay; the game has good playability especially when the gameplay is understandable, balanced, suitably difficult, and engaging. In multiplayer games, players’ social interaction also affects playability. Usability: The extent to which a product can be used by specified users to achieve specified goals with effectiveness, efficiency and satisfaction in a specified context of use. Additionally, other factors such as learnability, memorability and error prevention can be also considered to be part of usability. Video Game: A type of game existing as and controlled by software, run by a device with video terminal and played with an interaction interface.

ENDNOTES 1



2



3

http://www.usabilityfirst.com/glossary/ term_657.txl http://mgsuserresearch.com/ http://www.eamobile.com/Web/mobilegames/the-simpsons-minutes-to-meltdown

49

Comparing Two Playability Heuristic Sets with Expert Review Method

APPENDIX 1. EVALUATION HEURISTICS BY DESURVIRE ET AL. (2004)

50

ID

Category

GP 1

Game Play

Description Player’s fatigue is minimized by varying activities and pacing during game play.

GP 2

Game Play

Provide consistency between the game elements and the overarching setting and story to suspend disbelief.

GP 3

Game Play

Provide clear goals, present overriding goal early as well as short-term goals throughout play.

GP 4

Game Play

There is an interesting and absorbing tutorial that mimics the game play.

GP 5

Game Play

The game is enjoyable to replay.

GP 6

Game Play

Game should be balanced with multiple ways to win.

GP 7

Game Play

Player is taught skills early that you expect the players to use later, or right before the new skill is needed.

GP 8

Game Play

Players discover the story as part of game play.

GP 9

Game Play

Even if the game cannot be modeless, it should be perceived as modeless.

GP 10

Game Play

The game is fun for the player first, the designer second and the computer third. That is, if the non-expert player’s experience isn’t put first, excellent game mechanics and graphics programming triumphs are meaningless.

GP 11

Game Play

Player should not experience being penalized repetitively for the same failure.

GP 12

Game Play

Player’s should perceive a sense of control and impact onto the game world. The game world reacts to the player and remembers their passage through it. Changes the player makes in the game world are persistent and noticeable if they back-track to where they’ve been before.

GP 13

Game Play

The first player action is painfully obvious and should result in immediate positive feedback.

GP 14

Game Play

The game should give rewards that immerse the player more deeply in the game by increasing their capabilities (power-up), and expanding their ability to customize.

GP 15

Game Play

Pace the game to apply pressure but not frustrate the player. Vary the difficulty level so that the player has greater challenges as they develop mastery. Easy to learn, hard to master.

GP 16

Game Play

Challenges are positive game experiences, rather than negative experience (results in their wanting to play more, rather than quitting).

GS 1

Game Story

Player understands the storyline as a single consistent vision.

GS 2

Game Story

Player is interested in the storyline. The story experience relates to their real life and grabs their interest.

GS 3

Game Story

The player spends time thinking about possible story outcomes.

GS 4

Game Story

The player feels as though the world is going on whether their character is there or not.

GS 5

Game Story

The player has a sense of control over their character and is able to use tactics and strategies.

GS 6

Game Story

Player experiences fairness of outcomes.

GS 7

Game Story

The game transports the player into a level of personal involvement emotionally (e.g., scare, threat, thrill, reward, punishment) and viscerally (e.g., sounds of environment).

GS 8

Game Story

Player is interested in the character because (1) they are like me, (2) they are interesting to me, (3) the characters develop as action occurs.

GM 1

Game Mechanics

Game should react in a consistent, challenging, and exciting way to the player’s actions (e.g., appropriate music with the action).

GM 2

Game Mechanics

Make effects of the Artificial Intelligence (AI) clearly visible to the player by ensuring they are consistent with the player’s reasonable expectations of the AI actor.

GM 3

Game Mechanics

A player should always be able to identify their score/status in the game.

GM 4

Game Mechanics

Mechanics/controller actions have consistently mapped and learnable responses.

GM 5

Game Mechanics

Shorten the learning curve by following the trends set by the gaming industry to meet user’s expectations.

GM 6

Game Mechanics

Controls should be intuitive and mapped in a natural way; they should be customizable and default to industry standard settings.

GM 7

Game Mechanics

Player should be given controls that are basic enough to learn quickly yet expandable for advanced options.

Comparing Two Playability Heuristic Sets with Expert Review Method

ID

Category

Description

GU 1

Game Usability

Provide immediate feedback for user actions.

GU 2

Game Usability

The player can easily turn the game off and on, and be able to save games in different states.

GU 3

Game Usability

The player experiences the user interface as consistent (in control, color, typography, and dialog design) but the game play is varied.

GU 4

Game Usability

The player should experience the menu as part of the game.

GU 5

Game Usability

Upon initially turning the game on the player has enough information to start playing.

GU 6

Game Usability

Players should be given context-sensitive help while playing so that they do not get stuck or have to rely on a manual.

GU 7

Game Usability

Sounds from the game provide meaningful feedback or stir a particular emotion.

GU 8

Game Usability

Players do not need to use a manual to play the game.

GU 9

Game Usability

The interface should be as non-intrusive to the player as possible.

GU 10

Game Usability

Make the menu layers well-organized and minimalist to the extent that the menu options are intuitive.

GU 11

Game Usability

Get the player involved quickly and easily with tutorials and/or progressive or adjustable difficulty levels.

GU 12

Game Usability

Art should be recognizable to the player, and speak to its function.

51

Comparing Two Playability Heuristic Sets with Expert Review Method

APPENDIX 2. EVALUATION HEURISTICS BY KORHONEN AND KOIVISTO (2006) ID

Category

Description

GU 1

Game Usability

Audiovisual representation supports the game.

GU 2

Game Usability

Screen layout is efficient and visually pleasing.

GU 3

Game Usability

Device UI and game UI are used for their own purpose.

GU 4

Game Usability

Indicators are visible.

GU 5

Game Usability

The player understands terminology.

GU 6

Game Usability

Navigation is consistent, logical and minimalist.

GU 7

Game Usability

Control keys are consistent and follow standard conventions.

GU 8

Game Usability

Game controls are convenient and flexible.

GU 9

Game Usability

The game gives feedback on the player’s actions.

GU 10

Game Usability

The player cannot make irreversible errors.

GU 11

Game Usability

The player does not have to memorize things unnecessarily.

GU 12

Game Usability

The game contains help.

MO 1

Game Mobility

The game and play sessions can be started quickly.

MO 2

Game Mobility

The game accommodates to the surroundings.

MO 3

Game Mobility

GP 1

Gameplay

The game provides clear goals or supports player created goals.

GP 2

Gameplay

The player sees the progress in the game and can compare the results.

52

Interruptions are handled reasonably.

GP 3

Gameplay

The players are rewarded and rewards are meaningful.

GP 4

Gameplay

The player is in control.

GP 5

Gameplay

Challenge, strategy, and pace are in balance.

GP 6

Gameplay

The first-time experience is encouraging.

GP 7

Gameplay

The game-story supports the gameplay and is meaningful.

GP 8

Gameplay

There are no repetitive or boring tasks.

GP 9

Gameplay

The players can express themselves.

GP 10

Gameplay

The game supports different playing styles.

GP 11

Gameplay

The game does not stagnate.

GP 12

Gameplay

The game is consistent.

GP 13

Gameplay

The game uses orthogonal unit differentiation (units in the game should be designed so that they are functionally different)

GP 14

Gameplay

The player does not lose any hard won possessions.

53

Chapter 3

Lovely Place to Buy!

Enhancing Grocery Shopping Experiences with a Human-Centric Approach Hiroshi Tamura University of Tokyo, Japan Tamami Sugasaka Fujitsu Laboratories Ltd., Japan Kazuhiro Ueda University of Tokyo, Japan

ABSTRACT Ubiquitous services offer huge business potential for grocery stores, however they also for increase the shopper’s experience. This chapter especially devotes the issue of exploiting the possibilities of ubiquitous services while shopping. It presents clear guidelines and implications for the development of systems aiding the consumer through their shopping activities.

INTRODUCTION Grocery stores have been thought to be one of the promising areas of application for ubiquitous computing systems. There have already been a variety of systems developed not only for research purposes but also for business purposes (Roussos, 2004). One of the most famous cases is the Extra Future Store by Metro AG (http:// www.future-store.org/). The store has employed a variety of embedded and mobile computing DOI: 10.4018/978-1-60960-774-6.ch003

systems to improve shopper experience during his/her shopping trip within the store as well as to track and manage the store inventory both from the distribution center and within the store (Kalyanam, Lal, Wolfram, 2006). From the view of shopper experiences, he/she has come to exploit the services as everyday activities, for instance, some 24 percent of shoppers utilize Personal Shopping Assistant (PSA), which is a mobile kiosk terminal embedded with a shopping cart, and some 48 percent use the interactive kiosks scattered across the store including the produce, meat, and wine sections (Kalyanam, Lal, Wolfram, 2006).

Copyright © 2012, IGI Global. Copying or distributing in print or electronic forms without written permission of IGI Global is prohibited.

Lovely Place to Buy!

On the other hand, many shoppers have a feeling of distaste toward the service automation at a storefront due to the anticipation that it will further decrease human-based interaction services (Roussos, 2004). One of the reasons yielding such the disaffection is that services provided by the systems entirely fulfill the shoppers’ expectations. For instance, several systems proposed in this area adopt shopping list management as a principal service to enhance a shopper’s experiences (Newcomb, Pashley, Stasko, 2003)(Kourouthanassis and Roussos, 2003), even though shopper does not necessarily buy items strictly according to his/ her list. As (Thomas and Garland, 1996) revealed, 93 percent of shoppers do not shop exactly by their specified lists. It was also noted that, “on average, purchases by list shoppers exceeded the categories on their lists by about 2.5 times.” Then what are the promising alternatives? We consequently determined that we investigated a shopper’s behaviors and state of mind changes during his/her shopping trip in order to ingenerate evocative design information that underscores notable experiences for shoppers. Since 2003, we have conducted an empirical design project of ubiquitous computing systems at a supermarket, named Smart Shopping-Aid, or SSA. We have secured an operating store as the project site, in Fukuoka City, located in the southwest region of Japan. We had anticipated SSA to be a truly human-centered design project. Therefore, we conducted a general survey regarding grocery shopping both in a quantitative and qualitative manner, thorough fieldwork, moment-to-moment analyses, and systems development and deployment according to the precedent research results. In this chapter, we introduce our design process and implications which will contribute to intrinsically useful ubiquitous computing systems at a supermarket.

54

UNDERSTANDING GROCERY SHOPPING PROCESS Shopping process is a long-lasting research topic in retail marketing. According to Takahashi (Takahashi, 1999), about 70 percent of items at a supermarket and about 80 percent at a supercenter were bought without preexisting plans. Meanwhile, according to our survey conducted in the Tokyo metropolitan area in 2005, respondents who were housewives1 ranging in age from their 30s to 40s answered that about 62 percent of them planned their dinner at home and about 27 percent at storefronts (Tamura, Sugasaka, Naito, Sekiguchi, Horikawa, Ueda, 2006). These two seemingly conflicting facts implied a question: ”Why are grocery shoppers buying so many unplanned items even though a majority of them already had chalked up items they were determined to purchase?” We speculated that grocery shoppers tended to gradually articulate their plans along with their shopping trips which were neither necessarily limited to at-home nor in-store but extended to a consolidation of the both. One of the reasons came from another result of our survey which showed that there were almost equal influences of major factors from the both sides to their dinner-planning (Figure 1). Past works had already pointed out that there existed a same kind of phenomenon, although the mechanisms were still at all unapparent (Takahashi, 1999). We, therefore, decided to investigate in an entire shopping process from at-home to in-store, and conducted thorough ethnographic research on them.

RESEARCH PROCEDURE In-depth research was conducted individually with nine informants (two of which were for pilot studies) from August to September 2005. The informants were all female housewives ranging from their 30s to 50s, and chosen from customers of the test store. Prior to the research, we informed

Lovely Place to Buy!

Figure 1. Influential factors of dinner planning; Bars with vertical stripes are relevant to ”planning at home” and with horizontal stripes are relevant to ”planning at storefront”. The survey was conducted in the Tokyo metropolitan area in January 2005. (Sample size 486, multiple answers allowed of this question).

them that we would observe their entire process for dinner arrangements including grocery shopping, and requested that they behave as they normally would. The research procedure consisted of four steps described as follows: 1. Pre-interview at home (30-60 min.): interviewing with an informant regarding her rituals and attitudes of everyday grocery shopping; checking whether she had a shopping plan or not; and observing her preparation for the shopping expedition, i.e. looking in the refrigerator, 2. Participant observation at the storefront (20-50 min.): accompanying an informant on her shopping trip on an entrance-to-exit basis by using contextual inquiry techniques (Beyer and Holtzblatt,1998), 3. Participant observation at home (30-120 min.): observing the informant again at home regarding storing, using, processing, and cooking purchased items with contextual inquiry techniques, and 4. Post-interview at home (30-60 min.): debriefing of what was not clarified during the previous steps.

We also asked each informant to wear a video recording system in the second step (during the participant observation at the storefront) for posterior analysis. The camera, named Encolpia (Figure 2), had originally been developed for this purpose, with the feature of 150-degree wide vision and real time MPEG-4 encoding functions.

FINDINGS FROM THE OBSERVATIONS A series of observations uncovered that a shopping process at the storefront was never uniform but the process dynamically changed corresponding to each shopper’s context; her behavior and state of mind abruptly transformed along with her shopping trip. Even if she had her shopping list, she never did just a rundown of it. Rather, it was used as one of the artifacts including goods, price tags, and in-store signs, with which she iteratively interacted to articulate her plan until the end of the shopping. We discovered that there were generally three phases across two major context shifts in the pro-

55

Lovely Place to Buy!

Figure 2. Encolpia: our original wearable video recording system equipped with 150-degree wide vision CCD. The system was designed as minimally-invasive to informants. The bottom right is a snapshot of recorded images.

cess. Observed facts which implied existence of the first phase are shown below. •

•

•

Right after starting her shopping, an informant directly went to the deli floor, closely looking at some items, saying “they’re very helpful to plan my dinner as well as to know how to cook them!” (an informant in her 30s) As an informant already had planned her dinner at home inspired by a magazine article, she briefly looked around the meat floor right after initiating her shopping, then went to the produce floor and started choosing items referring to the assortment of the meat floor she remembered. (an informant in her 40s) (An utterance of an informant) “I basically start shopping from what I don’t have to forget to buy”. (an informant in her 40s)

There seemed to exist a warm-up phase right after initiating her shopping regardless of whether she had her plan or not at the moment. In this phase, she mainly replenished what she had already recognized concurrently with developing her plan for a main dish of the day. She was so serious to look for what she had to buy without

56

omission as well as was responsible for figuring out an attractive menu that, we could speculate, she felt pretty tense during this phase. Observed facts which implied existence of the second phase are shown below. •

•

(An utterance of an informant) “Because I had decided to cook hashed rice as today’s main dish, I wanted to choose an appropriate packaged beef for it. I compared several packages with each other, and decided to choose this one because this seemed the most fresh.” (an informant in her 30s) (An utterance of an informant; after picking up a package of aroids) “As I have decided to season today’s main dish with quite a strong hint of salt and pepper, I think aroids boiled with soy and sugar as a side dish will go nicely with the main dish.” (an informant in her 40s)

This phase was the closest to what we had assumed as the typical grocery shopping: Each informant implemented her plan by choosing and picking up specified items. In this phase, she fulfilled her plan which had been made in the previous phase as well as extended her plan for side dishes which went nicely with the main

Lovely Place to Buy!

Figure 3. Changes in the rates across five time-divisions according to the informants’ actual purposes of items chosen in each division

dish. We could see that she concentrated on how she could meet her plan during this phase, e.g. to buy higher quality items with cheaper prices. We could also confirm that her state of mind in this phase got less tense than in the previous phase. Observed facts which implied existence of the third phase are shown below.

to find out new goods and reduced items to try them out. In other words, this phase acutely triggered her impulse buying. We could see she was relaxed and feeling fun during this phase since, we speculated, she was freed from her responsibility of the shopping for the day.

•

FINDINGS FROM THE VIDEO ANALYSES

•

After declaring that they finished their shopping for the day, some informants still continued to look around aisles and found what they would like to buy. (An utterance of an informant) “I’m always feeling that I might forget to buy what I have to buy today, so I usually scour for something to tell me so in the closing stage of my shopping” (an informant in her 30s)

There seemed to exist a wrapping-up phase before terminating her shopping. In this phase, she tended to buy items which were not necessarily for that particular day. She was also willing

We did moment-to-moment analyses of the video data of the seven informants (two were omitted due to lack of their video data). We normalized their shopping durations because the lengths differed from each other, split them into consecutive five divisions, and plotted the items chosen according to actual purposes including “foodstuff for main dish,” “foodstuff for side dish,” “replenishment,” and “other uses including pure impulse buy (without any assumptions of use)” in each division (Figure 3). As the result, “replenishment” got

57

Lovely Place to Buy!

Figure 4. Three-Phase Model; describing consecutive changes of shopper’s behaviors and his/her states of mind

the highest in the first and the second division, ”foodstuff for main dish” was the highest in the third division, ”foodstuff for side dish” was the highest in the fourth division, and ”replenishment” again and ”other uses, e.g. impulse buying” were the highest in the final division. To have compared this result with the result in the previous section, we could understand that the first and the second division roughly corresponded to the first phase, the third and the fourth division to the second phase, and the final division to the third phase. Consequently, we developed Three-Phase Model, or TPM (Figure 4): From the first to the

58

second phase, it was differentiated by starting to buy a foodstuff for a shopper’s main dish and from the second to the third phase, it was differentiated by completed to buy what a shopper had to acquire for that particular day.

INFORMING DESIGN As a premise for TPM, what kind of information does a shopper need in each phase? In this section, we, at the outset, speculated implicit information needs in each phase based on the tasks as well

Lovely Place to Buy!

as the state of mind, both of which are clarified in the previous section. We then described an experience scenario referring to the information needs. Finally, we distilled principal service types for prototyping by referring to the scenario. We also examined when and how each service type should become activated.

•

INFORMATION NEEDS

Given that we would develop a mobile kiosk terminal embedded with a shopping cart which would be feasible to operate in a supermarket at that time, we conducted a brainstorming session in order to come up with a variety of services corresponding to the information needs in each phase and to weave them into an experience scenario. The result is as follows:

Information Needs in the First Phase Since a shopper experiences a heavier mental load during this phase, information delivered to him/her should be focused on the tasks as well as minimal. We assumed that there are two major information needs: • •

Information for him/her to turn a notion over a main dish, and Information of planned items.

Information Needs in the Second Phase Since a shopper tends to concentrate his/her mind on examining individual goods in this phase, information delivered to him/her should be detailed as well as thorough. We assumed that there are two major information needs: • •

Information for him/her to turn a notion over side dishes, and Information having him/her choose each good especially for dinner arrangement.

Information Needs in the Third Phase Since a shopper is freed from obligations and consistency in his/her shopping on the day in this phase, information delivered to him/her should make him/her curious. We assumed that there are two major information needs:

•

Information for him/her to turn a notion over yet another dish, e.g. snack, appetizer and dessert, which is favorable addition to his/her dinner arrangement, and Information having him/her know articles unseen yet intriguing for him/her.

EXPERIENCE SCENARIOS

Experiences in the First Phase “I come to the supermarket with the thought that a meat dish is better for today’s dinner because I cooked fish yesterday. I rent out a mobile kiosk terminal embedded with a shopping cart, turn on the system, and check today’s fresh recommendations on the screen. I discover that the price of every pack of pork is reduced by twenty percent, therefore I proceed to recommendations on the screen to retrieve recipes using pork. There are plenty of attractive pork dishes varying by parts and seasonings. Since I would like to have a hefty dish today, I choose ”pork spareribs grilled with barbecue sauce.” I bookmark the recipe and head down to the meat floor to see pork spareribs. While I’m passing over the egg section, I am reminded by a notification on the screen that there are just three eggs left in the refrigerator at home, and I, therefore, take a pack of eggs.”

Experiences in the Second Phase “I arrive at the meat floor and notice that there are two types of pork spareribs. I scan a barcode attached to one of the pricier one by using a bar-

59

Lovely Place to Buy!

code reader that is embedded with the cart. Then a description of the article appears on the screen. I discover that one of them has been raised on organic foods, so that it seems healthier than the other one. I decide to buy the organically-raised one and take a pack with an adequate amount. Because I remember barbecue sauce which is necessary to cook the meal is now out of stock at home, I go to the dried goods floor. I notice that recommendations on the screen have changed from fresh foods to packaged foods. I look over them all and notice a brand of barbecue sauce is included. It is not the one I usually buy, therefore I retrieve the description of the article and realize that it is made entirely with organic ingredients. Although it is at a reduced price for this week but still a little pricier than a general one, I decide to buy to try it out. While taking a bottle, I notice that three new recommendations are flashed in rotation at the corner of the screen. One of them is a jar of mustard. I remember mixing mustard with barbecue sauce sounds good. I remember that there remains enough amount of mustard at home however, so I don’t have to buy it today. Then I head for the produce floor. I feel something plain will be better for a side dish because the pork spareribs are somewhat greasy. I look into the screen and find that there are many recipes using in-season vegetables ready, which seems to suit me. I prefer ”tomato salad with boiled spinach,” therefore I display the ingredients by touching the particular icon. I confirm that all the ingredients, except spinach, are stocked at home. I directly come close the spinach shelf, and select a lively-leafed bunch. I also bookmark this recipe for remembrance’ sake.”

Experiences in the Third Phase “I have gathered most of ingredients for today’s dinner, so I settle on the idea that I will stroll down the aisles to see what I had better buy. I again notice that instances on the screen transformed. There are brand-new goods in this week now. I

60

check them one by one, find a brand of low-calorie ice cream, and get happy to know my favorite mango taste is lined up. I go straight down to the ice cream freezer and pick up two cups of mango and chocolate taste respectively for everybody in my family for after-dinner dessert. On the way to the checkout, I stop by the tofu section and notice that there are sales rankings of the section in the last week displayed on the screen. I realize that I have not tried out the most popular one. I hesitate but bring it forward, although it seems attractive really. Then I go straight to the checkout counter. At the checkout counter, the two recipes bookmarked are automatically printed out and I can obtain them for later use. Looking back on my entire journey, I should say what a lovely supermarket it is!”

ALLOCATION AND CONFIGURATION OF SERVICES Aiming at prototyping, we distilled five service types included in the scenario. We also speculated the best allocation and configuration of the service types by examining when and how each service type would become activated.

Recipe Recommendation As recipes seem effective information to a shopper’s comprehensive planning, they will tend to be used in an earlier part of the process, i.e. recipes for a main dish should be activated in the first phase, and those for side dishes in the second phase. Those for yet another dish, e.g. snack, appetizer, and dessert, may be effective in the third phase though.

Product Recommendation Although product information seems valuable throughout the process, each phase will favor

Lovely Place to Buy!

different kinds of information. In the first phase, information regarding a sale on fresh food may help a shopper to develop his/her plan for a main dish. In the second phase, information which will enable a shopper to compare prices, qualities, and features for an array of choices may help a shopper to implement his/her plan for a main dish as well as for side dishes. In the third phase, information regarding new and luxury goods may help satisfy his/her curiosity about grocery items.

Location and Preference Based Recommendation While the previous two service types aim at communicating from the retailer, information taking account of a shopper’s individual context, including where he/she is and what he/she prefers, may contribute to provide more appropriate information to a shopper him/herself. We assume that this service type will take an active role especially in the third phase by itself as well as will exhibit the multiplier effect by linking up with the previous two service types throughout the process.

Bookmark Functions This service type will complement the previous three service types. Since it aims at helping a shopper to return to information which he/she saw earlier in the process, it should be accessible at any time.

Scan-to-Deliver Information This service type will make an article itself as a trigger to retrieve its detailed information by using a handy scanner attached to the cart together with the items barcode. It may be especially useful from the second phase onward because it will provide a shopper with information which enables him/ her to choose and evaluate articles. Aside from the five service types mentioned in the foregoing, shopping list management shall contribute much to a better grocery shopping

experience, we, however, did not include it in our principal service types since there have been a number of its prototypes implemented and tested already.

EXPERIMENT We developed a mobile kiosk terminal embedded with a shopping cart, named SmartCart (See Figure 5), as a working prototype system. SmartCart equipped multiple sensor-units which could measure a number of metrics including location, speed, and angular velocity from which we could infer a user’s state in real time. We also developed the five types of services as we described in the previous section; those were ‘recipe recommendation,’ ‘product recommendation,’ ‘location and preference based recommendation,’ ‘bookmark functions,’ and ‘scanto-deliver information.’ Among the types of services, ‘recipe recommendation’ and ‘product recommendation’ were the ones providing shoppers adequate information according to the characteristics of each phase, e.g. recipes for a main dish were activated in the first phase, those for side dishes in the second phase, and those for yet another dish, e.g. snack, appetizer, and dessert, in the third phase. We then conducted an operation test at the project site for about two weeks in September 2006. Fifty customers used the system during their shopping trips, and twenty among them cooperated with our participant observations.

RESULTS By examining the informants’ access logs, we found that there were much smaller numbers of service access in the first phase than those in the subsequent phases (See Table 1). ‘Recipe recommendation,’ ‘location and preference based recommendation,’ and ‘Scan-to-deliver informa-

61

Lovely Place to Buy!

Figure 5. SmartCart: a working prototype system developed according to the five service types described in the previous section

tion’ were the popular services both in the second and the third phase. It was an unexpected result for us that the numbers of service access in the third phase were competing with those in the second phase; we had anticipated that the numbers in the third phase should be the smallest since shoppers would not need any help in this phase. As we got especially interested in the third phase, we tried to compare phase-durations between those in the first participant observations in 2005 and those in the operation test (See Figure 6). The lower bar chart denotes the average of the participant observations and the upper chart

denotes the average of those in the operation test. There is a simple main effect in between both the third phases. Then we divided 20 informants in the operation test into two groups as 10 active and 10 inactive users by referring to their numbers of service access. Figure 7 shows the comparison between the active and the inactive users, and an ANOVA shows that there is also a significant difference in between both the third phases. From the results so far, we could say that SmartCart contribute to lengthen the third phase.

Table 1. Numbers of service access in each phase (n=20) Service type

1st phase

2nd phase

3rd phase

Recipe

7

18

18

Product

4

6

7

Location

2

21

11

Bookmark

0

7

9

Scan-to-deliver

1

19

19

62

Lovely Place to Buy!

Figure 6. A three-way ANOVA under the two conditions; comparison between the first participant observations and those in the operation test

Figure 7. A three-way ANOVA under the two conditions; comparison between the active and the inactive users

63

Lovely Place to Buy!

DISCUSSION What are the key features of the third phase in terms of the system usage? We found that there were major differences in characteristics between the second and the third phase; the tendencies in utterances of the grocery shoppers highlighted the differences. Typical utterances in the second phase were as follows: •

•

•

“I’ve got interested in what kind of ingredients would be used if I buy those (recommended) groceries.” “This (system) gives me a good advice when I’m unable to make a quick decision in between similar items.” “I’m curious about which tofu and milk are the top sellers here in this store.”

The above utterances suggested that each grocery shopper sought information for her better decision-making, i.e. she proactively utilized the system to identify ‘best-buys’ according to the plan she had made so far. While on the other hand, the following utterances characterized the third phase: •

• •

“This (menu featuring spirinchus) looks attractive to me, but I don’t follow this recipe today because I’ve already fixed a plan.” “I’m so curious about this (pudding)...it’s so expensive even though it’s so tiny!” “I’ve got curious about this (yogurt) because it is uncommon for me...I could not find out the one I usually take.”

These utterances suggested each grocery shopper tended to gratify her own curiosity about individual items and/or novelty information by using the system. This implies that there is another chance for grocery retailers to increase numbers of item sold per a customer as well as to make fun of her at the same time because grocery retailers

64

seem to have overlooked the existence of the third phase itself. We, therefore, believe that ubiquitous computing systems like SmartCart will be able to contribute much to the retail businesses by paying attention to the third phase.

CONCLUSION We have reported the design process as well as the results of the experiment aiming at shopper’s decision aid at a physical grocery store by introducing a ubiquitous computing system embedded with shopping carts named SmartCart. We discovered that a grocery shopping process is basically consisted of three phases and utilized the process to develop the system. We, then, conducted an operation test and learned that the third phase could work for increasing numbers of item sold per a customer as well as making fun of her at the same time. Our experiences could give a couple of important implications while we design ubiquitous computing systems not limited to grocery shopping-aid systems. That is, •

•

Cognitive process is not monotonous User’s contexts when computing services, especially digital media services, could exert an effect are limited. It is, therefore, valuable to identify when user’s peak experiences exist, Grasping higher cognitive states are valuable: Recognizing user’s higher cognitive states are useful as inputs to the system, as our research has suggested. It is a future work for us how we could know those states efficiently and stably though.

As Abowd and Mynatt stated, one of the most major motivations for ubiquitous computing is to support the informal and unstructured activities typical of much of our everyday lives (Abowd and Mynatt, 2000). Through research in applications

Lovely Place to Buy!

at real retail outlets, we hope we will be able to contribute toward such the difficult problem.

ACKNOWLEDGMENT We would like to extend our deepest gratitude to Ms. Satoko Horikawa. Without her dedicated contribution including creating the contents of the services, we could not realize the research. We also would like to express our appreciation for Bonrepas Corp. She had been generous enough to provide a store for our longitudinal research base.

REFERENCES Abowd, G. D., & Mynatt, E. D. (2000). Charting Past, Present, and Future Research in Ubiquitous Computing. 2000. ACM Transactions on Computer-Human Interaction, 7(1). doi:10.1145/344949.344988 Beyer, H., & Holtzblatt, K. (1998). Contextual Design: Defining Customer-Centered Systems. San Diego, CA: Academic Press. Kalyanam, K., Lal, R., & Wolfram, G. (2006). Future Store Technologies and Their Impact on Grocery Retailing. In Krafft, M., & Mantrala, M. K. (Eds.), Retailing in the 21st Century. Berlin Heidelberg, Germany: Springer-Verlag. doi:10.1007/3-540-28433-8_7 Kourouthanassis, P., & Roussos, G. (2003). Developing Consumer-Friendly Pervasive Retail Systems. IEEE Pervasive Computing / IEEE Computer Society [and] IEEE Communications Society, 2(2). doi:10.1109/MPRV.2003.1203751 Newcomb, E., Pashley, T., & Stasko, J. (2003). Mobile Computing in the Retail Arena. In Proceedings of Conference on Human Factors in Computing Systems. New York: ACM Press

Roussos, G. (2004). Building Consumer Trust in Pervasive Retail. In Proceedings of International Workshop Series on RFID. Retrieved from http:// www.slrc.kyushu-u.ac.jp/ rfid-workshop/roussospaper.pdf Takahashi, I. (1999). Shouhi-sha Koubaki Koudou: Kouri-Marketing he no Shazou. Tokyo, Japan: Chikura Shobou. (in Japanese) Tamura, H., & Sugasaka, T. (2007). Harmonizing Human Eyes with Digital Sensors.In Proceedings of The Third International Ethnographic Praxis in Industry Conference, University of California Press, Berkeley, CA Tamura, H., Sugasaka, T., Naito, H., Sekiguchi, M., Horikawa, S., & Ueda, K. Exploring Everyday Activities for Pervasive Decision-Aid. 2006. In Proceedings of PICMET’06, Portland International Center for Management of Engineering and Technology, Portland, OR. Thomas, A., & Garland, R. (1996). Susceptibility to Goods on Promotion in Supermarkets. Journal of Retailing and Consumer Services, 3(4). doi:10.1016/0969-6989(95)00095-X

ENDNOTE 1



The leaders of grocery shopping in Japan in this era are married women having no regular employments, notably housewives. About 80 percent of them answered to our survey that each of them shopped at a specific grocery store at least once a couple of days. This implies that the major objective of her grocery shopping was not to do bulk buying but rather buy things for her meal arrangement for that particular day. Under these cultural contexts, we should note that our research focused on the shopping process of a housewife with which she arranged her dinner on a day-to-day basis.

65

66

Chapter 4

Portable Personality and its Personalization Algorithms: An Overview and Directions Stefan Uhlmann Tampere University of Technology, Finland Artur Lugmayr Tampere University of Technology, Finland

ABSTRACT With the advances in ubiquitous computing, there is an increasing focus on personalization of user information especially in web-based applications and services. Currently those personalized user profiles are scattered, mostly stored for each individual service. Therefore, this prohibits the usage of those profiles in different environments such as other web-based services, shopping in local stores or sharing interests among people. The so-called Portable Personality focuses on the management and distribution of personalized profiles (in form of a digital personality representing the real-world user) through mobile devices. These portability aspects merge with the idea of cross-system personalization using a single generic user profile. We will briefly introduce some aspects related to profile representation and management with focus on attempts towards such a generic representation. The main discussion will be concentrated around profile portability and its effects on personalization especially towards crosssystem support. We include different portable profile scenarios and their personalization methodologies. Furthermore, current personalization algorithms are considered with possible associations towards the presented portable scenarios. At the end, we reflect on existing challenges of current approaches in the field of portable personalization and try to provide some recommendations. DOI: 10.4018/978-1-60960-774-6.ch004

Copyright © 2012, IGI Global. Copying or distributing in print or electronic forms without written permission of IGI Global is prohibited.

Portable Personality and its Personalization Algorithms

Figure 1. Complexity of user profiles

USER PROFILES AND THEIR MANAGEMENT Personalization of any kind of information is evolving in a rapid manner especially for webbased applications and services whether in advertisement, search engines, online shopping, or social networks. Hence, the collection and application of personalized information is currently omnipresent. We understand personalization as tailoring and providing content and services to individuals and groups based on knowledge about their preferences and behavior. This can range from simple superficial factors such as custom ring-tones to the complex tailoring of the presentation of a shopping web site to a user’s personal interests and their previous purchasing behavior. To make use of such information, a so-called ‘personalized user profile’ (UP) is to be generated preferably without user intervention. Such user profile may vary from a rather simple to complex representations depending on how much and what type of information is gathered and stored. A simple relation is pictured in Figure 1. An example what kind of information might be stored in a rather complex user profile is illustrated in Figure 2. (Note that in the annual personalization survey from www.choicestream.com in 2009, personalization and recommendations are well received and considered useful to make purchases. However, they also found that the quality of recommendations decreased to previous year 2008 as well as

recommendations can widely vary depending on different retail categories.) The general concept is to gather user-specific information about the user (profiling), to manage and store this information (content management), distribute it to consumer applications or services (profile distribution, portability), and finally extracting those pieces of information valuable to the consumer current needs (profile evaluation, personalization). Current UPs are mainly used in web applications to personalize searches, advertisements and shopping recommendations such as music, movies, and books. Most of the time, the user has a different profile for every online shop, service, or website such as last.fm (http://www.last.fm – recommendations for music such as songs, videos and concerts) or FOAF (http://www.foaf-project.org/), which makes them mostly application-dependent. The project of OpenID (http://www.openid.net – supported by big players such as Google, Yahoo, Flickr, MySpace, Facebook, WordPress, AOL) tries to overcome this problem but it is more related to a single digital authentication identity across the Internet. Furthermore, the access and usage of those profiles is limited to a device connected to the Internet. This mainly means that, on the server-side, the user does not have much control over the information gathered about him or her and due to multiple profiles for different services, this leads to a fragmented personalization experience. Schuurmans (2004) already believed in the need for a cross-domain profile that is under control

67

Portable Personality and its Personalization Algorithms

Figure 2. Information stored in a complex user profile

of the user. However, the recently started data portability project (http://www.dataportability. org) tries to develop a standard that allows users to gain control over their own data again. Now imagine pushing this one step further, what happens if the user visits a local music, video, or book store? Its preferences about music, videos or books have not changed. Therefore, users should be able to carry around their own electronic UP in a portable manner instead of storing multiple UPs in a decentralized way for each website and interest. That shows, for this profile scenario, mobile devices such as phones and PDAs, which are nowadays so-called “smart” and multifunctional, provide an optimal platform for managing UPs in a single user-central place. Considering this, they can be easily distributed when necessary. However, portability does not solve the question how to manage the UP on the device itself. As in real life there is only one “you” combining all your associated interests,

68

preferences, and behaviors. This should ideally also apply for automatically generated UPs so that there is only one single UP representing the entire user’s personalized information about its likes and dislikes as illustrated in Figure 2. Therefore, this could be seen more as a digital personality rather than a user profile. Modeling such user behavior is a dynamic and eventually lifelong process. This arises some challenges in the procedure of user modeling on how to handle interoperability, scrutability, and privacy. Interoperability is the exchange of user profiles across various sources in a distributed environment. This can only be achieved by developing and adopting explicit and widely accepted protocols so to enable the discovery and exchange of user models, stored in various systems. Scrutability ensures control of the user over its own data how and what has been modeled. It further allows changing stereotypes and preferences and the way in which conclusions are inferred from

Portable Personality and its Personalization Algorithms

Figure 3. Sub-profile example of user profile representations

these data. Also, privacy considerations should be taken into account such as the Minimization, Consent, Openness, Access, and Accuracy principles stated by Kobsa (2007, p.30). The process of acquiring this personalized information can range from manually entered to fully automatically generated data for various types of preferences. Hence, there is a necessity for methods and approaches to combine and merge multiple UPs into one single UP in a smart and efficient way. Nowadays, there is vast number of services using their own UP type to store knowledge about users. Most commonly is to store all related user preferences for each single user individually (He, 2007; Chen, 2007; Amazon). The type of information gathered and stored will depend on the application, service and applied domain. Thus, single user preferences may contain just identity information (OpenID), one specific domain-dependent preference such as music (including songs, videos, and concerts see last. fm) or multiple preferences such as music and movies as in (Chen, 2007). Such profiles for users and domains can, of course, interconnect as seen in large online shops such as Amazon, where one user may have preferences in book, music, and movie domains. Yet another possibility is to represent the behavior and preferences of groups (Shtykh, 2009) or stereotypes (Castellano, 2007). While combining different user information, the UP can be divided into static and dynamic parts as done by (Magoulas, 2006; Yu, 2005;

Papadogiorgaki, 2008). On the one hand, the static part mainly includes all fixed activity or interest-independent user information, which does not change regularly, such as name and address. On the other hand, the dynamic part contains all the information and preferences about current activities and interests, which do evolve and need to be updated more frequently. Moreover, the dynamic part can further be split into short-term and long-term interests as done in (Papadogiorgaki, 2008; Park, 2009; Zhuhadar, 2009) where short-term profiles specify current interests which might change rather frequently and long-term profiles relate to more general interest which is also subject to change but slow and gradually over time. (Park, 2009) adds further differentiation between recent interests and most current interests to represent the UP according to recency, frequency and persistency. This also illustrates the different aspects of how to store certain parts of the user preferences. Short- and long-term profiles are one option; another is to either store them in different sub-profiles (Sutterer, 2007), personas (Gosh, 2007) or activities (Yu, 2005) as shown in Figure 3. The idea of profiles is obviously not new, and various profile types, management approaches and initiatives exist such as 3GPP Generic User Profile (3GPP:GUP - http://www.3gpp.org), Composite Capabilities / Preference Profiles (CC/ PP - http://www.w3.org/Mobile/CCPP/), and Open MobileAlliance - UserAgent Profile (OMA:UAProf

69

Portable Personality and its Personalization Algorithms

- http://www.openmobilealliance.org). Yet due to their network relations, they are not really relevant for user modeling linked to portable personalities. However, over the past years, research has increasingly focused on personalization and representation of user preferences and interests in a compact and efficient but extendable and machine readable form. An earlier approach was made by the European Telecommunications Standards Institute (ETSI) Specialist Task Force (STF) 265 to describe a standard on user profile management. Their finished document on “User Profile Management”, ETSI (2005), proposed that further work in this area is necessary to produce standardized user profile components that will help to provide the optimum user experience. People have been working on it since to achieve such an experience. Probably one of the oldest and simplest form of information representation is the Vector-Space model, where UP is a vector containing representative keywords or terms with associate weights. It is still widely used (Castellano, 2007; Zhou, 2007; Yu, 2006) due to its simple nature. Currently the most popular method in relation with the semantic web is the application of ontologies. An ontology can be defined as a formal representation of a set of concepts within a domain that provides a shared vocabulary, which can be used to model this domain including the type of objects and/or concepts that exist as well as their properties and relations among each other. In the context of user profiles, ontologies can represent and organize user information, their context and relationships more accurately especially considering the necessity of dynamic preference and interest changes. Furthermore, it offers an easy expandability by merging, expanding, and combining parts of existing ontologies into new ones. There are mainly two-forms for ontology representations also relevant for UPs, domainand foundation ontology. In domain ontology, the user preferences are commonly described in form of an interest hierarchy directly related and

70

based on the observed user behavior (Kim, 2003; Zhou, 2006; Anand, 2007; Sendhilkumar, 2008; Nakatauji, 2009). In foundation ontology, a model is described which unities common objects that are generally applicable across a wide range of domain ontologies. Therefore, it normally provides a core glossary to describe common objects in a set of domains. Over the years, a few attempts such as (Golemati, 2007), SOUPA (Chen, 2004), UPOS (Sutterer, 2008), GUMO (Heckmann, 2005) have been made to define such a standard mainly in the form of foundation (or upper) ontologies by employing modular structures which are extendable by referencing existing ontologies or vocabularies for particular concepts such as beliefs, desires, intentions, time, space, events, user profiles, and actions. A general idea of the structure is visualized in Figure 4. Considering this, the key goal is to focus on the basic user model dimensions and leave the general world knowledge to existing ontologies such as SUMO (http://www. ontologyportal.org/) and UBISWORLD (http:// www.ubisworld.org/). SOUPA was a first step in the right direction. Anyway, the authors of (Villalonga, 2009) think that it lacks consideration of users’ needs and support of mobile services and applications. Too overcome these limitations, a Mobile Ontology (http://ontology.ist-spice.org/) as part of the IST project SPICE was introduced. They extend the SOUPA approach by linking subontologies through a minimal core ontology from which all the sub-ontologies inherit. Already crucial subontologies are defined but it is clearly anticipated that further sub-ontologies will be defined to cover the mobile domain more comprehensively. A recent framework, which follows the GUMO approach building upon the notion of subjectpredicate-object statements, is the Grapple User Modeling Framework (GUMF) by (Abel, 2009). It specifies a common structure and language to provide user preferences, user observations, and user model representations within the modeling

Portable Personality and its Personalization Algorithms

Figure 4. Upper ontology separated into core and extension ontologies

infrastructure. GUMF aims to support various systems and integration of new kinds of statements and derivation rules within the user model format. However, in recent years, there has been an initiative named Attention Profiling Markup Language (APML - http://www.apml.org) which allows users to share their own personal Attention Profiles. The goal is to combine all types of Attention Data (blogs, browsing, music, photos, social networking) into a portable file format which then describes user preferences ranked by interest. These Attention Profiles are stored in such a way that computers and web-based services are able to handle and process them. APML has not been widely adopted yet but is regarded as a step in the right direction. The two most famous web site applying APML are probably Digg (http:// www.digg.com) and BBC (http://www.bbc.co.uk/ blogs/radiolabs/). At Digg, the generated Attention Profile is based on the categories a user was

interested the most over the past 30 days. At the BBC, their Radio Labs Pop service started in 2008 to allow users to export APML files based on their radio listening behavior. In (Niederée, 2004) an ontology-based unified user context model (UUCM) is presented which describes the relevant dimensions of the user and its working context(s). This approach uses the metaphor of a context passport that accompanies users on their travel through the information space. UUCM is developed concerning cross-system and cross-service application enabling improved support for multistep information activities. Other earlier or recent approaches such as Description Logic (Sinner, 2004), concept lattice (He, 2007; Kwon, 2009) or tag clouds (Pessemier, 2009) a valid research efforts but might not make it to widely accepted standards for user modeling. We can see that there have been many approaches to describe UPs and their data. So far,

71

Portable Personality and its Personalization Algorithms

there is no standard yet which defines a general or generic user profile which stores any type of interest or preference that can be used across systems and where the user is in full control over its data.

Management Before being able to use an UP and its represented user knowledge, this information has to be acquired and probably maintained first. Obviously, this part of information acquisition and management is not a simple task and there exist various methods and techniques to handle them. We will mention and focus on some more current approaches of the past few years. For information regarding general and earlier approaches we refer to the additional reading material. Generally, there are three major information acquisition methods for user profiles, which could be ranked by user interaction. Firstly, there is, of course, the manual way where the user provides its information directly; usually done in form of a brief questionnaire or survey. This might be acceptable for basic information rather than complex interest. However, the user will get bored and annoyed when she or he has to provide the same or similar data again for different services. Therefore ideally, this should be done only once such as OpenID for logins. Secondly, the information can be gathered from explicit data either in the form of documents such as web pages or feedback. Here, the user provides data, which represent his interests, but the actual extraction is done automatically (He, 2007). Thus, it could be seen as a semi-automatic technique. Finally, implicit methods focus on the automatic acquisition of user information by observing, imitating, and recording user’s behavior. However, it is not uncommon to combine explicit and implicit methods as done by (Zhou, 2007) and (Padadogiorgaki, 2008). Over the past years, researchers have introduced various information acquisition methods using a variety of data to gather from. The most

72

common one nowadays is the Internet especially the World Wide Web by using server-side (Sugiyama, 2004; Castellano, 2007; Papadogiorgaki, 2008) and client-side methods (Sendhilkumar, 2008). However, client-side acquisition became more popular by using web-browsing behavior related to page visits, time spent, and page length (Kim, 2003) or direct browser actions such as bookmarking, saving, printing as importance feedback (Sendhilkumar, 2008). Others (Chen, 2007) have also included private data such as calendar, schedule, email information or applied meta-data and tags while bookmarking (Michlmayr, 2007). A non-web approach of preference acquisition is used by (Reymann, 2007), where the way a user listens to its music on the computer is used to generate a music profile, which can be stored on a mobile phone for further use. This is extendable to other preferences as well. It should be understandable that just any cluster of acquired data does not provide meaningful user information or make a UP what so ever. Hence, the underlying hidden information out of that data chaos needs to be unraveled first and related to user’s interests to generate meaningful UPs. Eventually managing and maintaining UPs by evolving and updating them due to interest changes is a key component following the initial data acquisition and UP generation. We describe some general ideas by presenting the concepts assuming they could also be applied to any other domain. A popular and effective method is the use of clustering schemes such as fuzzy, hierarchical and conceptual clustering. (Castellano, 2007) used fuzzy clustering to combine similar interests of multiple users into groups and (Han, 2009) combines fuzzy clustering techniques with optimization techniques to construct ontology-based user profiles (FCOU). (Kim, 2003) implicitly learnt a user interest hierarchy based on user behavior where hierarchical clustering algorithm groups words (topics) into a hierarchy. This is an analogous approach as to build a subject taxonomy

Portable Personality and its Personalization Algorithms

for a book catalog in a library and then assigning books to the taxonomy. A more current scheme of hierarchical clustering is used in (Nasraoui, 2008). They employed a hierarchical version of an Unsupervised Niche Clustering (H-UNC) that used a Genetic Algorithm to evolve a population of candidate solutions through generations of competition and reproduction. Another web document clustering approach is presented in (Godoy, 2006), named Web Document Conceptual Clustering (WebDCC). It carried out incremental, unsupervised concept learning to generate user profiles as conceptual hierarchies. Non-clustering approaches have been explored as well. Formal Concept Analysis (FCA) employed by (He, 2007; Kwon, 2009) describes a lattice which consists of concepts and their weights that express how much that concept supports a certain topic. Their assumption for the weight calculation is: The more similar a concept is with other concepts in the lattice, the more the concept supports the topic. (Magoulas, 2006) illustrated another approach, where a Fuzzy Analytic Network (FAN) process is employed seeing user preference extraction as a multi-attribute decision making problem. (Marghny, 2006) focused on an adaptive system for learning the user profile, the dynamics and the rate of change of the user interests. This technique employed genetic algorithm for adapting to the user interest relying on user feedback. In (Sieg, 2007), a Spreading Activation algorithm is used to incrementally update the interest score of the ontological user profiles concepts. As the user interacts with the system by selecting or viewing new documents, the user profile is updated and the annotations for existing concepts are modified. The approach of personalized news content in (Papadogiorgaki, 2008) focused on a two-level learning process, which is employed on a mobile device side to automatically update the general and specific profile models. It involved the use of machine learning (ML) algorithms applied to the implicit and explicit user feedback. As ML algo-

rithms they used a weight adaptation depending on whether the user selects or ignores news items. (Zhuhadar, 2009) also employed ML techniques to detect user convergence within a lower-level semantic UP gathers. A higher-level semantic representation keeping track of the user’s general interests is used to detect shifts in the user activities which are then used to automatically update the overall user profiles.

USER PROFILE AND PERSONALIZATION Once the UP is generated and in a state of future updates and evolution, it can be used for its sole purpose to customize services by making product or service recommendations personalized to a particular user based on its UP interests and preferences. This personalization is achieved by personalization algorithms also called recommendation approaches. Parts of this work in the next chapters are based on an earlier work by (Uhlmann, 2008) and are updated with new current personalization approaches and concepts regarding profile portability and cross-system profiling. They can be classified into three main categories: content-based, collaborative, and hybrid recommendation system. Considering content-based ones the user will be recommended items similar to the ones the user preferred in the past. However, there are three general drawbacks of such system. Firstly, since only the content is analyzed it depends on the associated features related to the items the system recommends. Secondly, there is the challenge of overspecialization because the system can essentially only recommend items that score highly against an UP and, therefore, the user is limited to being recommended items that are similar to those already rated, used or bought. Finally, the biggest issue is the so-called new user problem. How does recommendation work when a new user enters the system? The challenge is the new user has to rate sufficient items before such a

73

Portable Personality and its Personalization Algorithms

content-based recommendation system can really understand the user’s preferences and present the user with reliable recommendations. Therefore, a new user, having very few ratings, would not be able to get accurate recommendations. In a collaborative recommendation system the user will be recommended items that people with similar tastes and preferences liked in the past. Obviously, this is not problem free either and one can also identify three drawbacks. First, there is the same new user problem, where the system must first learn the user’s preferences from the ratings that the user provides, in order to make reasonable recommendations. Second, there is also a new item problem. Since new items are added to such recommendation system on a regular basis, the system will not be able to recommend the new item until many users have rated it. This is due to recommendations on just user preferences. And thirdly, we have the so-called sparsity which depends on the availability of a critical mass of users. When having items only rated by a few users or users with a rather unusual taste compared to the mass, it will lead to rather rare or poor recommendations by the system. The authors of (Rafter, 2009) have explored the characteristics within collaborative predictions and one major implication regarding their observations is the importance of developing new algorithms that offer prediction improvements on extreme ratings because users need to receive reliable recommendations containing items they strongly like and avoiding items they strongly dislike. To use the advantages but overcome certain drawbacks of the content-based and collaborative recommendation systems, hybrid approaches combine both methods to make better recommendations. There are, however, different ways to combine them into a hybrid system. 1. Implementing collaborative and contentbased methods separately and combining their predictions

74

2. Incorporating some content-based characteristics into a collaborative approach 3. Incorporating some collaborative characteristics into a content-based approach 4. Constructing a general unifying model that incorporates both Regardless of the type of recommendation approach or system, it is important in personalization to avoid the problem of so-called tunnel vision. This principally means to focus too much on the main and most dominant user’s interest and preferences. This may narrow down exploration or in other words, the importance of serendipity providing recommendations outside of the main user’s interest space. It is essential to also “discover” new topics which might interest the user based on its current preferences. One option in that direction could be to present all found choices and highlight recommendations. Now the user can still explore the newly generated information space and might find something interesting but unrelated to current preferences and interests while browsing the results. So far, there has been no superior recommendations approach. Even though hybrid systems have shown high potential to make good recommendations, the underlying techniques and algorithms are still mostly dependent on the applications or task at hand. Most of the personalization techniques are related to information acquisition of user’s web behavior or computer interaction. Thus, this means that the application of the constructed UPs is mainly limited to the web or computer. To overcome this limitation, the focus has been shifted towards portable profiles. This shift is confirmed by an evaluation of user expectations in (Brugnoli, 2005) where the most popular idea among participants was the possibility of using a so-called “Simplicity Card” containing user profile and personal data in conjunction with a mobile phone. Most participants saw it as a “Personal ID” and as a kind of an extension of themselves.

Portable Personality and its Personalization Algorithms

On scenario to achieve portability is by mobile devices, which are able to store the profile data on the medium itself. Nowadays, mobile devices such as phones, PDAs or handhelds are often used to store UPs (Bartolomeo, 2006; Ghosh, 2007; Papadogiorgaki, 2008). Other device-like approaches including smart cards (Potonniee, 2002) or flash drives (Liffick, 2007) have been investigated as well. To extend the mobile only scenario, distributed UPs (Ghosh, 2007; Papadogiorgaki, 2008) are introduced by keeping service-related profile information on the service side besides the mobile device profile part. Of course, there is also the option of a centralized scenario where the UP is kept completely on a server and obtained on request (Ankolekar, 2006). Note that many user profile approaches mention application of mobile and portable devices but they never state how the exchange and usage of the profile is actually carried out or performed. There have been different approaches investigated to achieve portability. One of the earlier approaches is the “Digital Aura” (Ferscha, 2004) which considered profile portability, handling and storing in a mobile device. Those profiles would be exchanged and compared via Bluetooth when the devices are in close vicinity of each other. Another Bluetooth-based approach is presented in (Bartolomeo, 2006) where a UP stored on a mobile or portable device was inspired by 3GPP:GUP to create a Simplicity User Profile (SUP). The SUP data are viewed and edited on the device itself and the profile is intended to adapt services, applications and networks. In (Potonniee, 2002) a smart-card approach is introduced and demonstrated on an example in the context of interactive TV where collaborative personalization is realized on server side and the individual personalization on smart card side. Anyway, the introduced approach is generically applicable to any application. However, it can be seen as a disadvantage to store the UP on a smartcard since that means there is always a need to have a smart-card reader at hand whenever one wants to used its profile.

The BlueCard approach in (Ghosh, 2007) uses again Bluetooth enabled mobile devices. These devices use the OBEX Object Push Profile standard (OPP) which is amongst the most widely implemented Bluetooth specifications on mobile phones. Thus, this allows for the transfer of high-level objects between devices. For storing profile information they employ the vCard format (VCF) natively supported by all OPP devices. A VCF object is a structured collection of properties that may include not only information usually found on a business card such as names, addresses, telephone numbers but also other types of information describing resources such as audio, graphical objects or geo-positioning data. So in the proposed BlueCard approach, they create a new BlueCard on the mobile devices whenever a new service is been used otherwise an already existing BlueCard is used for authentication. The main idea is to use the BlueCard to assert general preferences and information about the user, which can then be combined with service-specific user profiles that are maintained at the service end. This approach was demonstrated as an implementation on the HP Labs Retail Store Assistant kiosk. So far, most user modeling and profiling approaches were specific to the task at hand. However, the ultimate goal of this process should be to separate user modeling from applications to make gathered information reusable across applications. An approach in support of cross-system personalization is investigated and presented in (Niederée, 2004) and (Mehta, 2005, 2006, 2007). The Unified User Context Model (UUCM – context passport) provides a basis for the realization of cross-system and cross-service personalization approaches that enable the exchange and reuse of user profiles scattered across multiple systems. The interaction between the user and the information system (IS) using the context passport can be summarized as follows. The user presents its context passport to an IS. The IS can then interpret the user’s requirements and activities supported. The relevant context-of-use is extracted and activities

75

Portable Personality and its Personalization Algorithms

are “transformed” according to that context-of-use. Now the IS can perform the supported activities based on information derived from the context passport. The user interaction feedback from the IS is used to update the context passport and keep it up to date. For such a cross-system personalization approach, it is assumed that the user context-meta model is publicly available as a shared ontology. All participating systems rely on (and need access) to this model. The exchange of such information requires a negotiation between activities that an IS can perform and those activities that the user context outlines. Hence, cross-system personalization needs to address 1. broader user models that can cope with the variety in user modeling, 2. handling heterogeneous personalization systems and approaches, and 3. giving more control to the user, which are all related to a generic UP idea. A quite interesting cross-service approach (Reymann, 2007) with extensions and applications (Bruns, 2007; Lugmayr, 2009) is the Portable Personality (P2) Project. The main idea is to carry a XML-based UP (representing more like a digital personality) on a mobile device which can then be used to personalize services. They introduce a framework which provides a platform for cross-service interchange of personal context information based on any generic metadata type. This framework architecture is designed for mining, enriching, and exchanging XML-based personal profiles between arbitrary multimedia services to allow • •

76

integration of multiple service specific metadata formats into one P2 profile, exchange of metadata across devices and services to accomplish a seamless service and getting rid of all the single services,

•

and support of sophisticated mining and personalization algorithms to gather and evaluate personal profiles.

The ultimate goal of P2 is to handle a portable personality profile rather than a common user profile as mainly used today. The overall framework is divided into four main parts, namely P2 Provider, P2 Service, AmbiNET, and P2 Consumer. The P2 Provider is responsible to gather metadata from all types of sources. However, within the framework it is not specified how this is done and, thus, is up to the Provider to apply appropriate algorithms for acquiring such metadata. The P2 Services are responsible to manage and merge the context metadata into a personal context profile acquired by the various Providers. AmbiNET is the communication component within the framework allowing information being exchanged through various technologies such as Bluetooth, IPnetworks, Internet, Infrared, or Wi-Fi according to availability. P2 Consumers provide the actually personalization based on the obtained personality profile. As for Providers, the framework does not specify how this is can or has to be achieved. It is up to the application on the Consumer side to implement a suitable personalization algorithm. Within their framework, a mobile device is only seen as a carrier of the UP between different application services. An interconnection between the different parts is shown in Figure 5. Within their framework they developed a socalled personality profile life cycle including stages such as Aggregate, Carry, Use, and Enrich. A more detailed description regarding this life cycle and the actual distribution of profiles between different entities can be read in (Lugmayr, 2009, p.192-193) and (Bruns, 2007, p.36-38). Considering their sample scenarios described in (Lugmayr, 2009, p.195-198), this is one of the first approaches taking into account automatically PC-generated UPs in traditional shopping context.

Portable Personality and its Personalization Algorithms

Figure 5. P2 AmbiNET interconnection between different applications

Supporting the idea of portable profiles, there is a common tendency nowadays to have distributed profiles (Ghosh, 2006; Papadogiorgaki, 2008) where different parts or profiles are stored on the mobile device and on a server or service-side. This is mainly to separate general and service specific information from each other. For example in Papadogiorgaki, 2008) a detailed user profile (short-term interest) for the news domain is maintained on client, a mobile device. Longterm interest, however, are stored on server side. They do not explicitly mention anything about moving profiles between different devices (e.g. mobile - PC) but say that it is easily applicable / extendable to other platforms.

As mentioned earlier, most UPs are stored on the server-/service side, which, obviously, limits portability if the UP is tied to the service or server it is stored on. However, if there would exist a centralized UP, using its Unified Resource Identifier (URI) might provide the desired portability while keeping the server-sided approach. This idea has been employed by (Ankolekar, 2006), where the HTTP GET method was extended by a parameter containing the URI of the user’s FOAF profile. A more generic approach based on this could be a portable profile on a mobile device which just contains a centralized URI pointing to the actual profile that is acquired from the used service up on request.

77

Portable Personality and its Personalization Algorithms

CROSS-SYSTEM PROFILE PORTABILITY Personalization and profile portability efforts so far have been mainly service or application dependent. Cross-system approaches such as UUCM and P2 are attempts to bring light to the jungle of proprietary and application dependent personalization by providing ideas for independent and generic frameworks. As mentioned before portability requires a carrier generally a mobile device to carry the profile around between different application services. For this reason, portable personalization can be divided and classified into three main scenarios regarding where and how the profile is stored and where the actual recommendation process takes place. We refer to them as: mobile device side, distributed between mobile and service provider, and centralized. We will describe the ideas behind them including drawbacks and advantages. Then we review some current personalization and recommendation system approaches and investigate their applications with respect to the described scenarios.

A. Distributed Scenario: Mobile Device (MD) – Service Provider (SP) Here parts of the UP are stored on the user’s mobile device and the service provider end. This can be further divided into three sub-scenarios depending on what information is stored where and how it is used.

1. MD (General) – SP (Specific) In this scenario, the mobile device stores the static and/or general preferences of the user whereas the SP creates a specific profile. The use case scenario could be seen as follows. A user uses a particular service (offered by an SP) the very first time. S/he can use its portable UP for a first initialization. After using the service,

78

the SP gathered some more specific user interests and preferences based on the user’s interaction and behavior. After the user made use of this service several times, the SP learns and adapts a specific UP started from the general mobile UP part. The main personalization would be carried out on the SP side where the specific UP information of the users can be utilized to provide detailed recommendations. However, one major drawback of such system is when the user interacts with another similar or different service (online or high-street shops), this generated specific UP stays with that particular SP and is not moved to the mobile device. Therefore, using a new service from a different SP, the procedure to learn and adapt to the user’s interests starts again from the general UP obtained from the MD. An application for such a scenario is mainly authentication purposes and to provide some information to an SP so as to avoid a cold-start with no information about the user what so ever.

2. MD (Specific) – SP (General) The vice versa scenario is when the MD stores the specific UP and the SP just keeps a record of the general main interests. Since all user information is stored on the MD, the actual recommendation system would be embedded into the device itself. This is a particular case where the MD is actively used with the service not just carrying the UP. When a user interacts with a service providing its specific UP, the service would extract and generate general information from it and stores a general UP on its side. Accessing the service with the MD again means the SP would provide new content related to the user’s general interest and on the mobile device side the recommendation system would personalize (rank) the information according to the specific user interests available there. A clear advantage over scenario A1 is that when the user goes to another SP the specific UP is stored on the MD and therefore controlled by the user. Thus, all the information is always avail-

Portable Personality and its Personalization Algorithms

able to the user. Anyway, there are some issues with this scenario. First, if the SP would be able to obtain a specific UP it would probably use it to provide a better service for the user. Because, obviously, just using general information to do some pre-selection or pre-filtering on the service side is not an efficient approach. However, users might prefer this as a security and privacy alternative. Furthermore, the recommendation system on the SP side would not be able to exploit its full power by just utilizing user’s general interest. Recommendations would probably lack accuracy, innovation, and serendipity.

3. MD (General, Specific) – SP (General, Specific) This could be seen as an extension to scenario A2 where the service side also keeps a specific UP. Besides overcoming limitations of scenario A2, this further introduces some other issues. Here we have the assumption that the SP UP can be stored on the mobile side. Otherwise it would be a combination of scenarios A1 and A2. If both sides are able to exchange their UPs then either the user ends up with various service profiles on its MD or more challenging a mechanism is needed to merge and update the SP UP and MD UP. Obviously, the same procedure is needed in the other direction of UP exchange when the MD UP is provided to the SP. Now the SP must also be able to merge or update the service UP based on the MD version which may has changed since the last time to avoid storing redundant information. If no merging or updating procedure is in place, the system would treat the user and its MD UP as a first time user appearance which is similar to the cold-start problem in A1 and it would, of course, add a certain level of complexity overhead. However, note that depending on the merging and updating procedure this might be a favorable approach. Furthermore, a generic or standardized UP would be highly beneficial for such a scenario, which does not exist yet; same

goes for the universal merging procedure, which kind of relies on a generic (standardized) UP.

B. Mobile Device Side: MD (General, Specific) – SP (None) In this particular case the entire UP would be kept on the MD of the user and the service would not use or store any user information. This can be seen as a special case of A3 regarding the coldstart challenge. However, as mention before in scenario A2 it is highly unlikely that a service would not use specific user information given to it to improve service. Therefore, such service system would probably not be used even though it would be favorable from a privacy point of view.

C. Centralized: Third-Party Storage This is a very special scenario of a portable UP. The idea would be to store and maintain a UP at a central point on a network-based server. Now the MD would just contain a reference to the UP server location. When using a service the MD and SP would exchange this URI information and the SP would be able to obtain the UP from that particular location. There would be, of course, privacy and security challenges with such an approach especially adding a new third-party into this process which we do not want to discuss here. However, there are two basic options. First, the SP is only allowed to access the UP as long as the user uses the service. This, however, would suffer from similar issues as scenario A3. The second option could be that the service is granted longtime access. The advantage would be that the SP is able to obtain the UP even though the user does not currently use the service. This may be used to periodically check for UP changes, feed them to the recommendation system and update recommendations. In either option it would require a procedure on the SP side to update and merge local changes back to the central UP. Such an approach would combine advantages from A3

79

Portable Personality and its Personalization Algorithms

regarding service recommendation as well as the portability factor similar to having the specific UP stored on the MD. Further, every SP would be allowed to access and update one single UP. This has, however, the need of a unified or standardized profile representation. Also the merging procedure to handle updates of interest (long/short term) needs to be specified and employed by each SP. Note that scenarios A and C can be locally combined within an SP domain (such as an SP with multiple online representations or high-street shops / branches). This means the UP would be centralized within the local domain of the SP providing up-to-date UPs in each branch, which is certainly employed by major companies. The aforementioned P2 framework can be considered as an A3 scenario where both sides, MD and SP, hold specific UPs which are exchanged, merged and updated. Here, the merging and updating procedure on the SP side handles the different metadata acquired from different sources. SPs are in charge of gathering the needed data and personalize the service according to the UP. Results are fed back to the MD where the user can browse the recommendations provided by the SP. Quite obviously, the profile exchange and merging is the most important part within in this P2 framework and personalization is done on the fly after obtaining the new profile due to merging. This might be highly complex for large content or services with a huge number of users (if a collaborative or hybrid approach is employed). However, within the P2 framework scenario, time might be not that critical since UP synchronization is done automatically after first time usage of a particular SP. Anyway, complexity could be reduced by just using the obtained UP updates during merging for new suggestions. This would be applicable since the rest of the UP is still the same and, thus, recommendations would be kept up-to-date by the SP. Then, existing and new recommendations just need to be merged for up-to-date user recom-

80

mendations based on the previous SP UP and the newly provided MD UP. A description of a scenario A1 approach is given by (Ghosh, 2007). They capture and maintain models of user profiles using semantic web technologies by aggregating and sharing distributed fragments of user profile information spread over multiple services introducing the Semantic User Profile management framework (SUPER). It supports the combination of portable user profiles on a mobile phone or PDA and service profiles. This enables the user to assert general preferences and information about themselves on the mobile device, which can then be combined with service (application, domain)-specific user profiles that are maintained at the service end. Furthermore, they integrate calendar and FOAF information from the user to make recommendations. They mainly use it at retail kiosk locations for identification and then getting a list of offers customize based on their service profile. The applied scenario could easily be swapped with other services providing their specific customizations and recommendations. The vice versa option is presented in (Papadogiorgaki, 2008), where a distributed client-server user profile for personalized news delivery to mobile users consists of two separate models: long-term interests are stored in a skeleton profile on the server and the short-term interests in a detailed profile on the mobile device. The available content is initially filtered on the server to derive a list of recommended items in the general preferred categories, while the matching of detailed user preferences in the mobile device results in displaying items in a ranked order. This is a representation of an A2 scenario and it is assumed that other retail services may be applied instead of the news domain. Similarly to P2, (Ghosh, 2007) and (Papadogiorgaki, 2008) can be considered generic representations of their kind using portable UPs and, therefore, independent of personalization and recommendation approaches. Hence, we want to

Portable Personality and its Personalization Algorithms

present some current personalization algorithms which could be employed in the aforementioned scenarios and illustrated possible benefits and drawbacks when applied to the mobile or service side. One simple form of UP representation is tag clouds (Pessemier, 2009). This is a content-based algorithm that recommends user-generated content with the aid of generally available metadata such as tags and categories. The recommendation algorithm will predict the rating that a user will give to a content item which contains a set of tags. To accomplish this task, the recommender will compare the set of tags T, with each of the tag clouds of the user profile one after the other. Based on these comparisons an obtain similarity value will indicate how the user previously evaluated content items with tags of T. This is used as a basis to predict the personal rating for the particular content item. Due to infrequent occurrence of tags a correction factor derived from the user profile is applied. This personalized correction factor gives less frequent or new tags a fair chance to get into the user profile, which will lead to more varied and novel recommendations for the end user. An interesting hybrid approach (for music recommendation) is presented in (Yoshii, 2006). The method integrates both rating and content data by using a Bayesian network called a three-way aspect model, where a set of latent variables describe substantial preferences. The latent variables represent user preferences; each latent variable conceptually corresponds to a genre, and a set of proportions of the genres reflects a musical taste of each user. A visual representation of the relations in this model is pictured in Figure 6. A possible explanation of this model is that a user (stochastically) chooses a genre according to his or her preference, and then the genre stochastically “generates” pieces and polyphonic timbres. The collaborative part tries to predict unknown rating scores of a target user for musical pieces that have not been rated by the particular user considering

someone else’s scores of those pieces whereas the content-based part ranks musical pieces on the basis of music-content similarity by representing user preferences in the music-content space. They are able to achieve high recommendation accuracy and rich artist variety as well as solving the challenge of finding items with low or no ratings. Another hybrid recommendation approach (Yu, 2006) uses content-based, Bayesian-classifier, and rule-based methods. They introduce a system that can handle three context categories for mobile usage - user situation context, user media preference, and media terminal’s capability. At first, a content-based approach is used to measure the similarity between a media item and the preference context. Then, Naïve Bayes classifier approach is applied to calculate the probability of the item belonging to the situation context. Finally, a weighted linear combination of these two sub-scores is calculated to get the overall score. Now, all media items are ranked according to the scores achieved through these three steps and they choose the highest score or three highest-scored items for user’s choices. At the end, a rule-based approach determines the appropriate form of the item to be presented in, given the capability context. Overall the system takes 3D input (MediaItem × UserPreference × Terminal-Capability) and recommends 2D output (Modality × Score). Here, Modality represents the final recommended format for a multimedia item—video, image, or text and Score represents the degree of user interest in the recommended item. The authors of (Nakatsuji, 2009) explore the domain of Japanese music blogs to make recommendations based on ontology similarity between a user and other users. Their key goal is to detect so-called “innovative topics”. At the beginning user-interest ontologies are generated to allow the construction of UPs as a hierarchy of classes. Details about that process can be read in (Nakatsuji, 2009, pp.109-111). Next, a user group GU is created which has a high similarity to

81

Portable Personality and its Personalization Algorithms

Figure 6. Bayesian three-way model

a particular user u. GU is obtained by measuring the similarity between user interests. The “innovative topics” for user u are then detected by determining a suitable size of GU and analyzing the ontologies within GU. This suitable size is obtained by using a heuristic threshold to derive X users who have a high similarity with user u. The ontologies of user u and X are compared where also a parameter of innovations is defined indicated by the number of hops needed to get from different instances of an ontology of X to a class of user u. At the end, recommendations are based on ontology instances unknown to user u but well-known to the X users determined earlier. Note that determining the most suitable size of GU is very important for detecting attractive and innovative instances. Too small the innovations might be too close to the user own interest. Too large, on the other hand, the innovations might

82

be too far off as a good recommendation related to the user interests. Novel recommendations are not easy to predicate. Therefore, (Zhang, 2009) partitions the user profile into clusters of similar items and the recommendations are in a list of items matching well with each cluster rather than fitting the entire user profile. In order to achieve this, the user profile is first partitioned into subgroups. The strategy is only applied to user profiles sufficiently large enough. Possible partitioning strategies aka clustering are Extreme clustering (one item per cluster, all items in a single cluster), Graph partitioning, K-Means and Modularity maximization. By applying a dimension reduction strategy such as Singular Value Decomposition before clustering the items the contrast in their similarity values can be enhanced, and thereby improving the clustering results. After partitioning, the recommendations

Portable Personality and its Personalization Algorithms

are then made by matching items to those pertained subgroups. Now the recommendations obtained for each subgroup are aggregated to form the final retrieval set. A current large-scale collaborative approach (Das, 2007), however, uses a linear model to combine different algorithms to generate personalized recommendations for users of Google News. They apply a mix of memory based and model based algorithms. As part of model-based approaches, two probabilistic clustering techniques namely MinHashing (MH) and Probabilistic Latent Semantic Indexing (PSLI) are used to model the relationship among users and news items. MH is a probabilistic clustering method that assigns a pair of users to the same cluster with probability proportional to the overlap between the set of items that these users have voted for. PLSI is employed to perform collaborative filtering. It models and learns the relationship between users and items by modeling the joint distribution of users and items as a mixture distribution. A memory based method for recommending items makes use of covisitation instances, where covisitation is defined as an event in which two stories are clicked by the same user within a certain time span. Thus, recommendations for a particular user can be generated by considering the union of all stories that have been clicked by the members of the clusters that this user belongs to and the set of stories that have been covisited with the set of stories in the user’s click history. For this reason, only stories from this union will get nonzero score, and therefore are sufficient candidate stories for recommendation. Another large scale recommendation system is introduced by (Chu, 2009) based on a featurebased machine learning approach. Data are recorded in multidimensional format with at least three kinds of objects: user · content · temporal context (timestamp). Considering all features in the user and content profiles, a family of predictive bilinear models is employed to discover pattern affinities between heterogeneous features. A set of weight coefficients is introduced to capture the

pairwise associations between user and content features. The parametric model is optimized by fitting observed interactive feedback which reveals the correlations between user patterns and content features. In general, the proposed framework is generic and flexible for other personalized tasks. The bilinear model provides a linear projection from the users’ preferences onto the item characteristics. This will provide a user score indication composed of three parts: 1. long-term personal preferences on content features learnt from historical activities; 2. dynamic characteristics, such as temporal popularity over the whole user population, i.e. item quality; 3. the tradeoff between static personal preferences and article item. Now this user score indicator needs to be related to different types of interactions. This is done by employing likelihood functions over Gaussian distribution. Thus, the posterior distribution of the weight coefficients is determined by a maximum-a-posteriori (MAP) estimation employing a gradient-descent method. MAP estimate is then applied to new user-item pairs predicting the indicator user score. A concept lattice has been applied in (Kwon, 2009) for the shopping-retail domain using sales data and user-item ratings with weather and location ontology. Their proposed profile lattice is constructed by a data matrix which is first normalized and then digitized to binary format (threshold implies sensitivity of occurrence). A node in the lattice can then be translated into an IF-THEN rule and a choice of recommendation is done by generating strong and weak rules by simply varying the threshold value for matrix digitization. Using the lattice, the recommender system can suggest a specific service according to the user dynamic profiles. From these algorithms and approaches it can be seen that hybrid recommendations are most

83

Portable Personality and its Personalization Algorithms

common combining content and user information. Most of them are already employed in a serverside scenario related to the Internet. Therefore, applications on service provider sides should be straightforward with no to minor adaptations. Instead of logging into the service via a PC, accessing would be realized by a mobile device and at the same time user profiles are exchanged (merging and updating done, if necessary). The obviously easiest approach would be to send the MD UP to the SP and the SP sends its recommendations back to the MD where the user can browse them. This has the advantage of utilizing the immense computer-power available on the SP side for their recommendations especially for large-scale system with huge content- and user group information such as in (Dias, 2007; Chu, 2009). Furthermore, changes in the system can now be quickly integrated into the recommendation algorithm / system to update existing structures and / or profiles. The ideas of “innovative topics” by (Nakatsuji, 2009) also falls into these SP side scenarios since it heavily relies on data from other users combined into similar user / interest groups. Due to the large amount of users needed to work reliably, constant changes and updates directly affect the user groups and their combined group profile representation. Generally, simpler and smaller-scale algorithms such as tag clouds (Pessemier, 2009) and concept lattice (Kwon, 2009) could also be employed on the mobile device with a few limitations. For the tag clouds approach an option might be to only send small chunks of information (new or updated information after last visit) matching general interests and detailed recommendations can be made on the mobile using the specific UP. In this case the algorithm would predict ratings for the new information and ranks them accordingly. Watching or purchasing a recommended item would update the weights in the tag cloud accordingly. For the concept lattice, the idea would be similar. Since the lattice is generated from a data matrix due to thresholding, the data matrix could

84

be transferred to the MD where the thresholding is applied by user interaction. Recommendations are achieved by varying the threshold. In both cases, obtaining the necessary data from the SP would be part of accessing the service. One can argue recommendations directly on the mobile device would be feasible as well. This is probably true since MDs become more advanced in their computational power and unlimited data transfers are quite common nowadays. However, for large-scale and / or commercial systems this approach is probably not realizable since we believe the amount of data would be too huge and computational power (not even considering effects on power consumption) not enough to use complex hybrid recommendation algorithms. However, in small-scale scenarios where a mobile application shares content only among (personal) friends (not strangers), the aforementioned approaches by (Yoshii, 2006; Yu, 2006; Pessemier, 2009; Kwon, 2009) may be applicable due to the smaller amount of content as well as the rather limited number of further user input.

Profile Merging An essential part of any mobile and service recommendation system or framework is the ability to efficiently sync, merge and update different existing profile versions on MD and / or SP side. Thus, the growth of UPs for different people and services have encouraged researchers to explore options to merge multiple UPs of a single person or a single SP-specific UP of multiple persons into a so-called common audience profile. A still existing challenge is, however, gathering information from multiple sources and representing the personal content in form of these unknown source structures. An earlier approach related to user profiling and profile management in a smart home is presented in (Salem, 2004). Their issue of merging multiple profiles is related to the processing of profiles when multiple users are using or be-

Portable Personality and its Personalization Algorithms

ing present in an aware environment. Therefore merging profiles of multiple users is needed to relate each user to the environment and ensure a cohesive response. Profile merging is used by the environment to either 1. modify and influence the environment response to the users, 2. to concurrently respond to the users, 3. or to direct an environment request to the users. The first case happens when there is neither a conflict of resources nor a conflict of interest. The second case occurs when there is a divergence of resources such as sharing a facility or service. Best example is probably the TV at home. Finally the third case is for situations when there is a conflict of interest when some user(s) want to have influence over other user(s). The merging technique is based on the statistical analysis of vector distribution in the meta-data space. It is a combination of Boolean Logic, a Vector-Space model and a probabilistic model. In (Yu, 2006), a TV program recommendation scheme is implemented where the user profile merging algorithm combines individual profiles to form a common user profile that reflects most and consistent preferences of the group. Therefore, the merging strategy is based on total distance minimization and is as follows. The user profiles are formed into vectors of features and weights. Based on the features, a lexicon is constructed. Then the universal vector for each user profile is generated by thresholding the feature weights. From these universal vectors features are selected using total distance minimization. After normalizing the weights, the target weights can be calculated to generate the common user profile. They results showed that the approach works best for homogeneous groups whereas heterogeneous group results were not satisfactory, which, of course, would be the more interesting scenario. A disadvantage of this method is that the profiles

are stored on the TV set and people have to log on via an interface to activate and use their profile. A portable solution is presented in (Reymann, 2008) as part of their P2 framework (Reymann, 2007) where UPs on mobile devices can be exchanged with TV-sets or set-top boxes via Bluetooth and then construct a common audience profile. They require that UPs are represented in XML format extended with specific P2 attributes such as p2:merge, which is used for merging items but does not change the existing structure. It is checked if all values of all defined identifiers and their location within the structure are equal. After that the child elements of the item can be merged by individual merging strategies. After merging, based on the performed decisions of the audience the existing profile can be updated on the P2 provider, in this case the TV equipment. Additionally, individual profile updates are distributed to each mobile phone of the audience. So far they are able to merge content of multiple sources embedded in the same namespace specification into one XML representation whereas merging of personal content originating from two different namespaces is not supported. Those two techniques could also be applied to other multimedia domains. The result is a collection of multiple namespaces holding personal content of multiple profiling sources. More inside into the procedure is provided by (Reymann, 2008). The challenge of merging different UPs of the same person has been tackled by (Morikawa, 2004) and (Yu, 2005). In (Morikawa, 2004), the system uses two major parts Profile Collectors (PCs) and Profile Aggregator (PA). PCs acquire and handle various profiles from different sources such as location information, web behavior, and purchase history. The PA aggregates those various profiles from diverse PCs and manages them as a Personalized Profile. They assume that a user’s home server can provide that functionality and data are synchronized periodically. The aggregation process updates the Personalized Profile based on a Resource Description Framework (RDF) triple

85

Portable Personality and its Personalization Algorithms

model and a constructing template. They also assume XML profile data, which is transformed into an RDF based input profile by style reformatting. Then the Personalized Profile is updated via a personalized profile update module, which provides functions for adding triples, updating literal values, and unifying the same representation of triples. They applied the system to a shopping list and purchase history scenario. The approach by (Yu, 2005) proposes an activity-based profile modeling approach. In this model, the complete user profile is created based on one or many activity profiles. During merging each attribute in the complete profile gets an annotation to indicate what activity it is associated with, when and where it should be used, what other attributes are highly related within the same activity. Now whenever the user sends a query, one activity profile is dynamically created by retrieving the corresponding attributes from the complete profile using the annotations. The GUMF (Abel, 2009) uses a novel approach, which they call User Pipes that allows user profile reasoning by mashing up different user profile data streams in RDF or RSS-format by applying Semantic Web Pipes (http://pipes.deri. org/) or Yahoo Pipes (http://pipes.yahoo.com). Basically, multiple data stream can be combined with other data streams to derive new user profile information. An example may be combining profile information obtained from a search query session with data from profile interests to find out whether the user’s preferences and search activities are thematically similar mashed up information with other RSS feeds from the Web. The benefit of the user pipe approach is that they result in user profile streams that can again be used by other profile reasoners, which allows for flexible and extensible user profile reasoning. The critical point of this approach is the immensely huge amount of RSS data on the Web that could slow down the processing of a pipe. Therefore, future work is to investigate options of caching

86

strategies (e.g. precompute pipes regularly and deliver the cached results). As with the personalization algorithm, due to the use of ontology profiles merging can be approached by ontology merging and adapting techniques including consistency checks among ontologies for applied context and tags (Thiagarajan, 2008). However, since domain ontologies represent concepts in very specific ways, they are not compatible most of the time. Thus, when a system relies on a domain ontology it often needs to merge the domain ontology into a more general representation. This is a challenge especially for the ontology designer since different ontologies in the same domain can also arise as a result of different perceptions of the domain based on cultural background, education, ideology, or because of a different representation language. Right now, merging ontologies that are not developed from a common foundation ontology is mostly a manual process. However, domain ontologies that use the same foundation ontology can be merged automatically. Current studies on generalized ontology merging techniques are still largely theoretical.

Challenges and Recommendations Up to now, most services deal with personalization in their own way by employing own user profile representations and recommendation systems. The introduction of APML to describe user preferences and interests in a common language is one step in the direction of making user profiles available to other services. This leads to what can be seen as the biggest challenge for crossservice personalization: the lack of standardized generic form of UP representation. An approach to overcome issues from a non-profile perspective is the P2 framework providing means to handle information from multiple sources by a common portable metadata repository. Another challenge rising from such generic profile is the tendency to gather more information from different nonrelated sources which are then to be combined into

Portable Personality and its Personalization Algorithms

a single complex representation of a user (digital personality). So far this concept of such a complex UP is just theory where focus has been on attempts to handle and structure the data in a meaningful way. However, no practical implementation has been realized yet. The next stage would be, of course, to find mechanisms to update and merge such profile representations in an efficient manner. Furthermore, since such profile consists of various information, which is interconnected to certain extend, inference methods are applicable to enrich the profile based on existing information. Such a generic UP representation would be the ideal long-term vision for cross-system personalization. Anyway, an intermediate solution could be the P2 approach were service providers have their own form of profile representation but there is a generic UP so different services could exchange UPs. APML might be worth investigating in that sense. However, such generic yet complex UPs are just one step. Considering this, personalization approaches need to be able to sync, merge, and update such UPs and extract relevant information for their service. Here, partitioning the UP might be one solution as proposed by (Zhang, 2009) but instead of doing it based on algorithms, this could be done during the data gathering stage and profile updating and synchronization. This is, again, related to how to handle and represent these UPs properly. Furthermore, most personalization approaches are already employed on service sides today but so far they did not have to consider users providing a detailed profile upon their arrival (first service usage). Hence, service providers and their recommendation systems have to adapt to such scenarios where fast (or instant) personalization and recommendation is required based on detailed user data. Possible options might be to either always use the profile provided by the user since this will be the most current one for recommendations or sync an existing and the user provided profile and perform the recommendation service just on changes between the two profile versions.

Note that when talking about personalization, user profiles, and especially the combination of these two across multiple systems rises many issues and challenges related to security and privacy. We have not discussed anything related to that but want to mention that these are important points to be considered when going down that road. (Cranor, 2003) discusses privacy risks associated with personalization and describes a number of approaches to personalization system design that can reduce these risks. Furthermore, privacy and security concerns are supported by the second part of the 2009 personalization survey from www. choicestream.com where the majority of people are concerned about their data being share to services they do not know about and that their data might not be secure on any service. The security concern would probably grow further considering the portability aspect among different systems. Furthermore, this would also have an influence on the fact that many people actually do not want every service to know everything about them. Yet another important fact that needs to be considered in cross-system personalization.

CONCLUSION With the growth of technology and its related services, personalization becomes more important and anticipated whether it is for web searches, music taste or (online) shopping. User profiles have been used before but nowadays there is more required. They should include semantic content and context as well as being adaptive and evolvable using short- and long-term preferences of any type; in short the long-term vision is a digital representation of a real-world personality. The application of ontologies and their related techniques seem to provide a promising direction towards that vision. To acquire and manage those user profiles, various approaches have been proposed but it is hard to compare their performance due to different data the user knowledge

87

Portable Personality and its Personalization Algorithms

is gathered from and the different domains these techniques are applied to. However, the portability aspect of user profiles has been picked up by the (research) community especially considering the current trend of mobile devices. At the moment, the distributed client-server profile model shows potential to combine user expectations and service needs, and should be further investigated. Therefore, recommendation approaches ought to look also into the area of distributed profiling and simultaneously considering the application of multiple profile domains. First steps are directed towards cross-system personalization utilizing a single user profile representation but it is just the beginning and more efforts and focus needs to be invested in that direction; and not just from the technical point of view but also considering privacy and security issues as well as psychological and ethological aspects. The closely related issue of merging user profiles has not caught much attention either yet. Current techniques are rather simple and more advance methods are required to push the personalized user profile towards a new portable personality experience.

REFERENCES Abel, F., Heckmann, D., Herder, E., Hidders, J., Houben, G., & Krause, D. (2009). A Framework for Flexible User Profile Mashups. In Proceedings of User Modeling. Adaptation and Personalization. Anand, S., Kearney, P., & Shapcott, M. (2007). Generating semantically enriched user profiles for Web personalization. ACM Transaction on Internet Technology, 7(4), 22:1-26. Ankolekar, A., & Vrandecic, D. (2006). Personalizing web surfing with semantically enriched personal profiles. In Proceedings of Semantic Web Personalization Workshop.

88

Bartolomeo, G., Martire, F., Rukzio, E., Salsano, S., Melazzi, N., & Noda, C. (2006). The Simplicity Device: Your Personal Mobile Representative. Technical Report University of Munich. Brugnoli, M., Hamard, J., & Rukzio, E. (2005). User Expectations for Simple Mobile Ubiquitous Computing Environments. International Workshop on Mobile Commerce and Services, 2-10. Bruns, V., Reymann, S., & Lugmayr, A. (2007). Distributed profiling in a partitioned ambient network. In Proceedings of the 6th International Conference on Mobile and Ubiquitous Multimedia, 284, 34-41. Castellano, G., Mesto, F., Minunno, M., & Torsello, M. (2007). Web User Profiling Using Fuzzy Clustering. International Workshop on Fuzzy Logic and Applications, 94-101. Chen, H., Perich, F., Finin, T., & Joshi, A. (2004). SOUPA: standard ontology for ubiquitous and pervasive applications. Proceedings of the 1st International Conference on Mobile and Ubiquitous Systems, 258-267. Chen, T., Han, W., Wang, H., Zhou, Y., Xu, B., & Zang, B. (2007). Content Recommendation System Based on Private Dynamic User Profile. Proceedings of the International Conference on Machine Learning and Cybernetics, 2112 – 2118. Chu, W., & Park, S. (2009). Personalized Recommendation on Dynamic Content Using Predictive Bilinear Models. Proceedings of the 18th international Conference on World Wide Web, 691-700. Cranor, L. F. (2003). ‘I didn’t buy it for myself’ privacy and ecommerce personalization. In Proceedings of the 2003 ACM Workshop on Privacy in the Electronic Society (Washington, DC). ACM, New York, NY, 111-117.

Portable Personality and its Personalization Algorithms

Das, A., Datar, M., Garg, A., & Rajaram, S. (2007). Google News Personalization: Scalable Online Collaborative Filtering, Proceedings of the 13th International Conference on World Wide Web, 271-280.

Kwon, O., & Kim, J. (2009). Concept lattices for visualizing and generating user profiles for context-aware service recommendations. Expert Systems with Applications, 36, 1893–1902. doi:10.1016/j.eswa.2007.12.064

Ferscha, A.,. Hechinger, M, Oberhauser, R., dos Santos Rocha, M., Franz, M., & Mayrhofer, R. (2004). Digital Aura. Advances in Pervasive Computing, 176.

Liffick, B., & Zoppetti, G. (2007). You can take it with you: profile transportability. Proceedings of the 12th Conference on Innovation and Technology in Computer Science Education, 106-110.

Ghosh, R., & Dekhil, M. (2007). I, me and my phone: identity and personalization using mobile devices. HP Labs Technical Report, 184, 1–14.

Lugmayr, A., Reymann, S., Bruns, V., Rachwalski, J., & Kemper, S. (2009). Distributing the personal digital environment throughout your entertainment environment: handling personal metadata across domains. Multimedia Systems, 15(3), 187–199. doi:10.1007/s00530-009-0162-4

Godoy, D., & Amandi, A. (2006). Modeling user interests by conceptual clustering. Information Systems, 31(4), 247–265. doi:10.1016/j. is.2005.02.008 Golemati, M., Katifori, A., Vassilakis, C., Lepouras, G., & Halatsis, C. (2007). Creating an Ontology for the User Profile: Method and Applications, Proceedings of the 1st IEEE International Conference on Research Challenges in Information Science, 407-412. Han, L., & Chen, G. (2009). A fuzzy clustering method of construction of ontology-based user profiles. Advances in Engineering Software, 40, 535–540. doi:10.1016/j.advengsoft.2008.10.006 He, H., Hai, H., & Rujing, W. (2007). FCA - Based Web User Profile Mining for Topics of Interest. Proceedings of the IEEE Internatioanl Conference on Integration Technology, 778-782. Heckmann, D., Schwartz, T., Brandherm, B., & Schmitz, M., (2005). Gumo – The General User Model Ontology. User Modeling, 428-432. Kim, H., & Chan, P. (2003). Learning Implicit User Interest Hierarchy for Context in Personalization. Proceedings of the Internatioanl Conference on Intelligent User Interfaces, 101-108. Kobsa, A. (2007). Privacy-Enhanced Personalization. Communications of the ACM, 50(8), 24–33. doi:10.1145/1278201.1278202

Magoulas, G., & Dimakopoulos, D. (2006). An Adaptive Fuzzy Model for Personalization with Evolvable User Profiles. Proceedings of the International Symposium on Evolving Fuzzy Systems, 336-341. Marghny, M. (2006). Evolutionary Algorithm For Learning The Dynamics Of The User Profile. Journal of Artificial Intelligence and Machine Learning, 6(3), 49–54. Mehta, B. (2007). Learning from What Others Know: Privacy Preserving Cross System Personalization. User Modeling. 57-66. Mehta, B., Hofmann, T., & Fankhauser, P. (2006). Cross System Personalization by Factor Analysis. AAAI. Mehta, B., Niederee, C., Stewart, A., Degemmis, M., Lops, P., & Semeraro, G., (2005). Ontologically-Enriched Unified User Modeling for CrossSystem Personalization. User Modeling, 119-123. Michlmayr, E., & Cayzer, S. (2007). Learning User Profiles from Tagging Data and Leveraging them for Personal(ized) Information Access. Proceedings of the 16th International Conference on World Wide Web.

89

Portable Personality and its Personalization Algorithms

Morikawa, D., Honjo, M., Yamaguchi, A., & Ohashi, M. (2004). Profile Aggregation and Dissemination: A Framework for Personalized Service Provisioning. Provisioning. Tokyo, Japan: KDDI Corporation. Nakatsuji, M., Yoshida, M., & Ishida, T. (2009). Detecting innovative topics based on user-interest ontology. Web Semantics: Science, Services and Agents on the World Wide Web, 7(2), 107–120. doi:10.1016/j.websem.2009.01.001 Nasraoui, O., Soliman, M., Saka, E., Badia, A., & Germain, R. (2008). A Web Usage Mining Framework for Mining Evolving User Profiles in Dynamic Web Sites. IEEE Transactions on Knowledge and Data Engineering, 20(2), 202–215. doi:10.1109/TKDE.2007.190667 Niederée, C., Stewart, A., Mehta, B., & Hemmje, M. (2004). A Multi-Dimensional, Unified User Model for Cross-System Personalization. Proceedings of the Workshop on Environments for Personalized Information Access. Papadogiorgaki, M., Papastathis, V., Nidelkou, E., Waddington, S., Bratu, B., Ribiere, M., & Kompatsiaris, I. (2008). Two-level Automatic Adaptation of a Distributed User Profile for Personalized News Content Delivery. International Journal of Digital Multimedia Broadcasting, Article ID 863613, 21 pages. Park, Y., & Chang, K. (2009). Individual and group behavior-based customer profile model for personalized product recommendation. Expert Systems with Applications, 36, 1932–1939. doi:10.1016/j.eswa.2007.12.034 Pessemier, T., Deryckere, T., & Martens, L. (2009). “Context Aware Recommendations for User-generated Content on a Social Network Site. In Proceedings of the 7th European Conference on European interactive Television Conference, 133-136.

90

Potonniee, O. (2002). Ubiquitous Personalization: a Smart Card Based Approach. In Proceedings of the 4th Gemplus Developer Conference. Rafter, R., O’Mahony, M., Hurley, N., & Smyth, B. (2009), What Have the Neighbours Ever Done for Us? A Collaborative Filtering Perspective. In Proceedings of User Modeling, Adaptation and Personalization, 355-360. Reymann, S., Alves, D., & Lugmayr, A. (2008). Personalized Social Networking: an Applied Scenario in a Portable Personality Environment. Proceedings of Mindtrek, 2008, 172–176. doi:10.1145/1457199.1457237 Reymann, S., Bruns, V., & Lugmayr, A. (2007). P2 - Portable Personality, a Middleware Solution for Smart User Profile Management and Distribution. Proceedings of the 5th European Conference on European interactive Television Conference, 78-83. Reymann, S., Rachwalski, J., Kemper, S., & Lugmayr, A. (2008), Development of a Generic XML Personality Metadata Handler for Distributed Entertainment Services. Proceedings of the 6th European Conference on European interactive Television Conference, 214-218. Salem, B., & Rauterberg, M. (2004). Multiple User Profile Merging (MUPE): Key Challenges for Environment Awareness, Proceedings of the 2nd European Symposium of Ambient Intelligence, 196–206. Schuurmans, J., & Zijlstra, E. (2004). Towards a continuous personalization experience, Conference on Dutch Directions in Human-Computer Interaction, 19. Sendhilkumar, S., & Geetha, T. (2008). Personalized ontology for web search personalization. In Proceedings of the 1st Bangalore Computer Conference, 1-7.

Portable Personality and its Personalization Algorithms

Shtykh, R., & Jin, Q. (2009). Integrating Search and Sharing: User-Centric Collaborative Information Seeking. Proceedings of the 8th IEEE/ ACIS International Conference on Computer and Information Science, 388-393. Sieg, A., Mobasher, B., & Burke, R. (2007). Web search personalization with ontological user profiles. In Proceedings of the 16th ACM Conference on Information and Knowledge Management, 525-534. Sinner, A., Kleemann, T., & von Hessling, A. (2004). Semantic User Profiles and their Applications in a Mobile Environment. Proceedings of the Artificial Intelligence in Mobile Systems. Sugiyama, K., Hatano, K., & Yoshikawa, M. (2004). Adaptive web search based on user profile constructed without any effort from users. Proceedings of the 13th International Conference on World Wide Web, 675-684. Sutterer, M., Droegehorn, O., & David, K. (2007). Managing and Delivering Context-Dependent User Preferences in Ubiquitous Computing Environments. Proceedings of the 2007 International Symposium on Applications and the Internet Workshops, 1-4. Sutterer, M., Droegehorn, O., & David, K. (2008). UPOS: User Profile Ontology with SituationDependent Preferences Support. Proceedings of the 1st International Conference on Advances in Computer-Human Interaction, 230-235. Thiagarajan, R., Manjunath, G., & Stumptner, M. (2008). Computing Semantic Similarity Using Ontologies. HP Labs Technical Report, 87, 1–17. Uhlmann, S., & Lugmayr, A. (2008). Personalization algorithms for portable personality. In Proceedings of the 12th International Conference on Entertainment and Media in the Ubiquitous Era, ACM, Tampere, Finland.

Villalonga, C., Strohbach, M., Snoeck, N., Sutterer, N., Belaunde, M., Kovacs, E., Zhdanova, A., Goix, L., & Droegehorn, O. (2009). Mobile Ontology: Towards a Standardized Semantic Model for the Mobile Domain. Workshops on Service-Oriented Computing, 248–257. Yingchen, X., Junzhong, G., Jing, Y., & Zhengyong, Z. (2009), An Ontology-based Approach for Mobile Personalized Recommendation. IITA International Conference on Services Science, Management and Engineering, 336-339. Yoshii, K., Goto, M., Komatani, K., Ogata, T., & Okuno, H. (2006). Hybrid collaborative and content-based music recommenddation using probabilistic model with latent user preference. In Proceedings of the International Conference on Music Information Retrieval, 296-301. Yu, S., Al-Jadir, L., & Spaccapietra, S. (2005). Matching User’s Semantics with Data Semantics in Location-Based Services. Proceedings of the 1st Workshop on Semantics in mobile Environments. Yu, Z., Zhou, X., Hao, Y., & Gu, J. (2006). TV Program Recommendation for Multiple Viewers Based on user Profile Merging. User Modeling and User-Adapted Interaction, 16(1), 63–82. doi:10.1007/s11257-006-9005-6 Yu, Z., Zhou, X., Zhang, D., Chin, C., & Wang, X. (2006). Supporting Context-Aware Media Recommendations for Smart Phones. IEEE Pervasive Computing / IEEE Computer Society [and] IEEE Communications Society, 5(3), 68–75. doi:10.1109/MPRV.2006.61 Zhang, M., & Hurley, N. (2009). Novel Item Recommendation by User Profile Partitioning. In Proceedings of the International Conference on Web Intelligence and Intelligent Agent Technology, 508-515.

91

Portable Personality and its Personalization Algorithms

Zhou, X., Wu, S., Li, Y., Xu, Y., Lau, R., & Bruza, P. (2006). Utilizing Search Intent in Topic Ontology-Based User Profile for Web Mining. Proceedings of the ACM International Conference on Web Intelligence, p. 558 – 564. Zhou, Y., Hu, Y., & He, H. (2007). Learning User Profile in the Personalization News Service. Proceedings of the International Conference on Natural Language Processing and Knowledge Engineering, 485-490. Zhuhadar. L., Nasraoui, O., & Wyatt, R. (2009). Dual Representation of the Semantic User Profile for Personalized Web Search in an Evolving Domain. AAAI.

ADDITIONAL READING Adomavicius, G., & Kwon, Y. (2007). New Recommendation Techniques for Multicriteria Rating Systems. IEEE Intelligent Systems, 22(3), 48–55. doi:10.1109/MIS.2007.58 Adomavicius, G., & Tuzhilin, A. (2005). Toward the Next Generation of Recommender Systems: A Survey of the State-of-the-Art and Possible Extensions. IEEE Transactions on Knowledge and Data Engineering, 17(6), 734–749. doi:10.1109/ TKDE.2005.99 Adomavicius, G., & Tuzhilin, A. (2005). Personalization technologies: a process-oriented perspective. Communications of the ACM, 48(10), 83–90. doi:10.1145/1089107.1089109 Bell, R., Koren, Y., & Volinsky, C. (2007). Chasing $1,000,000: How We Won The Netflix Progress Prize, In ASA Statistical and Computing Graphics Newsletter, 2. Bonhard, P., Harries, C., McCarthy, J., & Sasse, M. (2006). Accounting for taste: using profile similarity to improve recommender systems. In Proceedings of the SIGCHI conference on Human Factors in computing systems, 1057-1066. 92

Bruns, V., & Reymann, S. (2007). Portable Personality (P²) – Development of a Middleware Solution for Consumer Profiling and Advanced Profile Distribution, Master of Science Thesis, Tampere University of Technology, Tampere, Finland. Buriano, l., Marchetti, M., Carmagnola, F., Cena, F., Gena, C., & Torre, I. (2006). The role of ontologies in context-aware recommender systems. In Proceedings of the 7th International Conference on Mobile Data Management, 80. Cantador, I., Bellogín, A., & Castells, P. (2008). A multilayer ontology-based hybrid recommendation model. AI Communications, 21(2-3), 203–210. Carmagnola, F., & Cena, F. (2009). User identification for cross-system personalization. Information Sciences, Volume 179, Issues (National Council of State Boards of Nursing (U.S.)), 1-2, 16–32. Choeh, J., & Lee, H. (2008). Mobile push personalization and user experience. AI Communications, 21(2-3), 185–193. Crossley, M., Kings, N. J., & Scott, J. R. (2003). Profiles — Analysis and Behaviour. BT Technology Journal, 21, 56–66. doi:10.1023/A:1022404310934 Frasconi, P., & Smyth, P. (2003). Modeling the Internet and the Web: Probabilistic Methods and Algorithms (1st ed.). John Wiley & Sons. Godoy, D., & Amandi, A. (2005). User profiling in personal information agents: a survey. The Knowledge Engineering Review, 20(4), 329–361. doi:10.1017/S0269888906000397 Gretzel, U., & Fesenmaier, D. (2006). Persuasion in Recommender Systems. International Journal of Electronic Commerce, 11(2), 81–100. doi:10.2753/JEC1086-4415110204

Portable Personality and its Personalization Algorithms

Hinze, A., & Buchanan, G. (2006). The challenge of creating cooperating mobile services: experiences and lessons learned. In Proceedings of the 29th Australasian Computer Science Conference, 207-215. Jacobsson, M., Rost, M., & Holmquist, L. (2006). When media gets wise: collaborative filtering with mobile media agents. In Proceedings of the 11th international conference on Intelligent user interfaces, 291-293. Lehikoinen, J., Aaltonen, A., Huuskonen, P., & Salminen, I. (2007). Personal content experience: managing digital life in the mobile age. New York: Wiley-Interscience. doi:10.1002/9780470511022 Mehta, B. (2009). Cross System Personalization: Enabling Personalization Across Multiple Systems. VDM Verlag. Melville, P., Mooney, R., & Nagarajan, R. (2002). Content-Boosted Collaborative Filtering for Improved Recommendations. Proceedings of the 18th National Conference on Artificial Intelligence, 187-192. MIT Project, —Oxygen‖. Computer Science and Artificial Intelligence Laboratory [Online] Available: http://www.oxygen.lcs.mit. edu /Overview.html Newbould, R., & Collingridge, R. (2003). Profiling Technology. BT Technology Journal, 21, 44–55. doi:10.1023/A:1022400226864 Segaran, T. (2007). Programming Collective Intelligence: Building Smart Web 2.0 Applications (1st ed.). O’Reilly Media. Treiblmaier, H., Madlberger, M., Knotzer, N., & Pollach, I. (2004). Evaluating Personalization and Customization from an Ethical Point of View: An Empirical Study. In Proceedings of the 37th Annual Hawaii international Conference on System Sciences Volume 7 (January 05 - 08, 2004). IEEE Computer Society, Washington, DC, 70181.2.

Wang, Y., & Kobsa, A. (2010). Privacy in CrossSystem Personalization. Intelligent Information Privacy Management Symposium, Stanford, CA Yap, G., Tan, A., & Pang, H. (2006). Discovering causal dependencies in mobile context-aware recommenders. In Proceedings of the 7th International Conference on Mobile Data Management, 4. Zanker, M., Jessenitschnig, M., Jannach, D., & Gordea, S. (2007). Comparing Recommendation Strategies in a Commercial Context. IEEE Intelligent Systems, 22(3), 69–73. doi:10.1109/ MIS.2007.49 Ziegler, C.-N. (2005). Towards Decentralized Recommender Systems. PhD Thesis, June 2005, Albert-Ludwigs-Universität Freiburg, Freiburg i.Br., Germany.

KEY TERMS AND DEFINITIONS Cross-System Personalization: Combining portable personality and personalization by using user information gathered by a system A on another system B to obtain personalization without going through the information accumulation process of system B again. Digital Personality: Digital representation of a real-world user by a complex user profile which integrates every information of the user. Personalization: An action by utilizing an user’s profile to adapt a system or service to the user’s preferences. Portable Personality: Portable form of the digital personality which can be carried around (e.g. mobile device) to personalize each and every used service. Recommendation System: A system mainly consisting of algorithms and techniques to evaluate user profiles for personalization. User Profile: Electronic representation within in a system of an user’s preferences, interests and behavior accumulated by system interaction; user for personalization and recommendations. 93

Section 2

Learning, Training, and Knowledge Sharing

95

Chapter 5

The Integration of Aspects of Geo-Tagging and Microblogging in m-Learning Christian Safran Graz University of Technology, Austria Victor Manuel Garcia-Barrios Carinthia University of Applied Sciences (CUAS), Austria Martin Ebner Graz University of Technology, Austria

ABSTRACT The recent years have shown the remarkable potential use of Web 2.0 technologies in education, especially within the context of informal learning. The use of Wikis for collaborative work is one example for the application of this theory. Further, the support of learning in fields of education, which are strongly based on location-dependent information, may also benefit from Web 2.0 techniques, such as Geo-Tagging and m-Learning, allowing in turn learning in-the-field. This chapter presents first developments on the combination of these three concepts into a geospatial Wiki for higher education, TUGeoWiki. Our solution proposal supports mobile scenarios where textual data and images are managed and retrieved in-the-field as well as some desktop scenarios in the context of collaborative e-Learning. Within this scope, technical restrictions might arise while adding and updating textual data via the collaborative interface, and this can be cumbersome in mobile scenarios. To solve this bottleneck, we integrated another popular Web 2.0 technique into our solution approach, Microblogging. Thus, the information pushed via short messages from mobile clients or microblogging tools to our m-Learning environment enables the creation of Wiki-Micropages as basis for subsequent collaborative learning scenarios. DOI: 10.4018/978-1-60960-774-6.ch005

Copyright © 2012, IGI Global. Copying or distributing in print or electronic forms without written permission of IGI Global is prohibited.

The Integration of Aspects of Geo-Tagging and Microblogging in m-Learning

1. INTRODUCTION A remarkable movement towards geo-locating software has occurred in the last months, marking a renaissance of location-based mobile applications. One of the reasons is the availability of a variety of mobile devices providing integrated GPS1 receivers. Another reason is the rising number of mashup applications accessing freely available cartographic material via Web services, and thus providing added value for geospatial information. Almost in parallel, geo-tagging appeared. This technique denotes the marking of a digital resource with geographical coordinates and is mostly used for images. In the case of images these coordinates can be integrated into the image by using a set of Exif2 headers, which can be included in JPEG files. Leaving aside valuable discussion and concerns about privacy issues, this additionally tagged information offers new possibilities for teaching and learning, especially in fields which strongly depend on geo-located data, such as civil engineering, geosciences or archaeology. The combination of geo-tagging with other technologies connected to Web 2.0 provides a further contribution to e-Learning 2.0, as defined by Stephen Downes (Downes, 2005). This chapter presents our research on the development of an application, which is integrating (mobile) geo-tagging of images with collaborative authoring in order to enhance the learning experience in the aforementioned fields of education, as well as its extension by the integration of the Microblogging paradigm. The chapter is based on a conference contribution at the ACM MindTrek Conference 2009 (Safran, Garcia-Barrios, Ebner, 2009). The implementation of our solution proposal, called TUGeoWiki3, supports mobile learning (m-Learning) in reference to two scenarios: (i) a mobile application scenario, which focuses on information retrieval and real-time sharing of resources, and (ii) a desktop application scenario, which supports informal e-Learning by providing

96

a collaborative authoring tool. For the concrete fields of education mentioned above, TUGeoWiki represents a tool that supports field trips and excursions from the preparation phase, throughout the field trip itself and until collaboration-based review processes. As such, this combination of geo-tagging mobile applications and a Wiki as a collaborative learning tool provides a unique approach to enhance learning in-the-field. First evaluations in three excursions of civil engineering and geology lectures have revealed as a only major criticism that using the Web-based collaborative interface for the provision of textual information in mobile scenarios has shown to be cumbersome. Thus, to solve these user interaction problems within the mobile scenario, we propose to incorporate the novel Web 2.0 concept of Microblogging into the work with our TUGeoWiki. The remainder of this chapter is structured as follows. First, we give an overview over some topics of interest in the context of this research and discuss them in relation with our application. Subsequently, the development and functionality of the TUGeoWiki application are presented and design decisions explained. Next, the focus is set on the expansion of the existing solution with the integration of microblogging. And finally, a summary and some outlook on future work are given.

2. TOPICS IN CONTEXT The development of TUGeoWiki was based on related work from three areas: geo-tagged images, Wikis for collaborative learning, and mobile learning (m-Learning). TUGeoWiki represents a novel approach to the combination of these concepts for pedagogical aims while learning in-the-field. In subjects like civil engineering, geosciences, or architecture, higher education is strongly based on visual information of real-world objects. As pointed out by Brohn, the “language of intuition is visual, just as the language of analysis is abstract and symbolic” (Brohn, 1983). Taking civil engi-

The Integration of Aspects of Geo-Tagging and Microblogging in m-Learning

neering as an example for such a subject, several research activities have been able to point out the importance of the utilisation of visualisations, animations and interactive tools for educative purposes (Ebner, & Holzinger, 2003); (Holzinger, & Ebner, 2005); (Ebner, Scerbakov, Maurer, 2006). Especially for explanations of highly complex engineering models, new technologies offered a completely different way of teaching and learning. Still, visualisations lacked at one particular point: the connection of the abstract engineering model and the real world landmarks. The major competence of any practical engineer is assumed to be the capability of abstracting an appropriate model from nature in order to develop a quantifiable mathematical model. In this context the knowledge about the particular environment where a building will be placed is highly important. Such a connection of visualisations and real-world locations can be achieved by the usage of geo-tagged images. Considering another relevant point of view, Wikis, as online collaboration tools, were introduced by Leuf and Cunningham in 1995 (Leuf & Cunningham, 2001). The term itself is derived from the Hawaiian word wikiwiki, which means quick. The technology has been designed to provide a simple tool for knowledge management, which places at the disposal of all users a smart possibility to mutually create and edit content online. In addition, individual users may use the functionality of version history to retrace all content modifications and, if desired, revert to earlier content versions. As such, a Wiki is an easy-to-use application for the collaborative management of online contents. These characteristics, in particular, have made Wikis a tool of choice in informal learning (Fucks-Kittowski, Köhler, Fuhr, 2004). The didactical relevance of Wikis in e-Learning has lead Stephen Downes to list them as one of the basic technologies of e-Learning 2.0 (Downes, 2005). Mobile learning (m-Learning) can be seen as the combination of e-Learning and mobile computing, and promises the access to applications that

support learning anywhere and anytime (Tatar, Roschelle, Vahey, Penuel, 2004). Meanwhile, due to technological progress, hardware is considered a solved problem. However, innovative, affordable and usable software remains the greatest challenge. Handhelds, for example, should support project-based learning in context, that is, using the mobile device as an integral part of a learning activity (Norris, Soloway, 2004). One of the central advantages of mobile learning is ongoing assessment and possible feedback, as demonstrated in (Klamma, Chatti, Duval, Hummel, Hvannberg, Kravcik, Law, Naeve, Scott, 2007). In higher education, m-Learning is especially interesting for fields of study which rely on education on-site, i.e. in-the-field. One example for the use of mobile technologies for teaching purposes is addressed within the EU research project RAFT (Remote Accessible Field Trips), which was conducted from 2002 to 2005. The target of this project was the support of school classes with virtual excursions using portable Internet-conferencing tools (Kravcik, Specht, Kaibel, Terrenghi, 2008).

2.1 Alternative Collaborative Tools Wikis are only one example for collaborative tools which can be applied in learning and teaching. Alternatives include the application Google Docs and Google Wave. One of the main disadvantages of these two approaches is the fact that they are relying on external services hosted by a single company and the fact that therefore the control over the server-side application and date is limited.

2.1.1 Google Docs Google Docs is an online version of a word-processing and spreadsheet application. It is provided as a service heavily based on the collaboration of users in the creation of documents. As such, features like a version history and detailed information on the individual inputs of the users are available, which make the product well suitable

97

The Integration of Aspects of Geo-Tagging and Microblogging in m-Learning

for collaborative learning. Two main drawbacks make Google Docs unsuitable for the task to support the application scenarios of TUGeoWiki. First of all the documents are stored on external servers, with the limited control described above. Secondly the possibility to develop additional tools to support learning in-the-field is likewise limited, and the underlying paradigm of Google Docs is focused on individual documents. In contrast this paradigm can be enhanced in the context of Wikis to include additional information on individual locations. A more suitable and open approach for online collaboration in-the-field is Google Wave, described in detail below.

2.1.2 Google Wave In 2009, Google announced a tool to bridge the gap between social networks and collaborative tools. Google Wave4 is described as “an online tool for real-time communication and collaboration”. It is aimed to provide a unified solution to manage the own social networks, communicate with the own contacts and collaborate on documents. The announcement of Google Wave created a buzz in the media and the blogosphere. First videos indicated interesting technology and new paradigms of communications. This hype was increased due to the fact of a closed beta, with only some 100.000 invitations available. In the meantime, though, a lot of the early participants stopped using the Google Wave beta [Douglis, 2010]. One of the problems in this context could be the limited size of the community, which, according to Metcalfe’s Law also limits the usefulness of the network [Shapiro and Varian, 1999]. Moreover, the beta still has some drawbacks, which limit the usefulness. The communication in Google Wave is based on individual waves, which are the summary of all input of the participants. One of the major is-

98

sues is based on the fact of modifiability of these waves. Any user may edit any part of a wave, even the input of another user. These changes are not obviously labeled. It is possible to retrace all modifications by a replay feature, which presents the history of all actions within a wave, but in long waves, finding the exact modification searched can be time consuming. Another problem with the user interface is the fact that by default the other participants of a wave see user’s input in real-time. This means that each letter typed is transmitted and displayed immediately, which can be quite distracting, or, in some situations, even embarrassing. Another major issue is the fact, that a wave looks identical for all participants. This means i.e. that the automatic translation, which can be applied, translates the content for all participants, an approach with a doubtful usefulness in a multilingual environment. Aside from all previously mentioned issues of the early versions, Google Wave still provides a novel approach to the communication with communities of interest or practice. One of the main reasons for its presumable success is the fact that the Wave server and Wave client are independent developments. Most of the drawbacks stated before only refer to the Wave client, which could always be replaced by alternative clients. The underlying server paradigm, in contrast, allows radically innovative classes of applications. The important factor for Google Wave in this context will be to keep hold of early adopters. In a stable state and with alternative clients the underlying paradigm could be a possible replacement for the Wiki technology applied in TUGeoWiki, but in the current state this approach is unsuitable due to the facts that it is (a) reliable on remote servers hosted by a single company and (b) is still in development and in an unstable state as far as protocols and features are concerned.

The Integration of Aspects of Geo-Tagging and Microblogging in m-Learning

3. A GEOSPATIAL WIKI FOR M-LEARNING The previous sections introduced two relevant aspects: (i) the importance of visual information, particularly location-related visual information for several fields of education, and (ii) the advantages of collaborative learning with the usage of Wikis. Along these lines, this section introduces firstly the most relevant traits of the proposed solution, and gives then an overview over the solution itself, TUGeoWiki.

3.1 Why Lightweight, GeoTagged and Mobile? In order to combine the two aforementioned aspects, we designed and developed a solution approach for a lightweight, geotagging-based and mobile learning environment applying a geospatial Wiki. The term lightweight expresses our efforts to implement only the basic features of a geographical information system (GIS) for learning, namely (a) collecting and (b) displaying geo-tagged data (e.g. as map overlays). We consider further features of GIS, such as data analysis and modelling, to be out-of-scope here, as they are only necessary for geosciences professionals. Moreover, our notion of lightweight embraces our intent towards unobtrusive user interaction features based on well-known software practices. Especially as far as the acceptance of mobile technologies is concerned, lightweight also refers to the overall costs, as low-cost applications with low maintenance efforts have turned out to be best accepted (Tretiakov, 2008). Furthermore, our solution proposal concentrates on location-based information, and thus, on learning scenarios where such information is an essential part of the curriculum. In those cases, students can benefit from a clearly defined relation of learning material to a geographic location (i.e. geo-tagged materials).

Finally, we coin the term mobile onto our solution application in order to put emphasis on our intention to offer access to geo-tagged information and learning materials in-the-field, thus aiming at the enhancement of on-site learning whenever applicable. It is worth stating at this point that within the context of our solution approach, we focus on mobile phones and PDAs5 instead of other mobile technologies for the purpose of following our primary goal of a lightweight system, as such devices are widespread and handy to carry in the field. Moreover, utilising mobile technology should enable us to foster collaborative activities of learners wherever possible, whenever possible.

3.2 TUGeoWiki Our solution approach, the TUGeoWiki system, is a geospatial Web-based mobile application that aims at supporting the learning scenarios given so far. This section gives an overview on the main features of the system, for more details please refer to (Safran & Zaka, 2008). The TUGeoWiki system is based on the wellknown open source MediaWiki implementation. We have chosen MediaWiki for two reasons. First, it provides two well-defined mechanisms for the extension of functionalities: special pages and templates. Special pages are pages without informative content, they are generated on demand and are used to provide additional features to users, e.g., file upload (Mediawiki, n.d.). Templates are pages created for transclusion purposes, and usually contain repetitive materials or blocks of information (e.g., infoboxes) (Mediawiki, n.d.). Secondly, the user interface of MediaWiki is probably the best-known Wiki user interface, among others, due to the immensely broad use and high popularity of Wikipedia (Voelkel & Oren, 2006). For our TUGeoWiki, we have adapted the MediaWiki paradigm of pages for individual entries in order to define places, which are relayed to geographical coordinates, and thus represent real-world locations. Hence, in our terminology

99

The Integration of Aspects of Geo-Tagging and Microblogging in m-Learning

Figure 1. General notion for creating places in TUGeoWiki

the term place represents the entity in the system, while the term location denotes the actual geographical entity. This adaptation was achieved by using MediaWiki’s special pages to create location-based entries as well as templates to display them. Figure 1 depicts the concept of creating a place. These templates are designed as mashups, thus extending the Wiki entries with mapping material from Google Maps or Microsoft Live Search Maps. Additionally, a hyperlink to the MediaWiki extension Geohack provides access to numerous other map sources (Wikipedia, n.d.). This Wiki application can be used in classroom or remote learning scenarios to provide a tool for collaborative activities on geospatial information, resulting in two application scenarios: a desktop application scenario and a mobile application scenario. The desktop application scenario is based on collaborative authoring with the Wiki and fosters process-oriented learning and task-based learning. Possible use cases in this context are the preparation for field trips as well as the postprocessing and review of the information gained in such an excursion. The focus of this scenario is set on collaborative authoring in order to support informal learning on the topics of such an excursion. The mobile application scenario provides access to the learner’s current location by querying internal or external GPS sensors. The coordinates

100

retrieved from the GPS sensors are used in TUGeoWiki to search for places in the vicinity of the current location or to create a new place in the Wiki and start collaborative learning about the topics of the current location. The main goal behind this scenario is to satisfy an information need just-in-time concerning the current location as well as enabling real-time sharing of resources (mainly images) concerning the location. Due to the restrictions of the user interface (cf. i.e. (Parsons, Ryu, Cranshaw, 2006)), collaborative authoring in this mobile scenario is a non-trivial task, and thus the editorial work on places has been restricted to the creation and annotation of socalled place stubs. Place stubs (also called article stubs) can be seen as temporary mini-place objects that learners use at their mobile devices, and after submitting them to the Wiki server, they can be described in more detail through a desktop Web browser. Additionally for the mobile application scenario, TUGeoWiki provides a feature to create geo-tagged images with the mobile phone’s camera embedding the GPS coordinates in the Exif headers of the image files. In a separate step, these images (or images created with any other application for geo-tagging images) can be uploaded and relayed to existing places or used to provide an article stub for a new place in an arbitrary location around the corresponding coordinates. We have stipulated these two aforementioned scenarios in order to investigate an improvement

The Integration of Aspects of Geo-Tagging and Microblogging in m-Learning

Figure 2. TUGeoWiki component architecture – mobile application scenario

of learning activities in-the-field and on-site by supporting several steps in such learning journeys, i.e., activities before and after the journey with the desktop scenario and activities during the journey with the mobile scenario. The component architecture of the TUGeoWiki system as well as the interactions among the individual parts - with focus on the mobile scenario - is shown in Figure 2 (see next page). The mobile device (mobile phone or PDA) is equipped with the TUGeoWiki client and a Web browser. The client retrieves the current coordinates of the device either from an internal GPS sensor, or, via Bluetooth, from an external sensor. The TUGeoWiki client relays requests for upload of images to the mobile browser or directly to a server-sided application programming interface (API). Requests for information about the current location or requests for creating a new place for the current location are always relayed to the mobile browser. The browser is mainly used to access the adapted MediaWiki on the TUGeoWiki server, which shares a common database with the API. For each new entry, the Wiki displays a place template, which embeds a Google Map and hyperlinks (relaying the place’s coordinates) to

the Geohack extension as well as Google Maps and Microsoft Live Search Maps. A first version of this mobile client has been implemented using Java Mobile Edition (Java ME) to provide the basic functionality for a wide number of mobile devices. On the one side, this Java client provides access to (internal of Bluetooth-based) GPS sensors as well as to the mobile device’s camera. On the other side, the client forwards information about the current location of the device to the mobile browser. In turn, the mobile browser is used to access the TUGeoWiki server side application. The Java ME application acts only as a tool to provide data for the browser but does not access the TUGeoWiki server itself. Additionally, native applications for Symbian OS (respectively the S60 platform) as well as Android and the iPhone have been implemented. These applications provide the same features as the Java client, but are using a different software design. They access a server side API which is used to query, add and modify the Wiki data without the use of a Web browser, offering an alternative mobile user interface. An iPhone client for the TUGeoWiki system is currently under development. 101

The Integration of Aspects of Geo-Tagging and Microblogging in m-Learning

4. APPLICATION SCENARIOS As far as the application of TUGeoWiki for educational purposes is concerned, some scenarios have been identified to describe possible use cases. These scenarios are based on the desktop usage or mobile usage of TUGeoWiki and therefore categorised as e-Learning, respectively m-Learning scenarios.

4.1 Generic e-Learning Scenarios TUGeoWiki is designed to be usable in e-Learning scenarios without a mobile learning component. In this case, solely the Web application is used, without the mobile application. The two main scenarios for this application are, again, focused on the support of field trips. It is relevant to state here that we use the term generic to practically abbreviate the user-free generation character of our application by using the template-based extensions of the Wiki. The first scenario includes the preparation of students for field trips. For this scenario, various situations can be identified, where it is preferable that students have engaged a-priori information on the locations they are going to visit. For example, teachers may create place stubs in the Wiki before the trip. Another example is the integration of short articles into the trip definition. These short articles, already geo-tagged for a certain location, may contain little or no further data but the location’s title. These examples of a-priori data can be extended or modified by the students in advance, either collaboratively or in individual work. At his point consider also that the version history feature of the Wiki offers the teacher a valuable possibility to monitor the distribution of the work done over time as well as the individual contributions in collaborative tasks. The second possible generic sample scenario focuses on post-processing the information gained on a field trip. In a first step teacher provide an empty or rudimentary structure of places, which

102

already contain relevant coordinates. These so-called place stubs are used by students as anchors to add information gained in-situ at the corresponding locations. Moreover, geo-tagged images can later-on be added to the corresponding places using the image upload facility. Another alternative for in-the-field usage of the mobile TUGeoWiki application is the inclusion of a geo-tracker. A geo-tracker is an external device that logs geographical coordinates to timestamps, in other words, such devices produce tracks. After a field trip, the tracks can be used to post-process images by synchronising the coordinates of the tracks with the creation times of images. For this purpose, there already exist software tools that can be used to add the coordinates into the images’ EXIF header information; these (a-posteriori) geotagged images can be uploaded to the TUGeoWiki system and used to find fitting places in the Wiki.

4.2 Generic m-Learning Scenarios Considering the issues stated so far, one of the core concepts of the TUGeoWiki solution approach is its design and development for its use in-the-field. From our point of view, the most relevant learning scenario in this context is the use of mobile devices (e.g., mobile phones) to access the TUGeoWiki application. These mobile devices are used to retrieve the coordinates of the current location and access places of the TUGeoWiki in the vicinity of this location. Hence, the application is used to provide background information about the current location area, i.e. the learning materials precompiled by a teacher are delivered to students in a geospatial information context. The students can use this information during the process of learning in order to better understand relations of a location to theoretical concepts or other locations (Lonsdale, Baber, Sharples, 2004). The second learning scenario in the mobile context is the application of TUGeoWiki for the compilation of geo-tagged information on real-world locations by the students. On the one

The Integration of Aspects of Geo-Tagging and Microblogging in m-Learning

hand, textual information can be added to existing or newly created places. On the other hand, the mobile device can be used to create geo-tagged images of a location and add them to a place in TUGeoWiki. In this scenario, the a-priori creation of place-stubs in TUGeoWiki by the teacher is advisable in order to provide a core skeleton of the intended structure. Thus, the impact on the learning process lies in the compilation of the information itself, in the digestion of the direct experience gained during the field trip as well as in the informal learning during this task (Specht, Kaibel, Apelt, 2005). The previously given scenario can be extended by a collaborative component. The features of the Wiki allow several students to work on the same places and collaboratively compile geotagged information. Here, the learning process is supposed to be enhanced by discussions and by the need to create a unified perspective on the location area. The version history of the Wiki provides means for personal accountability of the students for their individual parts in the final work, which represents a central prerequisite for effective cooperative learning (Johnson & Johnson, 1994). The advantage of this third mobile scenario is that the students can do their work in the real-world by direct interaction with the locations concerned. Alternative scenarios can also be implemented with TUGeoWiki with one part of the collaborating students in-situ and another part of them working remotely, as proposed by (Kravcik, Specht, Kaibel, Terrenghi, 2008).

5. PRELIMINARY EVALUATION In the course of the evaluation of TUGeoWiki, three detailed scenarios where successively developed. After the initial development, our geo-Wiki approach was tested in a field trip of civil engineers, applying a variant of field trip post-processing. In this scenario, a lecturer was equipped with a Nokia N956 mobile phone, includ-

ing an internal GPS receiver and the TUGeoWiki mobile application. The lecturer was asked to use the phone to create geo-tagged images of the field trip, which were subsequently uploaded to TUGeoWiki and assigned to didactically relevant places. In this scenario, TUGeoWiki was only used as an application to create and provide geo-tagged learning material. The evaluation of this scenario aimed at providing basic feedback on the workflow of compiling information and images in-the-field as well as on the application of TUGeoWiki from the desktop. The feedback was collected through short interviews after the excursion. Among others, the lecturer stated that taking the photos with the mobile device was easily possible, although the localisation with GPS posed some problems, e.g., the initial synchronisation with the GPS signal can take several minutes, and GPS is not available without line-of-sight to the corresponding satellites. The upload of and subsequent search for existing places in a user-defined radius was perceived as extremely useful. As far as future development is concerned, the integration of additional data for a location, like geological or hydraulic data, was encouraged. The application of TUGeoWiki for field trip post-processing was well perceived, but the interviewed lecturer also pointed out the possible advantages of the application in a collaborative scenario. As a second detailed experiment, collaborative post-processing of field trips was implemented for another civil engineering field trip in a follow-up study. For this purpose, the students were equipped with a mixed technological equipment of digital cameras and one Nokia N95. Further, the teacher was equipped with a Holux M-2417 external GPS tracker and the students were asked to synchronise the time settings of the cameras with the data tracker. Images for the creation of the field trip report were taken collaboratively throughout the trip and, subsequently, geo-tagged using the GPS tracker. The resulting images were uploaded onto the TUGeoWiki system and relayed to the places

103

The Integration of Aspects of Geo-Tagging and Microblogging in m-Learning

created for the field trip. One-on-one interviews were conducted with the participants of the field trip to investigate the usefulness of this scenario. The possibility to identify locations of the individual images by geo-tagging and TUGeoWiki was generally well perceived, although several of the participants stated the fact that several places in the Wiki were created for the same real-world locations by different users. Moreover the feedback included the requirement of an bulk upload page. The original upload-page was designed according to the standard MediaWiki upload page to upload one image at a time. A bulk-upload page was subsequently implemented to allow the simultaneous upload of an arbitrary number of images for one location. The third detailed scenario implemented for the evaluation of TUGeoWiki focuses on a geology field trip. TUGeoWiki was used for the students to get prepared for the trip. The scenario combines the first generic e-Learning scenario (trip preparation) with the third generic m-Learning scenario (collaborative work in-the-field). The preparation of this scenario revealed the need for an extension of TUGeoWiki’s content paradigm. While the previous experiments had shown the basic usefulness of the place paradigm in a civil engineering scenario (which is basically focused on building sites), geologists have extended requirements, as information can rarely be mapped to individual (point-based) locations. Two additional paradigms for areas and tracks need to be supported. An area is represented by a polygon on a map and is useful for the description of larger-scale geological conditions. A track is represented by a line connecting a number of locations and describes an actual sequence of locations visited in the course of the trip. The implementation of these additional features and their evaluation represent currently an ongoing work. The preparations for the evaluations have been conducted as follows. A set of place stubs was created and prepared for the field trip. The places were collected into a Wiki category and collaboratively filled with information by

104

groups of students assigned to them. During the field trip, the group will be equipped with digital cameras, a mobile device and the GPS tracker. On the one hand, the mobile device will be used to add information in-the-field, and thus extend the previously prepared articles. And on the other hand, the images will subsequently be geo-tagged, uploaded and added to the existing places in order to enrich them with visual information from the actual trip.

6. MICROBLOG INTEGRATION So far, the two possible application scenarios for TUGeoWiki, mobile application scenario and desktop application scenario, have been described. It has been shown that the mobile scenario is mainly focused on the satisfaction of ad-hoc learning needs as well as on proactive information push to the Wiki, rather than on collaborative editing of contents. So far, the examples given for such an information push were the creation of place stubs and the extension of places with geo-tagged images. This approach, however, lacks a possibility to easily share information about and across images via the mobile application. For any textual information added to individual locations, the standard MediaWiki edit functionality must be accessed with the mobile browser. First evaluations with a small group of users showed that this functionality was perceived as cumbersome and avoided as far as possible. These reactions of the evaluation subjects are assumed to be a result of the typical mobile phones limitations regarding their small screen sizes and the complexity of writing with mobile or virtual keyboards. In order to solve this problem and to simplify the interactions with the Wiki, we propose the usage of an alternative technique, which is based on the principle of adding (small) notes to existing articles as a foundation for collaborative activities within our desktop application scenario. In analogy to the Microblogging paradigm (Templeton,

The Integration of Aspects of Geo-Tagging and Microblogging in m-Learning

Figure 3. Example for a TUGeoWiki micropage with one annotation

2008), short messages are sent by the users and integrated into the Wiki, creating Micropages.

6.1 The Notion of Micropages Due to the fact that the number of mobile devices connected to the World Wide Web is growing tremendously fast, microblogging has become one of the most interesting innovative applications at present. Microblogging can be seen as a variant of blogging, where small messages, usually not longer than 140 characters, are posted instantly and on-demand to Web-based microblogging services. According to (Templeton, 2008), microblogging can be defined as “a small-scale form of blogging, generally made up of short, succinct messages, used by both consumers and businesses to share news, post status updates, and carry on conversation”. Regarding the intentions of users, the following four categories are identified: daily chatter, conversations, sharing information, and reporting news (Java, Song, Finin, Tseng, 2007). Further, research work shows that microblogging is very useful for the fast exchange of thoughts and ideas as well as for a fast information sharing (Ebner & Schiefner, 2008). Considering the growing importance of mobility and mobile applications, Twitter (the largest microblogging platform

worldwide) became one of the prime examples for Mobile 2.0 (Griswold, 2007). To characterise the notion of Wiki pages that are based on small individual information pushes, we apply the term micropages. Thus, micropages are the Wiki analogy of microblogs, which describes our approach focusing on smaller parts of information. In a microblog, brief text updates are used as status messages to publish information for friends and other “followers”. By encouraging shorter posts, microblogging can fulfil a need for a faster form of communication (Java, Song, Finin, Tseng, 2007). Within the scope of this book chapter, we propose to use micropages as Wiki pages that are built out of short individual annotations on the topic of the page. In TUGeoWiki, each of these topics is a location, and each page is a place. Figure 3 depicts one example for a micropage in TUGeoWiki containing one annotation. The creation process of such a micropage by means of the Wiki’s special pages is depicted in Figure 4. A short message is created (usually on a mobile device) and geo-tagged with the user’s current location. A special page is used to find an appropriate place or create a new one, and to append the message at the end of the micropage. The building parts of a TUGeoWiki micropage are derived from the received messages and always appended in chronological order (earliest on top)

105

The Integration of Aspects of Geo-Tagging and Microblogging in m-Learning

Figure 4. General notion for creating micropages

instead of in reverse chronological order (as in weblogs) to better address the Wiki page paradigm. These parts are furthermore tagged with some metadata, such as the author’s username, the date and time of the post. Unlike microblogs, our micropages are not intended to serve as means for synchronous communication; so far, we share with the microblogging paradigm just the concept of information push of short messages. Further, a new micropage is not intended to represent a final content within the Wiki system; rather (as for Wiki contents in general) it should be iteratively revised and improved to a final form via collaborative authoring. In concrete, micropages represent stubs for content in a Wiki, i.e., short annotations added to “sketch” the final page anytime, anywhere, and in the case of TUGeoWiki in-the-field and justin-time.

the current position from the built-in GPS sensor and relays it to the server, which returns the list of existing places within the chosen area. On client-side, the user chooses either one of the existing places to annotate or creates a new place by entering a title. As previously mentioned, the message is then attached at the end of the place, accompanied by the user’s username as well as the date and time of the post. Some sample screenshots of the TUGeoWiki Android client during the annotation workflow are shown in Figure 5 and Figure 6. In the first screenshot, at left side of the figure, a message is composed. The second screenshot shows the selection of the distance for searching existing places. The third screenshot displays the list of existing places retrieved. Finally, the fourth screenshot displays the message included in one of these places, on the Wiki at server side.

6.2 Using Micropages with the Mobile Client

6.3 Integrating Microblogging Services

Micropages are currently supported by TUGeoWiki’s Android and iPhone client. In both versions, the annotation attached to a micropage has been implemented as for the upload of images. This process is described in the following. In a first step, the user writes a message (of 140 characters at most) to annotate her current location and chooses a distance from the current location for the search of suiting existing places in her vicinity. The client subsequently retrieves

Another possible source for the creation of micropages is the integration of a so-called microblogging service. The analogy of micropages and microblogs inspired us to define an additional user interface. As stated before in this section, the annotation feature of the mobile client is purely intended for in-the-field and just-in-time annotations of geospatial information by sending short messages that describe the current location. Due to the fact that the location is determined via GPS

106

The Integration of Aspects of Geo-Tagging and Microblogging in m-Learning

Figure 5. Screenshot of annotation feature of the mobile Android client

coordinates, a later annotation of resources is not feasible. The problem in this context arises when short annotations to already existing Wiki places are interesting for users after visiting the location, thus the following alternative user interaction seems to be userful. In TUGeoWiki, we integrated microblogging services that support Twitter posts. We have chosen the Twitter service because it is a well-known microblogging application with a well-defined API. Against the background described so far, a very interesting aspect of microblogging gained our attention: filtering information using a unique

letter. This technique is referred to as hash tagging and has been introduced on several microblogging platforms. It is used for search queries or marking special content. Hashtags are a simple way of grouping messages with a “#” sign followed by a name or special code. (Templeton, 2008) Hashtags in microblogs are especially meaningful when used during a particular period of time, as “it not only allows individuals to generate a resource based on that specific thematic, while using the hashtag, but also bridges knowledge, and knowing, across networks of interest”. (Reinhardt, Ebner, Beham, Costa, 2009)

Figure 6. Screenshot of annotation feature of the mobile iPhone client

107

The Integration of Aspects of Geo-Tagging and Microblogging in m-Learning

On the server side of the TUGeoWiki system, users may use a special page to mark a place as “microbloggable”. Moreover, individual users are provided a feature to append their Twitter user names to their user profiles. This information is relayed to a Web-based service, which periodically scans the registered users’ microblogs for tweets containing the hashtag “#tgw”, indicating a TUGeoWiki annotation message. This tweet must contain a second hashtag identifying the place via an URL. This hashtag is created using the URL shortening service bit.ly, which creates a 5-letter hash of an URL. Thus, for example, shortening the URL http://media.iicm. tugraz.at/geowiki/index. php/LKH_Klagenfurt_Neu results in the URL http://bit.ly/jBVbX. The corresponding hashtag, #jBVbX, is created for a place when marking it as microbloggable and added to the TUGeoWiki template. After identifying the TUGeoWiki-specific hashtags, the remainder of the Twitter post is added to the corresponding TUGeoWiki place as a new annotation signed with the corresponding user name, date and time.

7. SUMMARY AND CONCLUSION Already with the first uses of Wikis for education, it became rather clear that they would generate a great benefit for collaborative activities among learner groups. A lot of research work has been carried out in order to show that process-oriented learning is supported by Wikis in a very novel and smart way (Ebner, Zechner, Holzinger, 2006); (Ebner, Kickmeier-Rust, Holzinger, 2008). The lack of existing tools for the incorporation of geotagged resources into the learning activities of e.g. civil engineers or architects, lead to the development of our TUGeoWiki system. Our solution approach provides a possibility to collaborate on geo-tagged information in a Wiki, based on the concept of places as individual articles. Moreover,

108

it provides means for learning in-the-field by uploading geo-tagged images and, with the help of microblogging, also geo-tagged messages. In summary, it can be pointed out that our approach contributes to the enhancement of the collaborative activities between learners by enabling them to feed and compose geo-information with personal annotations (i.e., with the mobile part of the TUGeoWiki system) into a user-friendly environment for mutual authoring (i.e., the Wikibased server side of our system). In further studies and field experiments we will explore and evaluate how the underlying implementation framework finds applicability and usefulness in other research areas.

REFERENCES Brohn, D. (1983). Academic priorities in structural engineering - the importance of a visual schema. The Structural Engineer, 61(1), 13–16. Downes, S. (2005). E-learning 2.0. eLearn, 10(1). Ebner, M., & Holzinger, A. (2003). Instructional use of engineering visualization: interactiondesign in e-learning for civil engineering. In Human-computer interaction, theory and practice. Mahwah, NJ: Lawrence Erlbaum. Ebner, M., Kickmeier-Rust, M. D., & Holzinger, A. (2008). Utilizing Wiki-Systems in higher education classes: a chance for universal access? In Universal Access in the Information Society. Berlin, New York. Heidelberg: Springer. Ebner, M., Scerbakov, N., & Maurer, H. (2006). New Features for eLearning in Higher Education for Civil Engineering. Journal of Universal Computer Science, 93-106. Ebner, M., & Schiefner, M. (2008). Microblogging - more than fun? In Proceedings of the IADIS Mobile Learning Conference, 2008, 155–159.

The Integration of Aspects of Geo-Tagging and Microblogging in m-Learning

Ebner, M., Zechner, J., & Holzinger, A. (2006). Why is Wikipedia so Successful? Experiences in Establishing the Principles in Higher Education. In Proceedings of the 6th International Conference on Knowledge Management (I-KNOW 06), 527-535. Fucks-Kittowski, F., Köhler, A., & Fuhr, D. (2004). Roughing up Processes the wiki Way - Knowlede Communities in the Context of Work and Learning Processes. In Proceedings of the I-Know, 2004, 484–493. Griswold, W. G. (2007). Five enablers for Mobile 2.0. Computer, 40(10), 96–98. doi:10.1109/ MC.2007.346 Holzinger, A., & Ebner, M. (2005). Visualization, Animation and Simulation for Mobile Computers: Experiences from Prototypes. In Proceedings of the Central European Multimedia and Virtual Reality Conference CEMVRC, 37-41. Java, A., Song, X., Finin, T., & Tseng, B. (2007). Why we twitter: understanding microblogging usage and communities. In Proceedings of the 9th WebKDD and 1st SNA-KDD 2007 workshop on Web mining and social network analysis. San Jose, CA: ACM Johnson, R. T., & Johnson, D. W. (1994). An overview of cooporative learning. In Creativity and Collaborative Learning: The Practical Guide to Empowering Students and Teachers, Baltimore, MD: Brookes Press Klamma, R., Chatti, M. A., Duval, E., Hummel, H., Hvannberg, E. T., & Kravcik, M. (2007). Social software for life-long learning. Journal of Educational Technology & Society, 10(3), 72–83. Kravcik, M., Specht, M., Kaibel, A., &Terrenghi, L. (2008). Collecting data on field trips - RAFT approach. City, 478. Leuf, B., & Cunningham, W. (2001). The wiki Way. Quick Collaboration on the Web. Boston, MA: Addison-Wesley.

Lonsdale, P., Baber, C., & Sharples, M. (2004). A Context Awareness Architecture for Facilitating Mobile Learning. In Learning with Mobile Devices (pp. 79–85). London: Research and Development, Learning and Skills Development Agency. Mediawiki. (n.d.). Help:Templates. Retrieved from http://www.mediawiki.org/w/ index. php?title=Help: Templates&oldid=163132 Mediawiki. (n.d.). Manual:Special pages. Retrieved from http://www.mediawiki.org/w/ index. php?title=Manual: Special_pages&oldid =171875 Norris, C., & Soloway, E. (2004). Envisioning the Handheld-Centric Classroom. Journal of Educational Computing Research, 30(4), 281–294. doi:10.2190/MBPJ-L35D-C4K6-AQB8 Parsons, D., Ryu, H., & Cranshaw, M. (2006). A Study of Design Requirements for Mobile Learning Environments. In Proceedings of the Proceedings of the Sixth IEEE International Conference on Advanced Learning Technologies. IEEE Computer Society. Reinhardt, W., Ebner, M., Beham, G., & Costa, C. (2009). How People are Using Twitter during Conferences. In Proceedings of the 5th EduMedia conference, Salzbrug, Austria, 145-156. Safran, C., Garcia-Barrios, V., & Ebner, M. (2009). The Benefits of Geo-Tagging and Microblogging in m-Learning: a Use Case. In Proceedings of the MindTrek 2009 Conference. Tampere, Finland: ACM. Safran, C., & Zaka, B. (2008). A geospatial Wiki for m-Learning. In Proceedings of the International Conference on Computer Science and Software Engineering (Wuhan, China, 2008-12-12 - 200812-14, 2008). IEEE Computer Society, 109-112. Specht, M., Kaibel, A., & Apelt, S. (2005). Extending LCMS for remote accessible field trips in RAFT. City, 302-306.

109

The Integration of Aspects of Geo-Tagging and Microblogging in m-Learning

Tatar, D., Roschelle, J., Vahey, P., & Penuel, W. R. (2004). Handhelds Go to School: Lessons Learned. Computer, 36(9), 30–37. doi:10.1109/ MC.2003.1231192 Templeton, M. (2008). Microblogging Defined. Retrieved from http://microblink.com/ 2008/11/11/ microblogging-defined/ Tretiakov, A. (2008). anKinshuk Towards designing m-learning systems for maximal likelihood of acceptance. International Journal of Engineering Education, 24(1), 79–83. Voelkel, M., & Oren, E. (2006). Towards a Wiki Interchange Format (WIF). In Proceedings of the First Workshop on Semantic Wikis -- From Wiki To Semantics. Wikipedia. (n.d.). WikiProject Geographical coordinates. Retrieved from http://en.wikipedia. org/w/ index.php?title=Wikipedia: WikiProject_ Geographical _coordinates&oldid =174960235

110

ENDNOTES 3 1 2

4



5



6

7



GPS – Global Positioning System Exif – Exchangeable Image File Format TUGeoWiki – TU Graz (Graz University of Technology) Geospatial Wiki https://wave.google.com/, accessed 201003-15 PDA - Personal Digital Assistant http://www.nokiausa.com/ link?cid=PLAIN_TEXT_430087, accessed 2009-04-22 http://www.holux.com/JCore/en/products/ products_content.jsp?pno=341, accessed 2009-04-22

111

Chapter 6

Teaching Group Decision Making Skills to Emergency Managers via Digital Games Conor Linehan University of Lincoln, UK

Nina Haferkamp University of Muenster, Germany

Shaun Lawson University of Lincoln, UK

Nicole C. Krämer University of Duisburg-Essen, Germany

Mark Doughty University of Lincoln, UK

Massimiliano Schembri University of Naples & Institute of Cognitive Sciences and Technologies (ISTC-CNR), Italy

Ben Kirman University of Lincoln, UK

Maria Luisa Nigrelli University of Naples & Institute of Cognitive Sciences and Technologies (ISTC-CNR), Italy

ABSTRACT This chapter discusses how a focus on establishing the appropriate learning outcomes of an educational programme, and creatively incorporating these learning outcomes within the design of a game, can lead to the development of a useful educational game. Specifically, it demonstrates the process involved in generating game design criteria from a multi-disciplinary literature review. The design of a game that has been developed as part of a project to train emergency managers in group decision making and communications skills is presented, along with some initial evaluations of that game design. It appears that the game presented can function as a valid practical element of a programme for the training of group decision making and communication skills with emergency management personnel. DOI: 10.4018/978-1-60960-774-6.ch006

Copyright © 2012, IGI Global. Copying or distributing in print or electronic forms without written permission of IGI Global is prohibited.

Teaching Group Decision Making Skills to Emergency Managers via Digital Games

INTRODUCTION Games have recently been suggested as effective media for delivering educational content and for helping students to reach educational goals (Gee, 2003; Greitzer, Kuchar, & Huston, 2007; Kelly, Howell, Glinert, Holding, Swain, Burrowbridge & Roper, 2007; Pivec & Kearney, 2007). Specifically, the combining of psychological research and games design principles offers a framework for developing educational games that promote learning while maintaining high motivation of the players (Siang & Rao, 2003). Understanding how to create an effective educational programme based on game playing is an inherently multidisciplinary task, requiring expertise in Pedagogy, Human-Computer Interaction, Psychology and Games Design, in addition to extensive knowledge of the subject domain of interest. The current chapter demonstrates how an understanding of the appropriate learning outcomes of the educational programme, and a strong focus on incorporating these learning outcomes within the game design, can lead to the creation of a useful educational game. This chapter will deal with games designed with the intent to teach demonstrable and generalisable skills to those who play them. The work presented was carried out as part of the “Leonardo” project “DREAD-ED: Disaster Readiness through Education” funded by the EU Lifelong Learning Program (see http://www.dread-ed.eu/). The chapter will discuss the challenges faced in developing a game to teach group decision-making and communication skills to groups tasked with managing emergency events such as as floods, fires, volcanoes and chemical spills. While this is a very specific game design task, it involves considerations common to the design of all educational games and we will make recommendations on best practice regarding these considerations. The chapter will begin with some background on the task of managing emergencies and will discuss existing methods for training emergency

112

managers. The opportunities and challenges presented by using computer games to teach relevant skills to emergency managers will then be introduced. We will discuss issues common to all educational game design, such as providing timely and specific feedback to participants and ensuring that the target skill of the educational programme is intrinsic to game play. Challenges specific to the current project, such as the problem of teaching people who are already experts in their domain and that of understanding group decision making behaviour will be presented. A set of design requirements will then be formulated. We will then present the design of the DREADED game and describe how this design fulfills the outlined design requirements. An evaluation of that game design will then be presented, based on three separate studies, the results of which will be combined here. Conclusions on these studies and the project in general will then be presented. In the final section we will discuss future research directions for the field of educational games, both in terms of soft skills training projects and also more generally. The current chapter provides a valuable contribution to the current book as a detailed case study on how to approach the design and evaluation of a game for a very specific purpose and audience; that of training group decision making skills to emergency managers. It is intended that the approach taken here may be of use not only to those interested in emergency management, but may also serve as an exemplar on how to approach the design of games for very specific purposes in future. This chapter is an extension of an earlier publication (Linehan, Lawson, Doughty & Kirman, 2009) and presents a more complete description of the design challenges and solutions than was possible in the earlier work. In addition, the third experiment reported here was not included in the earlier paper.

Teaching Group Decision Making Skills to Emergency Managers via Digital Games

BACKGROUND In the following section we detail the challenges faced by any project that intends to teach useful group decision making skills to emergency managers. We discuss the state of the art in training those personnel, both via traditional and technologyenhanced methods.

Emergency Management As the events of recent years have graphically illustrated, nowhere in the world is immune from natural disasters and emergency situations. Advance planning and preparation for emergency management personnel is a critical factor in reducing mortality and damage caused by these unpredictable events. Those responsible for managing emergencies are managed by groups of people drawn from various disciplines and agencies and must learn how to react to unpredictable and fast evolving events. As emergency ‘events’ can vary in scope from localised corporate sites and their immediate environs to events that are spread out over large geographic areas, a broad education strategy combining domain specific knowledge and ‘softer’ skills such as communication and group decision making is required (Crichton & Flin, 2001). Therefore, the learning activities for those involved in the management of disaster and emergency response should ideally incorporate, for example, communication and understanding of information under conditions of stress, problemsolving with partial or contradictory information and decision-making in the face of competing demands. (Kowalski-Trakofler & Scharf, 2003). The DREAD-ED project was designed to train these ‘soft’ skills.

Training Emergency Managers Ideally, all emergency management personnel should have hands-on practical experience dealing with real disasters as part of their training.

However, thankfully, emergency events do not occur with the required frequency to provide this sort of experience as part of a training course. The next best methodology, and one that is commonly implemented both by local authorities and industry, is to create real-world role-playing simulations (Chrichton & Flin, 2001). Such tasks allow emergency management personnel to experience the stress of dealing with a real emergency, as well as learning the utility of the procedures learned on training courses, without the risks involved with exposure to a real-world emergency (Kincaid, Donovan and Pettitt, 2003). Personnel can clearly see the consequences of the decisions they make, and through post-hoc de-briefing sessions, can reflect on the manner in which they dealt with the situation. Unfortunately, real-world emergency simulations cost a great deal of both money and manpower to implement (Balducelli, Bologna, & Di Costanzo, 1995). This is due to the large amount of staff required to engage with the emergency management personnel as part of the exercise, but who are themselves not learning a great deal. As a result, emergency management teams typically have the opportunity to engage in such training only once a year. As a result, paper based problem solving exercises have been developed in order to provide emergency managers with the opportunity to practice leadership, command, decision making, communication and teamwork skills. Such exercises allow students to engage with the material on a functional level (i.e., understand why they learned the material on the course and why the methods that are taught are better than other ways of approaching the same situations). However, these paper-based classroom exercises do not offer an engaging environment, and thus poorly simulate the stress induced by real disaster scenarios (Jain & McLean, 2005). Emergency managers who make good decisions in a calm well-defined problem solving environment may not necessarily prove to be as capable under high amounts of stress (Chrichton & Flin, 2001).

113

Teaching Group Decision Making Skills to Emergency Managers via Digital Games

Training Emergency Managers with Computer Games Crucially, modern video game technology appears to combine the engagement and realism of a realworld role play session with the cost-effectiveness of paper-based classroom role playing (Sanders & Rhodes, 2007). This technology is capable of presenting content such as video clips, emails, SMS messaging, and maps to multiple users in real time. In addition it is capable of presenting incomplete information to some or all team members and simulating the general disorder of a real emergency management room (Sanders & Rhodes, 2007). Video game technology also has the advantage that all characters or ‘agents’ in the simulation, apart from the actual personnel who the training has been targeted at, can be simulated. Thus, a great deal of the expense involved in running these exercises is eliminated. The concept of using video game and roleplaying environments to situate scenario based training is not a new one. Business and economic simulations have made use of many of the features of computer and role-playing games in order to provide repeatable and relevant decision making training opportunities for learners in these fields (i.e., Hsu, 1989; Wolfe & Rogé, 1997). In addition, the use of virtual environments to provide 3D rendered views of an operational domain with information conveyed via interactive on-screen menus and heads up displays (HUDs) have also been exploited. Developments such as the First Responder Simulation and Training Environment (FiRSTE; http://firste.mst.edu/) and Play2Train (http://www.play2train.org) have both embraced the visual representation and immersive qualities of interactive 3D virtual environments. These 3D environments are useful because the learning outcomes specified by both the FiRSTE and Play2Train projects are based around the spatial planning of emergency response. The DREADED project specifies different learning outcomes, namely the development of communication and

114

group decision making skills, thus an immersive virtual 3D environment is not necessary. A novel game must be designed that is suitable for the learning outcomes of the DREAD-ED project.

FORMULATING DESIGN REQUIREMENTS FOR A GAME THAT TEACHES DECISION MAKING SKILLS The DREAD-ED game will form the experiential learning component of this larger training program that will also feature traditional classroom-based face-to-face training. As such, it is not intended for the game to explicitly teach the concepts of sound group decision making. Rather, the game is designed to represent a realistic environment in which to make decisions as a group. Essentially, the game provides the context in which to practice all of the relevant skills. Thus, the particular game design adopted must generate a game environment that accurately models the challenges faced by emergency managers when making decisions collaboratively under stressful and dynamically changing circumstances. In this way, a team that displays the appropriate skill should achieve a better ‘score,’ or outcome, on the game than a team that does not display the appropriate skills. Through guided repetition of the exercise, all teams trained on the DREAD-ED game should have the opportunity to demonstrate the appropriate decision making skills under conditions of stress. In order to design a game that fulfills the learning objectives of an educational programme, it is essential to understand the appropriate and feasable learning outcomes of the training programme, and to incorporate them within the design requirements of the game. A broad multidisciplinary literature review of relevant previous work is an essential starting point for this task. Over the course of the DREAD-ED project, we investigated best practice in teaching via games, the difficulty involved in teaching skills via

Teaching Group Decision Making Skills to Emergency Managers via Digital Games

games, the problem of dealing with experts, and the challenge of understanding group behaviour.

Teaching via Games It could be argued that all commercial games are educational as they train players to be increasingly fluent at manipulating the system for gaining success within that game. The challenge of progression within a game provides motivation to continue learning (Malone and Lepper, 1987; Vorderer, Hartmann & Klimmt, 2003). Essentially, good games make the process of learning fun (Woods, 2004; Koster, 2005). This is precisely the reason why games have recently been seen as an exciting development in education. If designed correctly, serious games have the potential to harness the inherent motivation demonstrated by game players to teach skills that are of immediate practical benefit (Greitzer, Kuchar, & Huston, 2007). It is this intended transfer of game skills to real world activities is what ultimately differentiates serious games from entertainment games

Intrinsic Learning A successful serious game is one where the task learned in the game maps directly on to the challenge faced in the real world. This feature has been referred to by Habgood (2007) as intrinsic learning and by Bogost (2007) as procedural rhetoric. Both authors essentially refer to embedding the learning outcomes of the project within the mechanics of the game. Bogost analyses a number of serious games that are deficient in procedural rhetoric (p. 49-51) and also a number of games that excel in this respect (p. 29). Furthermore, Habgood investigated experimentally the importance of integrating learning content with the mechanics of a game. Specifically, in two studies, the author found that a game in which learning was intrinsic to game play was motivationally and educationally more effective than an almost identical game in which learning was not intrinsic to game play.

Thus, a successful serious game must locate the learning within the game play mechanics, rather than as an addition to the game play mechanics.

The Importance of Feedback Engaging computer games excel at providing immediate, appropriate and specific feedback to players. This feature is at the heart of the motivation, sustained attention, learning and fun experienced by game players (Siang & Rao, 2003; Loftus & Loftus, 1983). It is also a feature of any sound manual or cognitive skills training program and is a reliable predictor of future performance of those skills (Catania, 1998). In the context of training emergency management personnel, it is not enough for the player to know that they ‘won;’ they must know why they won. Specifically, in conducting any manual or cognitive skills training, one factor that reliably predicts future performance is the specificity of feedback provided by the training programme (i.e., feedback should be both instant and specific to the actions taken). For example, consider the approach taken by driving instructors. The instructor examines the behaviour of the learner as they drive and delivers timely and specific feedback concerning the proficiency of the learners’ driving. The instructor does not allow the learner to drive for thirty minutes before producing a list of mistakes. Clearly, this approach would not allow the learner to effectively discern which of the many actions taken were successful and which were not. Unfortunately, the post-hoc learning style mentioned above appears to be the approach adopted by a number of serious games designed to teach aspects of emergency management. Both BBCs’ Supervolcano game (http://www.bbc.co.uk/sn/ tvradio/programmes/supervolcano/game.shtml) and the UN/International Strategy for Disaster Reductions’ Stop Disasters Game (http://www. stopdisastersgame.org/en/) require players to make a large number of decisions in advance of an emergency and then hit the ‘play’ button. Play-

115

Teaching Group Decision Making Skills to Emergency Managers via Digital Games

ers then watch the disaster unfold and are given a score. However, this approach does not allow the player to easily discern which steps taken were necessary to success, and which were detrimental or had no effect. The power of feedback has been consistently demonstrated as a key variable in the process of learning over the past seventy years by behavioural psychologists working under the paradigm of operant conditioning (see Catania, 1998; Ferster, Skinner, Cheney, Morse & Dews, 1957; Skinner, 1953; 1959; for in-depth analysis of this topic). Interestingly, Loftus and Loftus (1983) conducted an in-depth analysis of computer game playing using the concept of operant conditioning. The authors draw comparisons between a person playing Pac-Man and a rat in one of B.F. Skinners classic behavioural experimental preparations. Operant conditioning, and specifically the process of reinforcement, is proposed by Loftus and Loftus as an explanation of game player’s sustained attention and motivation. The authors suggest that successful entertainment games excel at delivering the correct type of feedback (both positive and negative) at the correct time. Thus, it appears that educational games must learn from the success of entertainment games, as the process of providing clear, immediate and specific feedback is essential in shaping behaviour of game players.

Skills Training It must be noted that games designed to train skills also face different challenges to those designed to impart information. As it is intended that the skill learned in the game will transfer directly to the real world, participants in a skills training game should be engaged in precisely the same behaviour in the game environment that they would be in the real environment. The game environment should consist of features that correspond to reality, both in terms of the ‘choice architecture’ (the dynamic system of game mechanics in which decisions must be made) and also in the consequences of

116

behaviour. Successful real world behaviour should have positive game consequences.

Training Experts Most serious games are aimed at the general public and are designed to generate behaviour change in large amounts of people, typically through delivering information. The current project has very different goals. DREAD-ED aims to teach practical skills to small groups of people who already have a great deal of knowledge in the target domain. In other words, DREAD-ED aims to train people who are already experts in emergency management to do it better. It must be noted that some serious games do teach players skills via the presenting players with the opportunity of interacting with a model of the system that is being taught (such as McVideo Gamehttp://www.mcvideogame.com/, Redistricting Gamehttp://www.redistrictinggame.org/ and 3rd World Farmerhttp://www.3rdworldfarmer. com/). However, these games are also typically aimed at a large number of people and assume a starting point of little or no knowledge about the system. The resulting game design must be radically different to the type of information-driven serious games that are the norm. Indeed, a number of challenges are presented by the very expertise of these target participants. The first challenge presented by the task of training people who are already experts is the problem of maintaining their motivation to keep playing. Specifically, because the participants are experts, centering the game play on information can lead to problems. If any procedural or informational discrepancy exists between the game narrative and that which is the case in reality, it will be noticed by these expert participants. This has the potential to break the participants’ engagement with the game environment, thus undermining the usefulness of the game as an engaging learning tool.

Teaching Group Decision Making Skills to Emergency Managers via Digital Games

The second challenge presented is the necessity for learning outcomes to be generalisable to a large number of different possible events. Specifically, because the procedural knowledge required to deal with an emergency is so specific for each different type of event, a game that is based on 100% accurate information for one particular emergency will not be generalisable to the countless other different events that these participants may have to deal with. A game designed to train people who manage forest fires in Spain will be of little use to a team that manages flood emergencies in France, if procedural knowledge and information is the primary focus of game play. Rather than focus on the information and procedures of management, for which training courses already exist, we have decided to focus on training generalisable group decision making skills using an abstract model of a developing emergency event. In this way everyone who plays the game will gain benefits. This approach will circumvent the problems of generalisability and of the potential for incorrect information to disengage players from the game. In addition, there is currently a lack of pedagogically sound, engaging courses designed to teach group decision making behaviours.

Decision-Making Groups Emergencies are typically managed by groups of people drawn from the emergency services, local authorities, and relevant private stakeholders. These groups must gather information on the current situation and develop a plan in order to minimise the possibility of casualties and damage to property caused by the event. Decision making groups are formed on the expectation that decisions made by the group as a whole should be better informed, more considered and ultimately more successful than decisions made by individuals. However, decades of research have demonstrated that groups typically make bad decisions (Janis, 1972; Karau & Williams, 1993; Kerr & Tindale,

2004; Stasser & Titus, 1985; Steiner, 1972). Few studies have reported that groups have performed as well as their best member would have individually, and fewer studies still have reported group performance that is better than the performance of any individual efforts. Thus, it appears that efforts should be made to identify a qualified individual and let that person make decisions rather than forming groups to do so. Unfortunately, real world emergencies must necessarily be managed by groups. It is not possible to set one person to manage an emergency, as one person on their own will never have access to all of the relevant information needed to manage the situation. Even if one person is ultimately responsible for the most important decisions made during an emergency event, this one person must still deal with a group of subordinates who gather and process information in a similar way to that observed in group decision making. Regardless of whether the team is set up with an authoritarian or democratic decision making structure, the core elements of information gathering and processing are omnipresent.

Design Criteria The above literature review has identified requirements essential for the design of a game that effectively teaches group decision-making skills to Emergency Managers. Specifically, in order to create a game to train groups in decision making skills, it is necessary to provide a game environment that resembles a real decision making environment as closely as possible. All decision making groups must perform an information search, share this information in a structured manner, must keep track of which member knows what information, must participate equally to ensure that a minority of members do not become dominant and must actively listen to and consider minority in order to avoid the symptoms of groupthink. In making a decision, these groups must combine the knowledge of how the environment works with

117

Teaching Group Decision Making Skills to Emergency Managers via Digital Games

the resources available in order to make decisions that are most beneficial. In addition, groups tasked with emergency decision making must deal with time pressure and stress, which tends to narrow a groups focus and leads to mistakes (see Linehan, Lawson, Doughty & Kirman, 2009a, for a more detailed discussion of how group processes impact on the design of the current project). Thus, a game designed to train group decision making should present players with these exact challenges. In addition, the game must provide timely and specific feedback to participants on their performance, must incorporate the learning outcome of information search and communication within the game mechanic, and must be based on an abstract model of a developing emergency event rather than domain specific information.

GAME DESIGN A game design was created based on the requirements identified by the literature review. The game places players in an emergency management team that is dealing with a developing emergency. Each team member is assigned a role that has unique abilities within the game. The information that is needed to solve the problem posed by the game is distributed among all game players in the form of personnel. In order to successfully manage the situation, personnel must be exchanged between group members. All players must effectively communicate their unique information to the other players and appraise the many courses of action available before making decisions. Because information is distributed between players information gathering and processing is required for success. Groups that do not communicate all relevant information necessarily have less chance of receiving positive feedback from the game than those that do. Thus, learning is embedded within the game play mechanics and the game state itself should provide feedback on how well the group is performing.

118

Goal The challenge presented by the game lies in managing the dynamically changing game state, which is represented by four abstract six-point scales (see Figure 1). The inclusion of abstract scales to represent the game state ensures that domain specific information is not required to solve the challenges posed by the game. Each scale represents an individual aspect of the emergency that can vary from 1 to 6, representing ‘perfect’ to ‘disaster.’ These scales are labeled as ‘casualties,’ ‘hazard risk,’ ‘operations,’ and ‘Public Relations’ (PR). The ‘casualties’ scale is the most important of the four scales in terms of evaluating team performance. If the casualty scale reaches its maximum, the team has lost the game. Conversely, if the management team ensures that the ‘casualties’ parameter does not increase, then they have completed the task successfully. Events, or ‘injects’ of information that alter the game state in an unpredictable fashion are introduced at specific points in order to model the dynamically changing nature of an emergency. This feature is designed to force players to plan in advance for unforeseen circumstances, as well as dealing with issues of immediate importance.

Game Mechanic The game mechanic is based on assembling and deploying teams of personnel in order to affect the values displayed on the game state scales. Each of the nine personnel classes has a unique effect upon the game state when deployed. In addition, each player character, or role, has a unique ability, some of which relate to particular personnel classes. Personnel classes are represented abstractly in the form of colours within the game. This abstract representation ensures that domain specific information is not required to solve the challenges posed by the game and that the skills learned to be successful at the game are more likely to be generalisable to situations

Teaching Group Decision Making Skills to Emergency Managers via Digital Games

Figure 1. Paper prototype representation of the game state

other than that specifically presented in the game. A high-achieving group will excel at getting the right personnel to the right players at the right time in order to control the emergency. A further mechanic was developed that limits the number of actions available to the group each round. This mechanic, coupled with the limited time available for discussion and collaboration, was designed to create a stressful decision making environment for participants.

of a game-play session. The first timed round is assigned four minutes for discussion and action, and each successive round is assigned twenty seconds less for discussion and action than the previous round. Once the full number of timed rounds has elapsed, an in-depth evaluation phase is initiated between the tutor and the participants.

Presenting Feedback

Three separate studies were carried out in order to verify whether the game met the outlined objectives. Studies 1 and 2 were based on a basic paper prototype using cards and a game board, while Study 3 was conducted with an online digital version of the game. The construction of a paper prototype allows for the careful examination of game mechanics without the development costs associated with an electronic version. Cards were used to represent the character roles, personnel classes and event injects, while a game board was used to represent the game state and also to keep track of the number of actions taken by groups in each round of the game.

The game has been carefully designed to present an environment where it is advantageous to engage in the appropriate group decision making and communication behaviors (see Linehan, Lawson, Doughty and Kirman, 2009a). Groups that do not work collaboratively to solve the problems presented in the game should perform poorly. In this way, the learning outcome is embedded within the game play mechanics and the game state itself should provide feedback on how well the group is performing in terms of soft skills. In addition to the ongoing feedback delivered by the game state, the game has been carefully structured to work in rounds, each separated by a phase in which a tutor has the opportunity to give more detailed feedback to players without interrupting the flow

EVALUATION

119

Teaching Group Decision Making Skills to Emergency Managers via Digital Games

Study 1: Examining Game Mechanics with a Tabletop Prototype Study 1 was designed to verify whether the game provides appropriate feedback to players (see Linehan, Lawson, Doughty & Kirman, 2009a, for a full description of the study). Specifically, the study focused on determining whether groups that performed well at playing the game were also the groups that displayed appropriate group decision making behaviours. Relevant dependent measures identified in the literature include equality of participation (Watson, DeSanctis & Poole, 1988; DiMicco, Pandolfo, & Bender, 2004), absolute amount of interaction (Buder & Bodemer, 2008), overlapping speaking time (Kim, Chang, Holland & Pentland., 2008) and speech segment length (Kim, Chang, Holland & Pentland 2008). Thus, it was intended that groups who performed well at the game should have relatively equal participation. Conversely, if a group contains members that either dominate or do not engage with the game and still perform well, the game design has not fulfilled the purposes intended. In order to test the questions identified above, eight participants were recruited (3 male, 5 female) from a sample of convenience and each paid £10 upon the completion of the task. Participants were divided into two groups of four players each. The game board was initially set to values that were common among both groups. Each player was assigned one character role at random and all players were dealt six cards from the pool of personnel cards. The order in which personnel cards were dealt was controlled, so that both groups received the same cards. In addition, the order in which event ‘inject’ cards were arranged was constant across both groups. Thus, the better performance of one group over another group could only be attributable to a better use of the resources available. As the presence of a tutor would necessarily direct behaviour towards that which has been defined as appropriate, the role of tutor was

120

Figure 2. Participants playing the paper prototype

omitted from this evaluation. Instead, the breaks between game rounds were simply used to re-set the necessary game parameters before starting the following round. Data was captured through the video recording of participants while they played the game, using a standard digital camcorder and tripod. Video files were then analysed manually by an observer in order to extract the necessary data. Initially, the video for each group was viewed carefully and all game events, including changes in game state, deployment of teams, exchanging of personnel, and injects of events were noted, along with the time that they occurred. Subsequently, video files were split into ten intervals, each corresponding to one game turn. For the purposes of coding participant’s behaviour, each of these video clips was divided into 500ms segments. A researcher worked through these video clips carefully and noted whether each player was speaking during each 500ms segment. In this way, a log of player communication and participation activity was created for both groups. Results indicated that of the two groups evaluated, the group who performed better in terms of game success also exhibited more equal participation of group members and more total time spent talking than the lower achieving group. These findings suggest that the game itself delivers appropriate feedback to players on their collaborative

Teaching Group Decision Making Skills to Emergency Managers via Digital Games

behaviour. Specifically, the group who behaved in a more appropriate manner for a decision making group were rewarded with more positive feedback from the game state, which was the expressed intention of the game design. Thus, the results of Study 1 suggested that the DREAD-ED game provides appropriate feedback to players on their successful performance of the target skill of collaborative decision making.

Study 2: Further Examination of Game Mechanics with a Tabletop Prototype Study 2 examined whether the challenges that are present in real world decision-making environments are also present in the game-world decision making environment (see Linehan, Lawson, Doughty & Kirman, 2009b, for full description of experimental procedure and evaluation). In order to provide a useful training environment, actions that are ineffective or dangerous in reality should also be ineffective or dangerous respectively within the game. The study examined group effectiveness, the making of unnecessary and dangerous actions, and individual versus group performance. Eighteen participants (10 male, 8 female) were recruited from a sample of convenience and paid £10 upon completion of the game play session. As in Study 1, a basic paper prototype involving cards and a game board was used to present the game to participants. Sixteen of the participants were divided into four groups of four players each. Participants were video recorded while they played, and these recordings were later analysed in order to evaluate group effectiveness. As in study 1, efforts were made to ensure that all four groups faced exactly the same challenge and were supplied with exactly the same resources. The remaining two participants played the game on their own, fulfilling all four game roles. Results of Study 2 suggest that groups who played the game demonstrated similar problems to those faced by

real-world decision making groups. Many actions taken by players were inefficient and many may have been dangerous in a real-world decisionmaking environment. No group performed as well as the two participants who played the game individually. Moreover, neither groups nor individuals suffered as few casualties as would have occurred if they took no actions at all.

Study 3: Evaluating the Digital Game as a Teaching Tool Study 3 consisted of a pilot evaluation of the online game that was developed from the paper prototype presented above. This electronic version featured the same rules as the paper-based prototype, the major difference being that rather than using a co-located paper prototype, participants played the game online in distributed locations using a personal computer and web-based software (The DREAD-ED software and all support materials are available online at http://www.dread-ed.eu). Participants were represented as avatars in the game, sitting together at a table. In order to communicate with each other, the users were given the opportunity to use text chat. The team of personnel available for each player was represented by a row of six coloured man icons located at the bottom of the game interface in the right corner. In order to model the developing nature of an emergency event, events were introduced over the course of a game session through short media clips of telephone calls, radio news or television reports. Unlike in Studies 1 and 2, a tutor was included. This tutor was able to send messages to the team during the playing phase in order to comment on possible mistakes or misunderstandings. The tutors inclusion was necessary in this study, as study 3 was interested in the effectiveness of the teaching programme, rather than simply the balance of game mechanics, which was the case in studies 1 and 2. In study 3 we were interested to see the reactions of the users to the ‘blended’ teaching style.

121

Teaching Group Decision Making Skills to Emergency Managers via Digital Games

Figure 3. Screenshot of the electronic version of the game

Two separate samples were recruited for Study 3. The first sample consisted of five members (mean age: 45 years) drawn from the Academy for Crisis Management Emergency Planning and Civil Protection in Germany. This academy is part of the Federal Office of Civil Protection and Emergency Aid of the German government. Among the tasks of the academy are the training of crisis management units, research on new training simulations as well as the analysis of disasters and development of prevention measures. Each of the five participants has worked as a member of crisis units before becoming a member of the academy. Therefore, the sample was very experienced with simulations and disaster training but unexperienced with regard to gaming or virtual communication training. The participants of the second sample (mean age: 24 years) were recruited from graduate and undergraduate media and computer science courses. Due to their field of study the participants were very experienced with virtual realities and computer-mediated communication but had no experience in training of soft skills or emergency management. In order to ensure that every participant across the two groups was able to use the game, the tutor and a research assitant introduced

122

the main functions of the virtual platform before the playing phase. A training session (15 turns) lasted about two hours. Each participant had a personal computer and was asked to communicate with the others only by using the text chat function of the game. While the text chat function may not have replicated all of the communication channels typically available in a real-world emergency, this constraint itself was purposely intended to increase the challenge of clear and organised communication. After having played the turns, the participants met with the tutor in a classroom to discuss the process of the session face-to-face, as well as problems and the game’s general applicability for the future training of crisis units. The initial results of this pilot study, presented below, are based on both the tutors notes and voice recordings of participants responses during the feedback phase. As expected, the members of the academy outperformed the students. Although these participants did not have the same level of experience with computer-mediated communication as the sample of students, their style of communication was shorter and more efficient. Indeed, their decision-making was based on short but efficient discussions. Moreover, the members of the academy were more focussed on the parameters than

Teaching Group Decision Making Skills to Emergency Managers via Digital Games

the students, while the latter rather tried to collect personnel of the same colour without analysing the values of the four parameters. With regard to the communicative behaviour, the tutor noted that the students communicated more emotionally and demonstrated more stress than the people of the academy. While the students stated that they felt stressed during the game, the members of the academy reported that they “don’t care about time pressure and the development of the crisis because we are used to these problems in reality” (male, 52 years). On the contrary, a 25-year female student stated that she became “impatient when the other users didn’t respond to my comments. I wanted to scream ‘hurry up’. It was really annoying.” The academy members evaluated media injections more positively than the students, because “these injects are close to reality” (2 participants). These participants stated that the media injects are important in order to get an understanding of the disaster situation: “The media events are a very interesting feature of the game, because they give us a better understanding of the whole disaster. For me, it’s quite positive, that the game aspect became less important while the training aspect became more important,” (male, 45 years). The students, on the other hand, reported that they did not focus on the media injections, but rather on the personnel and the swapping of the teams. “The media injects constrained our discussion because we wanted to use the time between the turns to discuss next steps” (female, 24 years). Another aspect that was evaluated differently by the two samples was the role of each player within the game. The academy evaluated the missing of specific roles more positively than the students (Note: all participants were called users and could only be distinguished by their individual number, e.g. user1, user2). “It was advantageous to leave out the roles due to the fact that we were all equal. This leads to a more democratized communication, in my opinion. It was not clear who of the guys had been my supervisor and this was quite good” (male, 40 years). The students,

however, missed the nomination of a leader who assumed control over the discussion. “Our performance would have been better if we had had a leader in our team who made the final decisions” (female, 23 years).

GENERAL DISCUSSION Studies 1 and 2 sought to evaluate whether the game design could function as a valid componnent of a larger soft skills training programme, while Study 3 sought to evaluate whether this game design translated well to electronic media and would be accepted as a useful tool by emergency management professionals. Studies 1 and 2 demonstrated that a lot of the defining characteristics of group decision making behaviour, especially the mistakes, are evident in groups that play the DREAD-ED game. In addition, groups who behaved in a more appropriate manner for a decision making group were rewarded with more positive feedback. Thus, it appears that the game delivered appropriate feedback to players. The game design should prove to be a valid environment in which to train, practice and evaluate the decision making behaviours of groups and function as a valuable and engaging part of a group decision making skills training course. It appears that the groups who played the game would benefit from training in sound group decision making processes, as envisioned for the wider training scheme. This training could help identify faulty processes that teams employed using examples from game play. These groups could then practice implementing the appropriate processes in the safety of the game environment. Importantly, as the game environment appears to replicate the features of a real-world decision making environment, any process gains achieved over repeated exposures to the game should transfer to real-world tasks. Whether this transfer does occur is an empirical matter that we will address with further work.

123

Teaching Group Decision Making Skills to Emergency Managers via Digital Games

Study 3 sought to evaluate whether the game design translated well to electronic media and would be accepted as a useful tool by emergency management professionals. It appears that participants from the academy were enthusiastic about using the software to practice soft skills which they had already learned during their work as crisis units. As the provision of an inexpensive online tool that could facilitate regular soft skills training exercises was a specified goal of the DREAD-ED project, the finding that these emergency management personnel were keen to use the software was an exciting result. However, it must be noted that further work must be carried out to validate the effectiveness of this training methodology. Interestingly, the comparision of emergency managers with naïve university students in terms of group performance underlines how the successful use of the target skills (effective communication and group decision making) leads to success within the game, while poor use of these skills leads to failure within the game. Members of the academy used their experiences in disaster communication to solve the game’s tasks while the students, who were not experienced with communication in stressful situations, faced difficulities. The academy members demonstrated more efficient decision making by having short discussions and factual agreements while the communication of the students was impacted by emotions and stress. It must be noted that the students in Study 3 were confronted with challenges they have never faced before. To such naïve participants, the game offers a new possibility to train general social skills which are important in various situations of daily life. Although the narrative of the game, namely the disaster, is less important for students, the game design itself is helpful to train their general communicative behaviour based on a fictitous scenario. Moreover, for both naïve and experienced samples, the DREAD-ED game provides an inexpensive, yet engaging method of training group decision making processes. Thus, it appears that the game provides a rich and engaging environment in which

124

participants face exactly the same challenges that are faced in emergency management situations and where they can practice the skills of group decision making safely. Crucially, these skills should help emergency managers to make wellinformed, rational and efficient decisions during the course of managing emergency responses to life-threatening events such as floods, fires, volcanoes and chemical spills. While the studies presented here have demonstrated that the DREAD-ED game can function as a valid practical element of a programme for the training of group decision making and communication skills with emergency management personnel, the programme as a whole must be validated in a controlled trial in order to demonstrate robust learning outcomes. This work is ongoing and the results will impact upon the extent to which the game is used for the purpose intended.

FUTURE RESEARCH DIRECTIONS In addition to the progress of the DREADED programme itself, the development of the DREAD-ED game has highlighted that such games may benefit from closer integration with work on computer mediated communication. Indeed, computer mediated communication, when compared with face-to-face communication, has been found to lead to more equal participation of group members, greater information sharing, less normative influence, and ultimately better decision making than face to face communicating groups (Hinds & Bailey, 2003; Kim, Chang, Holland, and Pentland, 2008; Watson, DeSanctis & Poole, 1988). However, there are also a number of disadvantages to CMC, including slower and asynchronous communication, decreased information flow and greater group conflict (Hinds & Bailey, 2003; Kim, Chang, Holland, and Pentland, 2008; Watson, DeSanctis & Poole, 1988). Thus, it is essential to take these considerations on board when using a face-to-face paper prototype to evalu-

Teaching Group Decision Making Skills to Emergency Managers via Digital Games

ate the effectiveness of an educational game that will ultimately be presented in a digital format. The field of computer mediated communication has also presented an exciting direction for the development of game-based training for group decision-making and communications skills. Specifically, recently developed tools may allow the analysis player data in real-time in order to provide informative in-game feedback to players on some aspects of their group decision making processes. For example, research has suggested that equality of participation is a sound predictor of success in group decision making (i.e., DiMicco, Pandolfo, & Bender, 2004). The game itself could analyse the relative contributions of team members and present this data graphically to players as they play. Interestingly, Group Decision Support Systems perform this very function and have been demonstrated to have positive effects on participant’s decision-making behaviour (Watson, DeSanctis & Poole, 1988; DiMicco, Pandolfo & Bender; Buder & Bodemer, 2008; Kim, Chang, Holland & Pentland, 2008, Hinds & Bailey, 2003; Leshed, Hancock, Cosley McLeod & Gay, 2007). However, this technology has not yet been implemented in game-based training courses. If incorporated successfully within the game design, this technology has the potential to shape players’ behaviour and reduce the workload of the tutor.

CONCLUSION Games have recently been suggested as effective media for delivering educational content and for helping students to reach educational goals (Gee, 2003; Greitzer, Kuchar, & Huston, 2007; Kelly, Howell, Glinert, Holding, Swain, Burrowbridge & Roper, 2007; Pivec & Kearney, 2007). The current chapter demonstrates how an understanding of the appropriate and feasable learning outcomes of an educational programme, and a strong focus on incorporating these learning outcomes within the game design, can lead to the creation of a useful

educational game. A game was created, based on insights from a multi-disciplinary literature review, that appears, on initial evaluations, to represent a useful tool for training group decision and communication skills to emergency management personnel. It is intended that the approach taken here may be of use not only to those interested in emergency management, but may also serve as an exemplar on how to approach the design of games for very specific purposes in future.

ACKNOWLEDGMENT This work was carried out as part of the “Leonardo” project “DREAD-ED: Disaster Readiness through Education” funded by the EU Lifelong Learning Program (see http://www.dread-ed.eu/). Additionally, this work is based on the following earlier published paper; Linehan, C., Lawson, S., Doughty, M., & Kirman, B. (2009). Developing a serious game to evaluate and train group decision making skills. In Proceedings of the 12th International Conference on Entertainment and Media in the Ubiquitous Era, 106-113.

REFERENCES Albrecht, P. L. (1984). Summary of ten major school precision teaching programs. PhD dissertation, University of Kansas, Lawrence. Balducelli, C., Bologna, S., Di Costanzo, G., Gadomski, A. M., & Vicoli, G. (1995). Computer Aided Training For Cooperating Emergency Managers: Some Results Of Muster Project. In H. Drager (Ed.), Proceedings of the MEMbrain Conference on International Aspects of Emergency Management and Environmental Technology (pp. 433–444). Oslo: A/Q Quasar Consultants Press. Bogost, I. (2007). Persuasive Games. Cambridge, MA: MIT Press.

125

Teaching Group Decision Making Skills to Emergency Managers via Digital Games

Buder, J., & Bodemer, D. (2008). Supporting controversial CSCL discussions with augmented group awareness tools. Computer-Supported Collaborative Learning, 3, 123–139. doi:10.1007/ s11412-008-9037-5 Catania, A. C. (1998). Learning (4th ed.). Englewoods Cliffs, NJ: Prentice Hall. Crichton, M., & Flin, R. (2001). Training for emergency management: tactical decision games. Journal of Hazardous Materials, 88, 255–266. doi:10.1016/S0304-3894(01)00270-9

Hsu, E. (1989). Role-Event Gaming Simulation in Management Education. Simulation & Games, 20, 409–438. doi:10.1177/104687818902000402 Jain, S., & McLean, C. R. (2005). Integrated Simulation And Gaming Architecture For Incident Management Training. In M.E. Kuhl, N.M. Steiger, F.B. Armstrong, and J.A. Joines (Eds.), Proceedings of the 2005 Winter Simulation Conference, pp. 904-913. Janis, I. (1972). Victims of Groupthink. Boston, MA: Houghton Mifflin.

Davies, M. (2009). Examining Game Pace: How Single-Player Levels Tick. Gamasutra.com May 12, 2009. Retrieved May 29, 2009 from: http:// www.gamasutra.com/ view/feature/4024/examining _game_pace_how_.php

Karau, S. J., & Williams, K. D. (1993). Social loafing: a meta-analytic review and theoretical integration. Journal of Personality and Social Psychology, 65, 681–706. doi:10.1037/00223514.65.4.681

DiMicco, J. M., Pandolfo, A., & Bender, W. (2004). Influencing Group Participation with a Shared Display. In Proceedings of the 2004 ACM conference on Computer Supported Cooperative Work, 614 – 623.

Kelly, H., Howell, K., Glinert, E., Holding, L., Swain, C., Burrowbridge, A., & Roper, M. (2007). How to build serious games. Communications of the ACM, 50, 45–49. doi:10.1145/1272516.1272538

Ferster, C. B., Skinner, B. F., Cheney, C. D., Morse, W. H., & Dews, P. B. (1957). Schedules of Reinforcement. New York, NY: AppletonCentury-Crofts. doi:10.1037/10627-000 Gee, J. P. (2003). What Video Games Have to Teach Us About Learning and Literacy. New York, NY: Palgrave Macmillan. Greitzer, F.L., Kuchar, O.A., & Huston, K. (2007). Cognitive science implications for enhancing training effectiveness in a serious gaming context. ACM Journal of Educational Resources in Computing, 7, Article 2. Habgood, M. P. J. (2007). The effective integration of digital games and learning content. PhD Thesis, University of Nottingham, UK. Hinds, P. J., & Bailey, D. E. (2003). Out of sight, out of sync: Understanding conflict in distributed teams. Organization Science, 14, 615–632. doi:10.1287/orsc.14.6.615.24872 126

Kerr, N. L., & Tindale, R. S. (2004). Group Performance and Decision-Making. Annual Review of Psychology, 55, 623–655. doi:10.1146/annurev. psych.55.090902.142009 Kim, T. J., Chang, A., Holland, L., & Pentland, A. (2008). Meeting Mediator: Enhancing Group Collaboration and Leadership with Sociometric Feedback. In Proceedings of the ACM Conference on Computer Supported Cooperative Work (pp. 457-466). San Diego, CA, 2008. Kincaid, J. P., Donovan, J., & Pettitt, B. (2003). Simulation techniques for training emergency response. International Journal of Emergency Management, 1, 238–246. doi:10.1504/ IJEM.2003.003300 Koster, R. (2005). A theory of fun for game design. Scottsdale, AZ: Paraglyph Press.

Teaching Group Decision Making Skills to Emergency Managers via Digital Games

Kowalski-Trakofler, K. M., & Scharf, T. (2003). Judgment and decision making under stress: an overview for emergency managers. International Journal of Emergency Management, 1, 278–289. doi:10.1504/IJEM.2003.003297 Leshed, G., Hancock, J. T., Cosley, D., McLeod, P. L., & Gay, G. (2007). Feedback for Guiding Reflection on Teamwork Practices. In Proceedings of the 2007 International ACM Conference on Supporting Group Work, 217-220. Lindsley, O. R. (1971). From Skinner to precision teaching: The child knows best. In Jordan, J. B., & Robbins, L. S. (Eds.), Let’s try doing something else kind of thing (pp. 1–11). Arlington, VA: Council for Exceptional Children. Lindsley, O. R. (1991a). B. F. Skinner (19041990): Thank you, grandpa Fred! Journal of Precision Teaching, 8, 5–11. Lindsley, O. R. (1991b). Precision teaching’s unique legacy from B. F. Skinner. Journal of Behavioral Education, 1, 253–266. doi:10.1007/ BF00957007 Lindsley, O. R. (1992). Precision teaching: discoveries and effects. Journal of Applied Behavior Analysis, 25, 51–57. doi:10.1901/jaba.1992.25-51 Linehan, C., Lawson, S., Doughty, M., & Kirman, B. (2009a). There’s no ‘I’ in ‘Emergency Management Team:’ Designing and evaluating a serious game for training emergency managers in group decision making skills. In Proceedings of the 39th Conference of the Society for the Advancement of Games & Simulations in Education and Training, 20-27. Linehan, C., Lawson, S., Doughty, M., & Kirman, B. (2009b). Developing a serious game to evaluate and train group decision making skills. In Proceedings of the 12th International Conference on Entertainment and Media in the Ubiquitous Era, 106-113.

Loftus, G. R., & Loftus, E. F. (1983). Mind at Play: The Psychology of Video Games. New York, NY: Basic Books. Malone, T., & Lepper, M. (1987). Making learning fun: taxonomy of intrinsic motivation for learning. In Snow, R., & Farr, M. (Eds.), Aptitude, Learning, and Instruction (pp. 223–253). Hillsdale, NJ: Erlbaum. Pivec, M., & Kearney, P. (2007). Games for learning and learning from games. Informatica, 31, 419–423. Sanders, R. L., & Rhodes, G. S. (2007). A Simulation Learning Approach to Training First Responders for Radiological Emergencies. In Proceedings of the 2007 summer computer simulation conference, Article 28. Siang, A. C., & Rao, R. K. (2003). Theories of Learning: A Computer Game Perspective. In Proceedings of the IEEE Fifth International Symposium on Multimedia Software Engineering, 239-245. Skinner, B. F. (1953). Science and Human Behavior. New York, NY: The Free Press. Skinner, B. F. (1959). Cumulative Record. New York, NY: Appleton-Century-Crofts. Stasser, G., & Titus, W. (1985). Pooling of unshared information in group decision making: biased information sampling during discussion. Journal of Personality and Social Psychology, 48, 1467–1478. doi:10.1037/0022-3514.48.6.1467 Steiner, I. D. (1972). Group Process and Productivity. New York, NY: Academic. Vorderer, P., Hartmann, T., & Klimmt, C. (2003). Explaining the enjoyment of playing video games: the role of competition. In Proceedings of the second international conference on Entertainment computing, 1-9.

127

Teaching Group Decision Making Skills to Emergency Managers via Digital Games

Watson, R. T., DeSanctis, G., & Poole, M. S. (1988). Using a GDSS to Facilitate Group Consensus: Some Intended and Unintended Consequences. Management Information Systems Quarterly, 12, 463–478. doi:10.2307/249214

Callaway, M. R., & Esser, J. K. (1984). Groupthink: Effects of cohesiveness and problemsolving procedures on group decision making. Social Behavior and Personality, 12, 157–164. doi:10.2224/sbp.1984.12.2.157

Wesolowski, J. (2009). Beyond Pacing: Games Aren’t Hollywood,” Gamasutra.com May 21, 2009. Retrieved May 29, 2009 from: http://www. gamasutra.com/ view/feature/4032/beyond _pacing_games_arent_.php

Karau, S. J., & Kelly, J. R. (1992). The effects of time scarcity and time abundance on group performance quality and interaction process. Journal of Experimental Social Psychology, 28, 542–571. doi:10.1016/0022-1031(92)90045-L

Wolfe, J., & Rogé, J. N. (1997). Computerized General Management Games as Strategic Management Learning Environments. Simulation & Gaming, 28, 423–441. doi:10.1177/1046878197284006

Kelly, J. R. (1998). Entrainment in individual and group behavior. In McGrath, J. E. (Ed.), The Social Psychology of Time: New Perspectives (pp. 89–110). Newbury Park, CA: Sage.

Woods, S. (2004). Loading the dice: The challenge of serious videogames. The International Journal of Computer Game Research, 4, Retrieved April 16, 2009, from http://www.gamestudies. org/ 0401/woods/.

Kerstholt, J. H. (1994). The effect of time pressure on decision-making behaviour in a dynamic task environment. Acta Psychologica, 86, 89–104. doi:10.1016/0001-6918(94)90013-2

ADDITIONAL READING Alexander, D. E. (2003). Towards the development of standards in emergency management training and education. Disaster Prevention and Management, 12, 113–123. doi:10.1108/09653560310474223 BinSubaihA. Maddock S., & Romano, D.M.(2005). Comparing the use of a ‘tabletop’ experiment and a collaborative desktop virtual environment for training police officers to deal with traffic accidents. In Proceedings of International Conference on Engineering in Education ICEE2005. Brodbeck, F. C., Kerschreiter, R., Mojzisch, A., Frey, D., & Schulz-Hardt, S. (2002). The dissemination of critical, unshared information in decision making groups: the effects of pre-discussion dissent. European Journal of Social Psychology, 32, 35–56. doi:10.1002/ejsp.74

128

Linehan, C., Lawson, S., & Doughty, M. (2009). Tabletop Prototyping of Serious Games for ‘Soft’ Skills Training. In Proceedings of 1st International Conference in Games and Virtual Worlds for Serious Applications, 182-185. Littlepage, G., Robison, W., & Reddington, K. (1997). Effects of Task Experience and Group Experience on Group Performance, Member Ability, and Recognition of Expertise. Organizational Behavior and Human Decision Processes, 69, 133–147. doi:10.1006/obhd.1997.2677 Lonka, H., & Wybo, J. L. (2005). Sharing of experiences: a method to improve usefulness of emergency exercises. International Journal of Emergency Management, 2, 189–202. doi:10.1504/IJEM.2005.007359 Lovaas, O. I. (1987). Behavioral treatment and normal educational and intellectual functioning in young autistic children. Journal of Consulting and Clinical Psychology, 55, 3–9. doi:10.1037/0022006X.55.1.3

Teaching Group Decision Making Skills to Emergency Managers via Digital Games

McEachin, J. J., Smith, T., & Lovaas, O. I. (1993). Long-Term Outcome for Children with Autism Who Received Early Intensive Behavioral Treatment. American Journal of Mental Retardation, 97, 359–372.

Turoff, M., Chumer, M., Hiltz, S. R., Hendela, A., Konopka, J., & Yao, X. (2006). Gaming Emergency Preparedness. In Proceedings of the 39th Annual Hawaii International Conference on System Sciences, 2, 38-47.

Miles, J. A., & Greenberg, J. (1993). Using punishment threats to attenuate social loafing effects among swimmers. Organizational Behavior and Human Decision Processes, 56, 246–265. doi:10.1006/obhd.1993.1054

Walia, A. (2008). Community based disaster preparedness: Need for a standardized training module. The Australian Journal of Emergency Management, 23, 68–73.

Moreland, R. L., Argote, L., & Krishnan, R. (1996). Social shared cognition at work: Transactive memory and group performance. In Nye, J. L., & Brower, A. M. (Eds.), What’s social about social cognition? Research on socially shared cognition in small groups (pp. 57–84). Thousand Oaks, CA: Sage Publications. Newson, C., & Rincover, A. (1989). Autism. In Mash, E. J., & Barkley, R. A. (Eds.), Treatment of Childhood Disorders (pp. 286–346). New York: The Guilford Press. Ordonez, L., & Benson, L. III. (1997). Decisions under Time Pressure: How Time Constraint Affects Risky Decision Making. Organizational Behavior and Human Decision Processes, 71, 121–140. doi:10.1006/obhd.1997.2717 Ritterfeld, U., Cody, M., & Vorderer, P. (2009). Serious Games: Mechanisms and Effects. Routledge Chapman & Hall. Stewart, D. D., & Stasser, G. (1995). Expert role assignment and information sampling during collective recall and decision making. Journal of Personality and Social Psychology, 69, 619–628. doi:10.1037/0022-3514.69.4.619 Takada, A. (2004). The role of team efficacy in crisis management. International Journal of Emergency Management, 2, 35–46. doi:10.1504/ IJEM.2004.005229

KEY TERMS AND DEFINITIONS Blended Learning: The mixing of different learning environments. Tuypically a combination of traditional classroom-based learning and Distance Learning. Computer-Mediated Communication: Any communication that occurs through the use of two or more networked computers. Emergency Management: The management of resources and responsibilities with regard to all aspects of emergency, in particular preparedness and response. Game Mechanics: The system of rules within a game that presents challenges and constrains behaviour of the game player. Group Decision-Making: The process through which a group or team collectively make a decision on a course of action. Intrinsic Learning: The incorporating of learning outcomes of an educational programme within the game mechanics of an educational game. Learning Outcomes: The stated, specific skills or knowledge that a learner should attain through an education programme. Paper Prototyping: A method popular in Games Design, whereby game mechanics can be evaluated at an early stage of development.

129

130

Chapter 7

Exploring Semantic Tagging with Tilkut Sari Vainikainen VTT Technical Research Centre of Finland, Finland Pirjo Näkki VTT Technical Research Centre of Finland, Finland Asta Bäck VTT Technical Research Centre of Finland, Finland

ABSTRACT Social bookmarking is one of the earliest examples of social media services. In bookmarking services there are two main approaches for adding metadata: user-generated freely chosen tags and keywords based on taxonomies or semantic ontology. This chapter presents a social bookmarking service Tilkut that combines the benefits of both of these approaches. Tilkut utilizes both freely defined tags and semantic tag suggestions based on predefined ontology. This chapter describes two different versions of the service and user experiences from a small scale user study and long-term test use in real context. Work related knowledge sharing was selected as a primary use case for the second version. The results from the first user studies were used as the starting point when developing the second version of Tilkut. A survey and workshop were organised to get more information of the requirements for enterprise use. In this chapter, we explain our approaches to adding semantics to social bookmarking, present the experiences, and discuss future research directions.

INTRODUCTION There are several online bookmarking and clipping applications available that support storage, sharing and retrieval of web links. Delicious.com, Connotea and Clipmarks are examples of such DOI: 10.4018/978-1-60960-774-6.ch007

social bookmarking applications. In these services bookmarks and image clips from web pages are typically organized using tags – user-defined keywords describing the content of the bookmark. Tagging is an easy way to add metadata to web content compared to predefined and formal taxonomies. The main advantage of tags, the free and bottom-up emergence of vocabularies, can

Copyright © 2012, IGI Global. Copying or distributing in print or electronic forms without written permission of IGI Global is prohibited.

Exploring Semantic Tagging with Tilkut

also become a disadvantage. An active tagger is quickly faced with a large and messy tagcloud, which does not support finding content so well any more. Our goal was to offer extra support for users to manage their tags and to find relevant content. Tagging can be enhanced semantically by utilising ontology. The purpose of increased semantics is to support managing and utilising tags and tagged content without losing the ease and power of user-defined keywords. We explored the usefulness of semantic tagging by developing a social bookmarking prototype and testing it with potential users. In this paper we present the results from the user tests and discuss the possibilities for combining user-generated tags and well-structured ontology. Semantic bookmarking could be especially beneficial in work-related knowledge sharing, and therefore have explored the opportunities for adapting the Tilkut1 application for that purpose.

BACKGROUND Challenges with Tagging A lot of the success of tags and tagging can be attributed to the freedom of being able to use any word as a tag. Tagging is typically flat: all tags are at the same level and any number (or a high number) of tags can be applied to a resource. This has some drawbacks for utilising them even for the users themselves and more so for applications that aim at utilising this information automatically. Well-known and frequent challenges with tags are that people use different words to describe the same thing, or a word has several different meanings (polysemy). People may also describe things at various levels of detail – an expert in a subject will use more detailed and specific words, whereas others use more general words. Also different forms of the same word (singulars, plurals, typos) exist. (Golder, 2006)

In addition to differences in vocabularies there are also differences between people in how they tag and why they tag. Also, applications have different restrictions and support to tags, which naturally affects the user behaviour. There are several research papers (Golder, 2006; Maala, 2007; Marlow, 2006; Xu 2006) that report studies of the type of tags people use. In these papers the work has been based mostly on Delicious2 or Flickr3 tags. In Delicious, the following tag categories have been identified: topic, type of referenced resource, proper name (person, company, product, event, and location), subjective tags (adjectives, ratings), self reference, toDo tags and time (Golder, 2006). In Flickr photo tags categories include place, time, event, name, action and camera (Maala, 2007). The results of these studies were used when defining the tag categories for our prototype. When the aim is to utilise tags, different types of tags give different opportunities. Topics (like travel, semanticweb, cat, cars) can be used for analyzing users’ interests as well as characteristics of the tagged resource. Proper names can be used as an indication of interests as well, particularly when additional information related to them can be found on the web. In our approach, we developed methods for automatic analysis of tag categories and methods for adding semantics to different type of tags. The aim is to use this additional metadata for finding and combining similar resources.

Adding Semantics to Tags Semantics can be utilised at two different levels. The first level is the tag level. Semantic knowledge bases like Geonames4, KOKO5, Freebase6 and DBpedia7 that offer application programming interfaces (APIs) and widgets to developers have become available, and they can be used to add semantics to tags. There are two alternatives to adding semantics to tags. The first alternative is to help users to use tags that have a semantic meaning is a semantic knowledge base. With the

131

Exploring Semantic Tagging with Tilkut

help of new web interface technologies, this can be implemented in a user-friendly way. The other alternative is to try to infer the meaning of the tag afterwards. We have used all these options, and our semantic tagging widget and semantic tag analyser utilise existing knowledge bases. Also, the management of tag and tagging data can be based on ontology. Commonly accepted tag ontology facilitate interoperability and reuse of tagging data across systems and enable automated processing. Several ontology have been defined relating to tagging with different foci. An extensive summary of the state of the art, comparison and alignment of tag ontology can be found in articles (Kim & Passant, 2008; Kim & Scerri, 2008). One of the first ontology for tagging is Richard Newman’s Tag ontology, which defines the key concepts like tags, resources and their relations (Newman, 2005). These basic concepts are utilized also in other tag ontology. SKOS8 (Simple Knowledge Organization System) is quite widely used for describing thesauruses and taxonomies semantically. In the tag ontology, the Tag class is defined as a subclass of SKOS’s Concept, so the properties of SKOS like broader, narrower and related can be utilized also for describing the tags and their relations. From the information retrieval point of view, one problem with tags is that a tag may have different meanings depending on the context. The MOAT9 (Meaning Of A Tag) ontology aims at managing this by providing a way to describe different meanings for the same tag. The meanings of the tags are described with help of semantic knowledge bases like DBpedia, Geonames or KOKO. In addition to tags, also the resources that have been tagged, people, who have tagged, and networks and sites, where the activity has taken place need to be described semantically. There are vocabularies and ontology that can be used as building blocks for that information. For example, combining the tag ontology with SIOC10 and FOAF11 offers great opportunities to describe

132

and combine user activities in different sites in a semantic way. Good examples of co-utilising SIOC, FOAF and SKOS as well as MOAT, can be found in the article (Bojãrs, 2008). Our ontology is built by utilising these existing ontology. There are some social bookmarking applications that have semantic features. Notitio.us12 is a bookmarking application where bookmarks are organised into user defined directories instead of adding tags to each entry. There directories can be annotated semantically with WordNet13 or DMoz14, and this information is utilised for suggesting users additional directories that the user might be interested in. Gnizr15 social bookmarking software supports semantics by modelling tag relationships semantically based on the SKOS ontology. Users are able to define tag relations like related, narrower or broader. It also supports exporting data in semantic format by utilizing SIOC and Tag Ontology. ZigTag16 lets the user select from among tags with definitions. Faviki17 lets its users use Wikipedia terms as tags. Terms are available in several languages. What our application adds to the already published works is that it supports many different knowledge bases for adding and analysing tags. This gives better support for different languages and better coverage of tag suggestions relating to different topics. Our approach does not force users to use suggested tags, they can create their own tags the meaning of which is then analysed automatically. In the prototype we support semantics many different ways such as automatic tag suggestions, a semantic tagging widget to help users to add tags and automatic analysis of tags.

Social Media in Enterprise Knowledge sharing has been viewed as a crucial factor for creating and maintaining competitive advantage for companies (Davenport & Prusak, 1998). IT tools for gathering and sharing knowledge are important in making knowledge sharing possible, and most recently, social media or Web

Exploring Semantic Tagging with Tilkut

2.0 tools have gained interest also in enterprise use. The opportunities for using various social media applications, including social bookmarking, in enterprises have been explored to some extent. These easy to use tools seem to hold a promise also for enterprise use, but there are some special issues to consider as well. One of these is that on the web, the number of potential users is practically limitless, whereas in an enterprise it isn’t. Another important difference is that in an enterprise, people often know each others and there are positions and roles, which mean that typical social media activities like commenting and rating may not be as straight forward to do as in public social media applications on the web. Reasons for using social media tools at work include things like efficiency in easily and quickly reaching people, managing one’s own knowledge, initiate discussion, and to keep up with people, news and events. Reasons for not using social media are such as too few users, no managerial encouragement for using, information overload and begin afraid of sharing confidential information. (Paroutis & Saleh, 2009). The most important factors preventing people from adopting and using Web 2.0 tools are history – people are hesitant to give up tools that they have learnt to use, perceived benefits and rewards are estimated to be low, lack of perceived organisational and managerial support, and not enough trust, either on the correctness of the information that is available or trust in other users in general. People may also be hesitant to use this kind of tool because they are afraid of revealing confidential information (Paroutis & Saleh, 2009). We can see that getting people to use social media tools like social bookmarking at their work depends on many factors, some of which relate to the application itself, but many relate to other issues like the estimation of the balance between effort and benefit, and managerial and organisational issues. Regarding semantics and semantic tagging, their main area of potential impact is in reducing

effort and bringing more reward by making it easier and more effective to find relevant information and people.

THE FIRST VERSION OF THE TILKUT APPLICATION To study the tagging conventions and acquire user needs relating to bookmarking, we made a preliminary user study with current Delicious users. Based on these interviews and user tests, we developed a social bookmarking service called Tilkut that supports semantic tagging and tag management. A screenshot of the Tilkut service can be seen in the Figure 1. Tilkut software consisted of a web application and a Greasemonkey18 script for Firefox. The script was used for selecting the web content that will be stored into Tilkut. The title, selected text, URL, pictures and possibly available date and place information of the original page are automatically filled into a metadata form. The user may modify the information as well as add remarks and tags. The goal of the Tilkut service was to offer extra support for users to manage their tags. Instead of flat tags, it had a three level approach consisting of folders, tag categories and tags. Users could define their own folders for different tagging purposes (e.g. gathering ideas, creating lecture) and select which tag categories to use in each folder out of the eight predefined categories. The tag categories were defined based on literature and our Delicious user studies. The categories were topic, type, place, product, company, project, importance, and miscellaneous. The purpose of the tag categories was to help users to organize their tags, and support information retrieval. Tilkut supported suggestion-based semantic tagging by utilising existing semantic datasets. When a user started to type a tag, suggestions from the ontology vocabulary starting with the same letters were shown below the tag field. The

133

Exploring Semantic Tagging with Tilkut

Figure 1. Tilkut user interface with clips and tag browsing

user could decide whether to use the suggestion or write an own tag. In the user tests, YSO and place ontology of Finnish Ontology Library Service ONKI19 was used for topic and place suggestions.

USER STUDY Users The Tilkut service was tested for three weeks by six participants (four women, two men). Initially eight users were selected, but two of them dropped out because of lack of time. The ages of the users varied from 25 to 60 years. All users were experienced web users, but their background in using bookmarking services varied from no or little experience to heavy using with several links per day. All test users were interested in sharing

134

knowledge on the web and categorising information in a new way. Some users liked the simplicity of Delicious whereas for others it was too minimalistic or too hard for everyday use. According to the users, the quantity of bookmarks easily becomes a problem. The current services do not provide well-designed and easy ways for organising bookmarks. Also, defining good tags can be hard even for people who do it professionally. One of the challenges is choosing the proper level of tagging: a tag must be specific enough to be helpful for oneself but also general enough to support findability and sharing with others.

Methods The user test consisted of an initial interview and briefing (face-to-face or Skype video conference),

Exploring Semantic Tagging with Tilkut

Table 1. The total number of tags in different categories Topic

Type

Companies

Places

226

52

49

45

Products

Misc

Importance

Project

TOTAL

13

12

8

7

412

an independent test period of three weeks, and an end interview. Four users were taught the basics of how to use Tilkut at the end of the initial interviews, and three users were asked to test the service by themselves. This served as a light-weight usability study to test the intuitiveness of the user interface. A blog was used to communicate use experiences and problems to researchers and among the test users during the three weeks’ test period. In the end interviews, the users were asked about their experiences with the Tilkut service, ways of using it, problems encountered during the test, and ideas for further development. During the interview, the users demonstrated how they had used the Tilkut service and what kind of clips and tags they had saved, as well as which features did not work as they had wished. The usage statistics was retrieved from the Tilkut database after the test period for more specific content analysis.

Results During the test period, 118 clips were added to the service by six test users, on average 20 clips per user. Tilkut was used both for work and hobby related themes, and both as a general note book and for some specific purposes. Users created folders like Books, Holiday trip, Blues music, and ToDo, but 45 percent of clips were added to the default folder called “Bookmarks”. Altogether 412 different tags were added into the eight predefined tag categories. On average 4.3 tags per clip were used (varying from 0 to 11 tags per clip). Most of the tags were added into the Topic category (55%), after which the most commonly used categories were Type (13%),

Companies (12%) and Places (11%). Products, Project, Importance and Misc were not very much used. The number of tags in different categories can be seen in the Table 1. The tags were mostly added to corresponding categories, but some creative use of categories was also found. Because there was no category for people tags, Misc, Topic or Company were used for that instead. Only 14% of the tags were the terms suggested from the YSO ontology. In the tested version, the suggestions were based on the English language, whereas many users wrote their tags in Finnish.

Use Experiences The use experiences of the Tilkut service were mainly positive. Most users found that the basic concept of clipping and tagging suited their needs and clear use cases could be found already during the short test period. The background of users affected strongly on their way of using Tilkut. Some users used the service as a quick bookmarking tool and did not bother to add many or any tags at all. On the contrary, some users wanted to have their clips and tags well organised and especially appreciated the structured tagging available in Tilkut. However, the grouping of tags into categories was not regarded very useful. Users found that there were too many categories and it was difficult to decide into which category each tag should be put to. The effort relating to tagging grows, when the user has to make many decisions and remember his or her earlier decisions about how to use the categories. During the three weeks’ test period,

135

Exploring Semantic Tagging with Tilkut

the usefulness of categories in searching the clips could not yet be evaluated, since the number of clips in each folder was small. Some users appreciated that adding a clip required some effort: tagging and categorising served as a check point to make sure that the clip was worth saving and to think about its meaning when creating and selecting tags. Browsing tags was not very easy in the tested version of the prototype. It was suggested that connections between tags and suggestions for related terms and related content could be shown for the user. Semantic tagging could also be supported when using Tilkut with a group of colleagues or friends: users suggested that the group could define their own vocabulary or ontology for their tags in advance to keep terms in order and consistent.

Conclusion of the First User Study Two types of taggers were identified among the test users based on whether they preferred the ease of use or preciseness of tagging. The first group, here called “pilers”, preferred easy and quick storing of data, loved folders and hated ontology. They did not want to do any extra work or spend time with categorising information. For them, the folders partly replaced the functions of tags, although some tags were added as well. The second group, “filers”, preferred structure and organisation and valued the predefined vocabularies and hierarchies. For them it was also important to be able to create rules and ontology for tagging within a team. In order to suite the different user needs and preferences, the opportunities to customise the level of detail at tagging are important. In Tilkut, we do not force users to use any tags, and the user could also select, which tag categories to use in each folder. There were many tag categories, which made it in some cases difficult for users to decide the best category for each tag. The use of different categories would offer more direct benefits, if category specific ways were used for tagging. For

136

example places could be selected from a map, or the importance could be marked with stars. Users appreciated tag suggestions and semiautomatic tagging but had very different preferences as to how the suggestions should work. Tag suggestions from the YSO ontology clearly divided users’ opinions: some users did not even want to see the suggestions, whereas some would have been ready to use only the predefined ontology terms. One reason for not appreciating suggestions was that the implementation only supported YSO suggestions in English, and not in the test users’ mother tongue Finnish. Anyway, as a requirement for the future development, we see that users should be able to select themselves which ontology to use for tag suggestions, and also to be able to switch the suggestions on and off easily. Also, if the original web page contains tags, these tags should be automatically presented as tag suggestions in Tilkut.

REQUIREMENTS FOR ENTERPRISE USE The benefits of the added semantics appear to have the most potential, when a social bookmarking tool is used in connection to work related and other such purposes where knowledge accumulation and sharing are important. In order to find out about the requirements of the working environment and to assess whether Tilkut could be adapted to serve the needs of enterprise use, a workshop and a survey were organised. The survey was carried out in our organisation in Q2/2009. It was open for anyone to participate but it was especially directed to four knowledge centres. In total, 59 people participated in the survey; 45% women and 55% men. Different age groups and working times were well represented. People were asked whether they were happy with their current opportunities to knowledge sharing within our organisation and with external partners. 48% of the respondents were happy with their

Exploring Semantic Tagging with Tilkut

current practices for knowledge sharing within our organisation and 35% with external partners, and only 5% to 7% were very discontented with current practices. The attitudes towards using a web application for knowledge sharing were mixed. Many of the respondents valued the opportunity to discuss in person, and to develop ideas further through interaction, and therefore felt that ICT applications have only limited applicability. Email was the only popular application used for sharing links. It could also be seen in the results that people are already using many applications, and they are not very interested in adopting yet another one. For the developers this means that a successful application must be very easy to use and that it must be possible to see the benefits of the application immediately. Nine people participated in a workshop where different aspects of knowledge sharing where discussed and the opportunities of and requirements for using Tilkut in link and knowledge sharing were discussed. Following important points came up in the discussion. In knowledge sharing, one hindrance is to know who should be informed about an item that one finds important. This is the key issue when sharing knowledge personally or by email. A social bookmarking application removes this problem to some extent. The idea in a social bookmarking application is to store the item and let those who find the topic interesting to find it. Here the tags and semantic tags in particular are of great importance, so that people can easily check or be alerted if something relevant comes up. People also reported to be increasingly hesitant about sending email, in order not to disturb others without a good reason. Also receiving such an email requires that the recipient decides whether and how to react to such an email. Social bookmarking does not require immediate communication between participants, so link and knowledge sharing require a bit less effort than using email. Even though people are careful not to disturb their colleagues, it is also required that one can

see if and when other people have seen a link one considers important for others to see. Also otherwise, seeing that other people notice and read one’s contributions is an important motivator. It was also pointed out that sharing a link is a way to interact, and it has a social value coming from wanting to share something that one finds important or fun. Another requirement is that there should be one, widely used way of sharing links so that people would not need to check different sites or applications. Another area of requirements relates to the different lifespan of links. Some links are of interest only for a short time whereas others are of more permanent interest. An ideal system for sharing links would be able to separate these groups and give the short term links less emphasis.

TILKUT APPLICATION V2 Based on the first user tests, users could be divided into two different groups: “pilers” and “filers”. We decided to keep and develop features to support both user groups. The “pilers” liked the folders that partly replaced the functions of tags. We decided to continue to support this functionality but wanted to add more flexibility. The folders of the first version were visible only to the user herself, and it was possible to store a clip only in one folder. The folders were replaced with a Group functionality: users may create public or private groups and invite other users to join (See Figure 2). A clip can be shared to one or several groups, and also directly to one or several specified persons. This group functionality gives more flexibility than folders and meets the user need for group level sharing. The “filers” wished for features that make it possible to build a shared vocabulary within a group. In this second Tilkut version, the user is able to see which tags have been used in a spe-

137

Exploring Semantic Tagging with Tilkut

Figure 2. Examples of the groups in Tilkut. A user is able to create and join groups as well as share content to groups.

cific group and it is easy to reuse them. This helps in harmonising the use of tags within a group. The second Tilkut version was enhanced with some other social features, such as better support for sharing content, support for discussion and a profile page to learn about other users’ interests. It is also possible to create a clip without a web link. This can be used to make and share notes, ideas and observations. A mobile software application called TagIt is integrated into Tilkut so that also mobile messaging is possible. TagIt supports text and picture messages and uses ontology to add semantics to messages. TagIt makes it possible to combine clips of real world (e.g., photos of events) with web clips (Nummiaho, 2009; Vainikainen, 2009). In our internal Tilkut installation, there is also a “Buzz monitor” page that lets users see the new and hot topics, and find out about new users and statistics. In the first user tests, tag suggestions and semi-automatic tagging were regarded as useful, but grouping tags into categories received mixed opinions. In the second Tilkut version we now support automatic tag suggestions based on the

138

clipped content. There is also a new alternative way of adding tags by using a Semantic tagging widget that uses only one tag field (See Figure 3). When the user selects a tag from among the semantic tag suggestions presented to him or her, we use the semantic knowledge and make the tag categorisation automatically. We developed a semantic tagging widget for this purpose and it supports giving tag suggestions from several knowledge bases. By utilising several knowledge bases the coverage of tag suggestions relating to different topics is better since different knowledge bases cover different areas of knowledge, and there is no single comprehensive knowledge base. Since users are not forced to use tag suggestions and they still can use their own tags, we developed semantic tag analysis methods which are used for inferring the meanings of freely given tags automatically. The user interface also lets users add and change the meanings of their tags (See Figure 4). Alternative meanings of a tag are shown to the users as descriptions provided by the knowledge bases. The aim of these new semantic features is to help and support users for

Exploring Semantic Tagging with Tilkut

Figure 3. The semantic tagging widget supports giving tag suggestions utilising several knowledge bases. Here the suggestions come from Freebase, KOKO and Geonames. The icon after the name indicates the knowledge base.

creating and finding clips. Clips can be searched by browsing tags and tag categories as before, or by using the new search feature. The following chapter explains the semantic features more in detail.

SEMANTICS IN TILKUT SERVICE Semantics in Tilkut is supported in multiple ways: • •

Automatic tag suggestions are made by using the external Zemanta service. Users are supported in adding semantic tags with help of semantic tagging widgets.

•

• •

Meanings of free tags are automatically analysed by using existing semantic knowledge bases. Users can add tags to different tag categories such as topic, place and company. Content is stored semantically using OSMO ontology.

Automatic Tag Suggestions At the moment we utilise Zemanta20 API for analysing the text of bookmarked content and for producing automatic semantic tag suggestions (See Figure 5). We utilise Google’s Translate service for non English texts before running the

Figure 4. The user is able to edit the meanings of tags in the Tilkut social bookmarking software

139

Exploring Semantic Tagging with Tilkut

Figure 5. An example of automatic tag suggestions. The user is able to add the suggested tag into his or her tags by clicking the + sign.

text through Zemanta. We also give the tags of the original webpage as tag suggestions if the original webpage uses the tag microformat21 for presenting them.

Semantic Tagging Widget In the semantic tagging widget, semantic tag suggestions are presented to the user utilising various knowledge bases, after the user has typed a couple of characters for a tag. When the user selects the suggested tag, its meaning is defined as an URI to the database. Based on this, more knowledge can be accumulated. For example, geo-coordinates can be added to location which makes it possible to view clips on a map view. The widget supports tag suggestions from different knowledge bases such as KOKO, Freebase, DBpedia and Geonames. The widget can be configured to use one or several databases. When the user is using the view with only one tagging field, semantic tag suggestions come from several knowledge bases at the same time. When a clip is created by using different tag categories, the Finnish KOKO ontology is used for tag suggestions in the topic tag category. The KOKO ontology includes YSO (General Finnish ontology) as an upper ontology but also other domain ontology like museum ontology, applied art ontology and photography ontology. The semantic tagging widget

140

supports restricting suggestions to certain topics; Geonames location ontology is used for making tag suggestions in the place category and Freebase in the person, product and company categories. When tailoring Tilkut to different use cases, tag categories and semantic knowledge bases can be adjusted based on different requirements.

Semantic Tag Analyser Automatic semantic analysis of tags uses publicly available knowledge bases such as WordNet, KOKO, DBpedia and Freebase to analyse freely given tags and other keywords and turn them into semantic tags. The tag analyser is used in Tilkut to create meaning for the user-generated tags so that the clips can be searched more intelligently.As the first step, the analysis tries to recognise the language of the tag. We support lexical analysis and misspelling corrections for Finnish and English tags. The analysis proceeds differently depending on the language. The analysis for English tags starts with WordNet, then followed by the use of OpenCyc and DBPedia and completed by trying to find meanings from KOKO. For Finnish tags, the analysis starts with KOKO which is designed for Finnish concepts, and is then followed by the use WordNet and finally with OpenCyc and DBpedia.

Exploring Semantic Tagging with Tilkut

The analysis uses linked data to get relations to different databases. The benefit of using DBpedia and Freebase is that they have concepts in different languages and they contain additional information relating to the concept, e.g., information relating to persons and locations. Once we have found a meaning for a tag, either by user using the semantic tagging widget or automatically by using the semantic tag analysis, we can expand the knowledge by accessing the data referred by the URIs. This data contains information such as different language versions of tags (rdfs:label, skos:prefLabel, etc.), alternative labels (skos:altLabel, etc.), descriptions (skos:definition, rdfs:comment, etc.), classes (rdf:type) and geo-coordinates for location tags. This additional information offers different utilisation opportunities: different language versions can be used to localize the tags, class information to categorise them and location coordinates to display them on a map. There are still some tag categories like importance and project that are based on each user’s personal preferences and they cannot be inferred automatically. These are put into the miscellaneous category by the automatic tag analyser. We store the gathered additional information in RDF format in a Virtuoso RDF storage. The features of the MOAT (Meaning of a Tag) ontology are used here, since it enables determining alternative meanings for the same tag. With time, the Tilkut database itself will accumulate semantic knowledge that can be used to support tagging.

OSMO Ontology Our social metadata ontology (OSMO) supports describing social media content, activities, users and sites. It has been built by utilising existing ontology such as SIOC, MOAT and FOAF with extensions to support our special requirements, such as tag categories. The OSMO ontology takes into account the special features and requirements of the Tilkut application.

Tags and tagging related information is defined as a combination of Tag ontology, SKOS and MOAT. SIOC is used for describing the content of clip as well as Tilkut service itself. FOAF is used for describing users. Also Dublin Core vocabularies are utilised. Two classes, Tag category and Category, were defined as subclasses of skos:ConceptScheme. MOAT ontology is used for describing tags, but also the properties of tags:Tag as well as skos:Concept are utilised. This approach is possible, because these ontology are interlinked. The hierarchical structure of tagging in Tilkut set additional challenges to the implementation of the OSMO ontology, because users may use the same tag in different tag categories. It is necessary to store the information of which user has used which tag, in which category and when. This information can be regarded as the context of tagging and tags:RestrictedTagging is utilised for describing it. The information is utilised when browsing clips. The user can take different views on the bookmarks with the help of the tag categories. Each user is able to select which tag categories to use. The user selected tag categories are linked to each user’s bookmark category with dcterm:hasPart property. Users are not forced to use the suggested semantic tags, but they can use any words they want as a tag. This means that a tag may have a clear meaning expressed with help of semantic web resources or only a label without exact knowledge about the meaning. The meaning of a tag is expressed with help of moat:Meaning class. Figure 6 shows the relationships between the utilised classes. Users, sharing of content, user created groups and visibility of content needed also to be defined in the ontology. The FOAF ontology together with the sioc:User and sioc:Usergroup classes from SIOC ontology were used for managing information relating to users and usergroups. The foaf:Agent subclasses such as foaf:Person and foaf:Group are utilized from the FOAF ontology.

141

Exploring Semantic Tagging with Tilkut

Figure 6. Interlinking Tag ontology, MOAT, SIOC and SKOS with the Category and Tag_category classes defined for the Tilkut application.

In the Tilkut application, the visibility of the bookmarks is either public or private. The user is able to share bookmarks to groups or other users. Also groups can be public or private. Based on the combination of these options, a set of rules were defined for the visibility and the actions that other users can and cannot do. In the OSMO ontology, the visibility is expressed by using the Dublin Core’s dcterm:RightsStatement class, and the dcterm:rightsHolder property is used to express to whom a bookmark or group is visible. The Visibility class with sub classes Public, Private and Protected, was defined as a sub class of the dcterm:RightsStatement class. The instances of the Protected class are created based on the sharing information. The visibility of the content can be expressed with the property dcterm:accessRights. To validate the ontology, we created a dataset according to the described ontology and tested it with SPARQL-queries. The queries relating to tags and tagging, user created content, users’ social networks and content visibility were tested with a test dataset. This included queries such as “get the meanings of a tag”; “get tags, tag categories and meaning of tags relating to the clips of the certain user”; “get user’s tags in certain category

142

like Topic”; and “get user’s tags in different services”. Relating to visibility and sharing queries such as “get the user’s own clips and such clips by others that have been shared directly to user or based on the membership of a group”. We could successfully extract the queried information. The OSMO ontology is used as the data format in the central RDF-storage and as the exchange format between the developed components and applications. It also supports importing from and exporting to other than our own applications. The Openlink Virtuoso database was used to store the RDF data. Tilkut SQL database were mapped to Virtuoso RDF database using the OSMO ontology. Tilkut and mobile messaging software TagIt were integrated by using the RDF database to store and transfer the common data. TagIt mobile application handles information about the groups that users belongs to in Tilkut. When creating a mobile message for Tilkut, the user is able to share it with other users in the same way as in Tilkut web application. The message created with the TagIt application is stored in the RDF storage and sent to Tilkut.

Exploring Semantic Tagging with Tilkut

Figure 7. OSMO ontology and RDF storage is used in Tilkut and TagIt integration

USER EXPERIENCES OF THE SECOND TILKUT VERSION There are two installations of the second Tilkut version: one for internal use and one for public use. The internal version has all the new features developed for the second version, whereas the public one has most of them, but not the Semantic tagging widget and automatic tag suggestions. Both versions have been online for more than a year, and they have been used in real work-related cases in order to find out, how the Tilkut service in general and semantically enhanced tagging in particular supports knowledge sharing at work. The company internal version is used for personal information storing, knowledge sharing and discussions within project groups and teams. It has also some entertaining use like sharing jokes and funny videos. Other use cases include documentation, bug reporting and tips for using applications. The public version is used mostly in collaborative projects that involve partners from different organisations. The use cases include sharing project related links and articles, and collecting and updating technology reviews collaboratively. Both versions have been used approximately in the same intensity: there are 112 registered users and 805 clips in the public version, and 130 users

and 966 clips in the internal version after one year. Since the service is still a prototype in development, it has not been marketed heavily, and only few people have used the service actively. The vast majority of users have only been looking at other users’ clips. The requirement to install the bookmarklet seems to have been a challenge for a number of users. Experiences of the use of the second Tilkut version was gathered via a targeted email survey to its most active users. Nine most active users were contacted via email or discussion and five answered were got. Only the most active users were selected, because only they had enough experience of the different features. Additional information of user experiences was obtained from the user feedback that had been sent to the developers. One of the most important new features was that folders were replaced by groups and sharing the clips for others became easier. Approximately half of the clips were added into groups (56% in internal version, 45% in the public version), which shows that knowledge sharing to a defined group of people has been an essential purpose for using Tilkut. In the internal version 70% of the clips were public, which means that they are visible

143

Exploring Semantic Tagging with Tilkut

to all company users, whereas only 37% of the clips in the public version were shared to all users. The service in general was seen as a practical and useful tool. Tilkut was used at intervals and its use was linked to such periods or tasks where information was searched intensively and results shared with others. The typical use case relates to a research project with partners from different parts of the organisation or from different organisations and information sharing easily via an online tool that is accessible to all partners. The active users found using Tilkut to be an easy and quick way of sharing project related articles, news and information about new technologies with others. The shared bookmarking tool helped in having a common view in project related issues and reduced the need for overlapping work.

without burdening themselves with classifying the clip with tags. Other users’ clips were browsed using either the tag list or the newest clips in a certain group. Tag categories were considered helpful in finding the right tags and the relevant content. There was no clear winner as to whether to use the tagging widget with only one tag entry field or the first version with different entry fields for tags in different categories. The decision to support several knowledge bases in the tagging widget gives at times very many choices, which was experienced as demanding by some users. The fact that not all tags could be categorised correctly and some of them ended up in the miscellaneous group, lead to that some users preferred to make the tag categorisation by themselves.

“Information retrieval does not become easier [with Tilkut], but it helps in storing the found information in a reasonable place, adding notes (copy of the original text and own comments), and sharing to others (everyone or certain people).” (Woman, using internal Tilkut for knowledge sharing with colleagues and project members)

“Practical service. Good features, like browsing tags with different criteria and adding pictures into clips.” (Woman, using internal Tilkut for sharing links to information services within the organisation)

In the internal Tilkut version, on average 2,6 tags were added per one clip. The active users regarded tags as a way to categorise and summarise the content as well as to harmonise the use of terminology within a group. “Normally I add tags. They provide an easy way to glance, what the article is about. Tags are a kind of a summary.” (Man, using public Tilkut for project related sharing) The decision whether to use tag suggestions or write one’s own tags varied depending on the user. The “filers”, who like to have information organised, prefer ontology-based tags in order to keep the categorisation coherent with others. On the contrary the “pilers” tended to use the groups to share information for a certain group of people

144

The biggest worries of the users were related to how to find the important and interesting clips also, when the service gets more popular and there are thousands of clips in different groups. Semantic tags should help in finding the most relevant clips for oneself. The opportunity to rate others’ clips was proposed as one possible way of making it easier for everyone to find out the most interesting clips.

FUTURE RESEARCH DIRECTIONS When developing semantic tagging, it is important to keep in mind that tags are a means to support finding resources and linking related items and people with similar interests, and are of value only when they contribute to offering useful features. Even though using only semantic tags gives the best opportunities for automatically processing,

Exploring Semantic Tagging with Tilkut

it will probably be too limiting for the average user. If the user is limited to using an ontology that does not contain the concepts she or he needs, this leads either to limited use of the system or misusing available concepts. For the application development, support for both semantic and freely generated tags set many challenges. We can see two research directions from here: one to developing methods that make selecting and using semantic tags easier, and the other relying more or even completely on automatic tag generation. The tags produce a personal interest profile. The profile has potential in enterprise use to show the areas of interest and expertise and gives opportunities to locating other people with similar or complementing interests. The profile could further be developed into a personal ontology that has potential in any knowledge related application. When the Tilkut application has many items and users, recommendations and aggregations could be made based on the content and the personal interest profiles. The profiles could also be used to search and process external knowledge sources. Our own experiences and those reported by others show that social media tools have potential in enterprises for creating and strengthening weak links. A topic for future research is to find the correct combination of features so that the tool would support both link and knowledge sharing and building and maintaining networks. In the future research we will focus on developing new features based on the added semantics, so that the benefits of the semantic tagging become clearer to users. These new features include automatic content aggregation, and analysis and visualisation of content based on semantics. Also using semantic user profiles for recommending content, groups and users will be addressed in the future work. We will also develop automatic tag suggestions that are based on our semantic tag analysis. This way we will get better support for the Finnish and Swedish languages. Rather a small number of users were involved both in the first user studies as well as in the

long-term use of the second version in real context. Therefore more user studies are needed in order to evaluate the possibilities and restrictions of semantic tagging more extensively. Both the user experience of tagging and tag based content search, as well as the larger context of workrelated knowledge sharing in large scale should be studied further.

CONCLUSION The Tilkut application lets users either create their own tags or utilize ontology based tags to describe their resources. Even though our user tests only had a small number of users, it became clear that people have very different preferences and both approaches have their supporters. The preferences also depend on the usage context – for personal bookmark management a very simple approach may be enough but to support findability within a company or larger group of peers, semantic support is useful. Because of these varied user needs and preferences, the possibilities to customize and fine tune the application are important, and one should try to avoid defining use cases and purposes too strictly in advance. The first Tilkut version, which was used in the initial user tests, required several decisions from the user. We developed semantic features such as automatic tag suggestions so that the user’s task would be reduced to accepting some of the proposed ones, and adding only those that describe a very personal view or relate to personal importance. The use of the predefined categories was not received well by those who did not find the category they would have needed, or who did not experience the tag categories relevant. Using categories for other than the intended purpose also adds noise to the system and reduces the accumulative value of tags, and should therefore be avoided. We simplified tagging by making it possible to use only one tagging field, and mak-

145

Exploring Semantic Tagging with Tilkut

ing the tag categorisation automatically with the help of various knowledge bases. Since some users appreciate exact control on how tags are categorised, we still support adding tags directly to different tag categories as well. By using several external knowledge bases we can cover a wide gamut of topics and the user will likely be able to find a correct tag from among the suggestions, but there are also challenges and drawbacks in this approach. There are overlapping concepts in different knowledge bases which may feel confusing to the user who needs to try to select the most appropriate one. Also the exact meaning of the suggested term may not always be clear because the available explanations may be very brief. Giving the users the opportunity to choose which vocabularies to use and when to use them, improves the user experience and reduces the misuse of tags. Supporting several knowledge bases sets additional requirements to the application development as well. One of the implementation related challenges is that there is no unified way to access semantic knowledge bases as each of them has their own APIs and protocols. It also requires understanding of what kind of knowledge is available in the different knowledge bases, and how it can be best processed to offer intelligent features to users. When using Linked data online, we are also dependent on the services and their response times. When developing semantic bookmarking applications, attention should be paid particularly to the topic category. The number and granularity of the tags becomes quickly very large and heterogenic in that category. Tools should be developed to utilize the semantic information and offer various ways for viewing and organizing tags. These can be built, when there is both semantic information of the meaning of the tags, and of tagging as a whole. Social bookmarking has potential in an enterprise for promoting knowledge sharing and giving a tool for seeing what people are currently

146

interested in. There is also potential for creating and strengthening weak links. The challenge is to get people to adopt a new way of working and a new tool as a part of their daily working routines. In order to succeed in this, ease of use and intuitiveness at the main use cases like adding and sharing new content, exploring one’s own and other’s content, and seeing the latest and hottest items based on one’s own interest profile are in key role. In spite of the improvements, we still see need for a more simplified user interface that reduces the requirements of user decisions and activities. There are also opportunities in developing new features that utilise the semantics of the system, such as automatic content aggregation and visualisation, and personalised recommendation of content and users.

ACKNOWLEDGMENT This work is based on an earlier work: Experiences of semantic tagging with Tilkut in Proceedings of the 12th international conference on Entertainment and media in the ubiquitous era, (2008) (c) ACM, 2008. http://doi.acm. org/10.1145/1457199.1457236 This is a major revision to the earlier published article. The new material includes: • • • •

The survey of the requirements for enterprise use The second version of Tilkut with enhanced features The user experiences of the second version of Tilkut Enhanced semantic features; the semantic tagging widget, automatic tag suggestions, the semantic tag analysis of freely given tags, automatic categorisation of tags, updated OSMO ontology

Exploring Semantic Tagging with Tilkut

REFERENCES Bojãrs, U., Passant, A., Cyganiak, R., & Breslin, J. (2008, April). Weaving SIOC into the Web of Linked Data. Paper presented at Linked Data on The Web workshop (at WWW2008), Beijing, China.

Nummiaho, A., Vainikainen, S., & Laakko, T. (2009). Utilizing Existing Semantic Knowledge Bases for Creating Annotations on Mobile Devices., In the 2009 Tenth International Conference on Mobile Data Management: Systems, Services and Middleware (pp 554-559). IEEE Computer Society.

Davenport, H. T., & Prusak, L. (1998). Working knowledge: How Organizations Manage What They Know. Boston, MA: Harward Business School Press.

Paroutis, S., & Saleh, Alya, Al. (2009). Determinants of knowledge sharing using Web 2.0 technologies. Journal of Knowledge Management, 13(4), 52–63. doi:10.1108/13673270910971824

Golder, S., & Huberman, B. A. (2006). Usage Patterns of Collaborative Tagging Systems. Journal of Information Science, 32(2), 198–208. doi:10.1177/0165551506062337

Vainikainen, S., Nummiaho, A., Bäck, A., & Laakko, T. (2009). Collecting and Sharing Observations with Semantic Support. In 3rd International AAAI Conference on Weblogs and Social Media (pp 338-341). AAAI Press.

Kim, H. L., Passant, A., Breslin, J. G., Scerri, S., & Decker, S. (2008). Review and Alignment of Tag Ontologies for Semantically-Linked Data in Collaborative Tagging Spaces. In Proceedings the 2nd IEEE International Conference on Semantic Computing, Santa Clara, USA, August 2008. Kim, H. L., Scerri, S., Breslin, J. G., Decker, S., & Kim, H. G. (2008, September). The State of the Art in Tag Ontologies: A Semantic Model for Tagging and Folksonomies. Paper presented at the 8th International Conference on Dublin Core and Metadata Applications, Berlin, Germany. Maala, M. Z., Delteil, A., & Azough, A. (2007, April). A Conversion process from Flickr tags to RDF descriptions. Paper presented at Social Aspects of the Web Workshop, Poznan, Poland. Marlow, C., & Naaman, M. boyd, D., & Davis, M. (2006, May). Position Paper, Tagging, Taxonomy, Flickr, Article, ToRead. Paper presented at Collaborative Web Tagging Workshop (at WWW 2006) Edinburgh, Scotland. Newman, R. (2005). Tags. Retrieved February 10, 2010 from http://www.holygoat.co.uk/blog / entry/2005-03-23-2

Xu, Z., Fu, Y., Mao, J., & Su, D. (2006, May). Towards the Semantic Web: Collaborative Tag Suggestions. Paper presented at Collaborative Web Tagging Workshop (at WWW2006), Edinburgh, Scotland.

KEY TERMS AND DEFINITIONS Ontology: A formal representation of concepts and their relationships within a domain. Semantic Web: Machine understandable data. Social Bookmarking: Social media service where users are able to create and share bookmarks of web resources. Social Semantics: Social media + Semantic web; Social semantics combines the best of these worlds -easiness of social media and intelligence of semantic web technologies Tag: User defined keyword. User Study: A service is evaluated by testing it with users. Web Application: An application that is accessed via a web browser over an Internet.

147

Exploring Semantic Tagging with Tilkut

ENDNOTES 3 4 5 1 2

8 9 6 7

148

http://owela.vtt.fi/tilkut/ http://delicious.com/ http://www.flickr.com http://www.geonames.org/ http://www.yso.fi/onki2/ overview?o=http%3A%2F%2Fwww.yso. fi%2Fonto%2Fkoko&l=en http://www.freebase.com/ http://wiki.dbpedia.org/About http://www.w3.org/2004/02/skos/specs http://moat-project.org/

12 13 14 15 16 17 18 10 11

21 19 20

http://sioc-project.org/ http://xmlns.com/foaf/spec/ http://notitio.us/ http://wordnet.princeton.edu/ http://www.dmoz.org/ http://code.google.com/p/gnizr/ http://zigtag.com http://www.faviki.com/ https://addons.mozilla.org/fi/firefox/addon/748 http://www.yso.fi/onki2/ontologies?l=en http://www.zemanta.com/ http://microformats.org/wiki/rel-tag

149

Chapter 8

A Knowledge-Based Multimedia Adaptation Management Framework for Ubiquitous Services Ning Li The Open University, UK Abdelhak Attou University of Surrey, UK Merat Shahadi Kings College London, UK Klaus Moessner University of Surrey, UK

ABSTRACT The range of multimedia contents and services on the Internet, the diversity of terminals, and the heterogeneity of network technologies make it less and less feasible and rather costly for providers to prepare contents and services in advance in all conceivable formats. There is a need to incorporate dynamic adaptation management into existing multimedia content/service delivery networks. We propose an Adaptation Management Framework (AMF) that provides architectural and functional support allowing dynamic and autonomous content/service adaptation without introducing additional complexities to the actual content/service provider or the user. The AMF provides functionalities needed in such an automated adaptation process, including context representation, adaptation decision making and adaptation operations selection across heterogeneous entities and platforms. It alleviates the complexity of those tasks using ontology representation formalism and knowledge-based processing techniques. It deploys itself as well as associated third-party applications, such as adaptation tools, as Web Services to enhance the interoperability among different entities. The AMF can be plugged into content/service delivery networks as an adaptation engine and serves as an invisible service enabler for ubiquitous content/service delivery. DOI: 10.4018/978-1-60960-774-6.ch008

Copyright © 2012, IGI Global. Copying or distributing in print or electronic forms without written permission of IGI Global is prohibited.

A Knowledge-Based Multimedia Adaptation Management Framework for Ubiquitous Services

INTRODUCTION The concept of ubiquitous services has been attractive to service providers, telecommunication operators and technology manufacturers alike because of the increased revenue prospects. As the name implies, ubiquitous services represent communication scenarios where services can be accessed anytime, anywhere and anyhow without explicit involvement from any players in the service delivery process. Today’s network access technologies, such as WiFi, WiMAX, seem to bring the vision of ubiquitous services closer to reality, another major barrier to ubiquitous services provision remains still as a challenge, that is, to deliver a mix of contents and services via a multitude of heterogeneous access networks and technologies to a wide range of access devices users may have as well as user’s different preferences and likings. Delivery of a content and service mix, such as multimedia, in comparison with uni-modal web contents, in such heterogeneous environment is technically more challenging because multimedia formats themselves can be heterogeneous, for example in terms of their encoding. Even for the same coding format, it still can vary in encoder settings such as spatial and temporal resolution, colour depth etc. Due to this heterogeneity, today’s end users are in general not able to access multimedia ubiquitously. Some kind of adaptation and delivery management is necessary (Jannach et al., 2006; Li & Moessner, 2007). Previous approaches to this problem, such as multi-authoring (Hanrahan & Merrick, 2004), are static in that they require providers to prepare multiple versions of the same content for a number of possible devices which may render the content, or simply differentiate their contents for mobile devices from PCs typically for web contents by starting the content URL with mobile or ending with .mobi etc. This is neither a cost-effective approach from providers’ point of view, nor does it provide the flexibility to incorporate new devices that may reach the market after content has been

150

generated. Different from those approaches, the adaptation management system we aim to develop should have the capability to manage adaptation dynamically and autonomously without explicit involvement from either users nor content/service providers, the extensibility to incorporate new media types and support new devices, the inter-operability with standardisation efforts in relevant domains. To facilitate a seamless and ubiquitous user experience and enable persistent service access when user moves across networks and changes devices without user’s explicit involvement in configuration, it requires not only constant sensing of user’s surrounding context, such as available devices and their capacities, but also linking the context to the requested multimedia content. Effective context description scheme and representation formalism play a key role. Apart from user’s context, an adaptation management process also needs to be aware of the context of multimedia contents as well as that of any available multimedia adaptation operations. Basically, all those contexts form a knowledge base and need to be described in a way that is amenable for processing those knowledge as well as their instances. Ontology, particularly with its technological development in Semantic Web field, has become popular in recent years as the means for knowledge representation due to its added layer of semantic that provides a common and formal understanding of domain concepts on top of the syntax modelling provided by existing schema languages, typically XML. There have been a number of efforts that introduced ontology and its technologies into multimedia adaptation and delivery domain, such as the work in (Jannach et al., 2006; Soetens et al., 2004; Yu et al., 2006). However, they either, as in (Jannach et al., 2006; Soetens et al., 2004), focus only specifically on issues of how to semantically describe adaptation operations in order to facilitate the autonomous composition of a multi-step adaptation operations with planning

A Knowledge-Based Multimedia Adaptation Management Framework for Ubiquitous Services

methods from Artificial Intelligence field, or, as in (Yu et al., 2006), only on the descriptions of context and multimedia content while leaving those of adaptation operations out and therefore not involving them in adaptation decisions making process. They lack systematic view of and therefore did not address how to organise and develop an overall adaptation management system which starts from when a user raises a request for multimedia service to when the user is responded with that service which suits his/her circumstance most. The issues encountered in the design of such a system, besides contextual knowledge representation, adaptation decision making and adaptation operation orchestration, include also how to build system architecture and organise the functionality to facilitate the interactions among all the involved entities. Those entities may still use legacy technologies for their own benefits and may not yet take the advantage of a knowledgebased approach. In this chapter, we present an Adaptation Management Framework (AMF) endeavored to provide the architectural and functional support for multimedia adaptations when applied to a ubiquitous communication environment. It provides solutions to a number of core issues and major complexities in an adaptation process, such as context modelling, adaptation decision making, and adaptation operations selection etc. These tasks may have been separately investigated before, but not sufficiently addressed when they are interlaced with and have to work together with each other in a multimedia adaptation process. With the recent developments in knowledge modelling formalisms and knowledge processing techniques based on those formalisms, the tasks mentioned above become more approachable. This chapter presents our approaches in how to use knowledge-based technologies to tackle the issues and complexities in a multimedia adaptation process. This chapter is organised as follows. Section 2 describes the definition of an Adaptation Ontology which provides vocabulary for the content/

service adaptation domain using OWL semantic modeling language. The Adaptation Ontology serves as the Knowledge Base (KB) for the AMF and models context of the entities involved in an adaptation process. Some background information for ontology modeling is given. In section 3, the architecture and functionality of the AMF are described. The AMF comprises of two major components, the Adaptation Manager (AM) for context modelling, context analysis and adaptation decision making and the Content Adaptor (CA) for the selection and coordination of adaptation operations to fulfil adaptation tasks. The functions and the main algorithms used for each component are described. Section 4 gives implementation details of the AMF for the purpose of a proof-ofconcept demonstration. In section 5, we present three user scenarios, which helps to demonstrate the AMF. Section 6 provides a further discussion about the system features and concludes the work.

ADAPTATION ONTOLOGY The information from all the entities involved in a multimedia adaptation domain form a contextual knowledge base that needs to be shared by different operating entities in order to come up with an adaptation decision autonomously. In recent years, ontology and ontology languages have been recognised as the knowledge description scheme and knowledge representation formalism respectively. They together provide addition of semantics to the knowledge and thus facilitate the knowledge processing. However, to define a set of commonly-agreed vocabularies to facilitate multimedia adaptation remains as an uneasy task because it involves different communities with each having its own solution.

151

A Knowledge-Based Multimedia Adaptation Management Framework for Ubiquitous Services

Adaptation Domain Context Modeling The adaptation domain contains information of users, multimedia contents as well as adaptation operations. To describe information of users and multimedia contents, many efforts have been seen in recent years aiming to reach a description standard in order to achieve maximum acceptance and interoperability among communities. So far, the widely acknowledged and practically adopted standards include CC/PP1, UAProf2 and MPEG-21 Digital Item Adaptation (DIA) (IST/37, 2004)) for users and MPEG-7 (Martínez ed., 2004) for multimedia contents. For adaptation operations, fewer efforts have been seen so far mainly because adaptation operations, such as video codecs, vary in their implementations, and therefore to define an abstracted description on top of heterogeneous implementation interfaces is far more complex. Web Service standards are one of those standards which ensures interoperability over the Internet and comes with a description mechanism, namely Web Service Description Language (WSDL)3. Semantic Web Services are steps further to enhance Web Service descriptions with semantic markups. The efforts include WSDL-S4, OWLS5, SA-WSDL6 and a few more. WSDL-S and OWL-S model each web service in terms of its Input, Output, Precondition and Effect (IOPE) parameters, which is handy in helping work out a composition of adaptation operations in the case of content adaptation. However, automatic detection of suitable adaptation operation services and composition of service chains can only be done on the basis of a shared ontology, i.e. the adaptation operation services have to have a common understanding of the terms being used in their IOPE parameters. MPEG-7 and MPEG-21 DIA can work as such a shared ontology. However, not all the required operations are in the form of Web Service yet and quite in the contrary that a majority of adaptation operations are in legacy formats, such as Java APIs or console commands.

152

Among all those efforts, MPEG-7 and MPEG21 DIA provide a good combination to link multimedia content description with user environment description besides their well-established comprehensiveness in describing their respective domains. MPEG-7 offers several tools, i.e. Description Schemes (DS), to annotate multimedia content at different levels. These include Description Definition Language (DDL), Visual Schemes, Audio Schemes and Multimedia Description Schemes etc. The most relevant part within MPEG-21 standard for the adaptation domain is Digital Item Adaptation (DIA) though MPEG-21 provides other tools to describe the environment enabling transparent multimedia creation, delivery and consumption among heterogeneous environments. The DIA provides tools to describe the user environment including: User characteristics, such as user info, preferences, usage history and physical characteristics, Device characteristics, such as display, memory and battery, Network characteristics, such as error characteristics and bandwidth, and Natural Environment characteristics such as noise and illumination. Although MPEG-7 and MPEG-21 DIA standards have been acknowledged for their strengths in multimedia domain description and delivery, their formats are XML which ensures most interoperability at the time of development. Nowadays, their strengths can be greatly enhanced by adding machineprocessable semantics via ontology representation languages, such as OWL7. Recent research in multimedia adaptation domain has reflected the recognition of using MPEG-7 and MPEG-21 DIA, together with ontology-based technologies, to support multimedia content adaptation (Jannach et al., 2006; Soetens et al., 2004). However, though OWL has been chosen as the description language, in (Soetens et al., 2004), there is only a limited usage of MPEG-21 vocabularies due to the immaturity of this standard at the time of writing. In (Jannach et al., 2006), though MPEG-21 DIA vocabularies are adopted to form the domain ontology, the representation

A Knowledge-Based Multimedia Adaptation Management Framework for Ubiquitous Services

remains its original format of XML. With the actual adaptation operations being described as semantic web service, their work realises the integration of the different representation formats at a technical level using XML transformation and the adaptation decision making using logic-based technologies. When the context of multimedia contents, usage environment and adaptation operations are all semantically described, adaptation strategies can be worked out simply by imposing some rules on top of the semantically described metadata. However, before any Semantic Web Service Standards as well as associating semantic annotating tools become maturely developed and widely accepted, the intermediate approach as in (Jannach et al., 2006) can still be used sometimes to bridge the semantic gap between legacy system and semantic-enhanced system.

Adaptation Ontology Constructions In our work, MPEG-21 DIA and MPEG-7 vocabulary are chosen to model the knowledge of users and multimedia contents respectively. They also serve as the vocabulary base for modeling the IOPE parameters of adaptation operations. However, to take advantage of the reasoning power of formal Description Logic (DL) to facilitate automatic adaptation decision making, they need to be described in a semantic-rich language and OWL-DL is the best choice to this need. There exist several efforts to construct semantically rich representation of MPEG-7 and MPEG-21 ontology using ontology languages like OWL and RDF(S) (Tsinaraki et al., 2004; Garcia & Celma, 2005). For example, in (Tsinaraki et al., 2004), an ontology based on MPEG-21/7 was constructed and used in the Domain-Specific Multimedia Information and Filtering Framework (DS-MIRF) to facilitate the development of knowledge-based multimedia applications such as multimedia information retrieval, filtering, browsing, interaction, extraction, segmentation, and content description. Those efforts construct

ontology automatically by means of Extensible Stylesheet Language Transformations (XSLT)8 according to some rules specified as in (Tsinaraki et al., 2004; Garcia & Celma, 2005). By automatically converting the XML tree structure, the obtained ontology describes the relationship between the types of the tree element instead of describing the relationships between the semantics embodied by the tree elements. Although this approach expresses the XML-based standards in an OWL or RDF(S) format, it does not add much semantic expressiveness to them. Such approach would be applied in any automatic XML schema to OWL conversion regardless of the semantics of the respective domain. We argue that, for an expressive OWL representation of the XMLbased standards, manual conversion is necessary. Manual conversion of MPEG-7 into RDFS has been seen in (Hunter, 2001) and the limitation of RDFS has been discussed. There are no rules on how to manually convert an XML schema description into OWL ontology. Manual conversion has to examine the elements and the attributes of the XML schema, study their semantics, and translate them into OWL constructs (Li et al., 2007). We manually constructed the MPEG-21 DIA ontology by observing the semantics of the terms as explained by MPEG-21 DIA authors (Li et al., 2007). The relationships between classes and between classes and their properties are carefully assigned. The manual conversion shows a clearer hierarchy that makes more sense to human readers and organises relationships between ontology concepts according to the semantics of the term in the multimedia domain as described in the MPEG-7 and MPEG-21 DIA specification. In particular, the base concepts underlying the Adaptation Ontology are defined in MPEG-21 DIA Universal Environment Description (UED) and MPEG-7 Multimedia Description Schemes (MDS). Thus constructed ontology forms the Knowledge Base (KB) for the targeted multimedia adaptation domain. The Adaptation Ontology is represented using OWL-DL. The availability of

153

A Knowledge-Based Multimedia Adaptation Management Framework for Ubiquitous Services

well studied and researched DL reasoners accessible via standardised interaction interface9, such as Fact10, Pellet11, makes it possible to check the consistency of the KB to ensure that extension or modification does not cause inconsistencies or incorrect results of reasoning.

Using Adaptation Ontology in AMF The constructed Adaptation Ontology forms the KB for the proposed Adaptation Management Framework (AMF). Generally, the input to the AMF is in the form of profiles describing instances of the adaptation domain. For example, a content profile describing a piece of video, a device profile describing the phone Nokia N70 or a user profile describing the user Dave, his preferences, his PDA and the network he is connected to. If these profiles are all described using vocabularies from the Adaptation Ontology, they will be understood to and thus can autonomously interoperate with each other. For example, if the resolution property of a video has the same semantics as the resolution property of a device display, an adaptation decision can be made upon whether this piece of video can be delivered to that device straight away by simply comparing the values of these two properties. However, it is unreasonable to expect all profiles, such as device profiles dynamically collected from user’s environment, are in the form exactly compliant with the Adaptation Ontology under current circumstances where multiple description standards co-exist and may be used by difference device manufactures for example. This does not imply that they are excluded from the AMF. When the input profiles are expressed using vocabularies from other standards, such as CC/PP, UAProf, as well as in other formats such as XML, RDF, they can be mapped in terms of both vocabulary and format to those of the Adaptation Ontology. Contrary to the large number of vocabularies involved when constructing the Adaptation Ontology KB, the number of terms used in a context profile is much less and hence the

154

semantic mapping of vocabularies is rather easy. The format mapping can be more efficiently done using XSLT transformation than manually since all formats are syntactically standardised. Therefore, it is efficient to do both mapping together using XSLT transformation by defining and applying corresponding conversion rules. This results an OWL instance file that imports the Adaptation Ontology and contains only ontology vocabularies.

ADAPTATION MANAGEMENT FRAMEWORK In a nutshell, the AMF takes relevant inputs, makes adaptation decisions, carries out adaptation operations if needed, and outputs the adapted content/ service which is now compatible with the devices within user’s environment where the request for content/service originates. We propose two major functional entities to provide the required functionality for content/service adaptation management, that is, the Adaptation Manager (AM) for context acquisition, context analysis and adaptation decision making, and the Content Adaptor (CA) for coordinating adaptation operations to fulfil the adaptation tasks (Li & Moessner, 2007). This highlevel AMF architecture is depicted in Figure 1. The proposed AMF logically resides between the user environment and the content/service provider to dynamically manage any required adaptation. When it comes to physical locations, depending on who and how to deploy this system, all AMF functionalities can be centralised on devices in user environment, content servers or intermediate third-party servers. However, with a modular design of the AMF, it is possible to distribute the functions of the AMF across networks to, for example, make optimal use of network resources or avoid performance bottlenecks etc.

A Knowledge-Based Multimedia Adaptation Management Framework for Ubiquitous Services

Figure 1. Adaptation Management Framework architecture

Adaptation Manager

Figure 2. AM architecture: main components

As the core of the AMF, the AM performs the tasks of acquiring context information, formatting it, analyzing it and making corresponding adaptation decisions. These correspond to three major functional entities referred to as Context Provider (CP), Context Reasoner (CR) and Adaptation Decision Engine (ADE) respectively. Figure 2 depicts the internal architecture of the AM. In general, an adaptation request triggers an adaptation process. Within each adaptation process, the functionalities of CP, CR and ADE are invoked in sequence.

Context Provider The Context Provider performs mainly two functions, context extraction and context formatting. An adaptation request normally is accompanied by one or more profiles defining the request context. The context includes not just user’s information, such as his device capability and his preferences, it also contains the context of the requested content, i.e. content metadata, as well as the capability of Adaptation Operations (AO). In situations where no description comes with the content, context extraction needs to be performed for different media types to extract content description. Once the raw context descriptions are obtained, they need to be formatted to the required format i.e. OWL-DL. The resulted files import the Adaptation Ontology and are instance ontologies that

describe particular instances from the real world such as the user Dave possessing a Nokia N70 phone and requesting an online video service from BBC. The Context Reasoner then takes the output of the Context Provider and refines it into the decision input parameters which are then fed into the Adaptation Decision Engine to make adaptation decisions.

Context Reasoner Prior to any use of the context, context reasoning is required mainly for two different purposes. Firstly, for inference of high-level context from low-level context, or interpretation of context from one type to another, this may be needed by one particular service to be able to adapt its behaviour. For example, a device-oriented service may base its

155

A Knowledge-Based Multimedia Adaptation Management Framework for Ubiquitous Services

response upon what category the device belongs to, such as high-definition or low-definition, instead of resolutions in numerical formats. Secondly, it is for maintaining knowledge consistency and detecting any subsuming relationship among context when more and more knowledge are being newly defined or being derived from existing ones. Therefore, to some extend, context reasoning is part of the adaptation decision making process. For example, the CR can deduce that there is a screen size limitation if the multimedia service required display size is more than the user’s device screen size. This limitation can be modelled in OWL 1.112 and in DL notation as follows: Small_Screen_Device ≡ ∃screenSize <X

This statement means that class Small_Screen_ Device is equivalent to the existence of an instance which has a screenSize datatype property such that the value of that property is less than a certain matrix X. The class name generally is self-explanative. In this case, X represents the required screen size by the original multimedia content. Therefore, when applying context reasoning to a user’s device context ontology instance which contains a screenSize datatype property whose value is less than X, it is deduced that this device is a small screen device and belongs to the Small_Screen_Device class. An alternative solution to this new class and datatype definition is using user defined rules that can be applied to ontology with the aid of rule languages such as Semantic Web Rule Language (SWRL)13 and rule engines such as JESS14 to reason upon defined rules. For example, if a class of Memory_Constrained_Device has the memory size property restricted to less then 20 Mb, a device instance that has the memory size property of 19 would be automatically classified by a reasoner as a memory constrained device. To do this, the following SWRL rule can be used:

156

Device (a?) ∩ Memory (b?) ∩ hasMemory (a?, b?) ∩ size (b?, c?) lessThen (c?, 20) → Memory_Constrained_Device (a?).

All defined rules can be saved into rule ontology separately.

Adaptation Decision Engine The task of Adaptation Decision Engine is to use contextual information obtained so far and make adaptation decisions on how to achieve the most suitable content according to the collected context. The context of Adaptation Operations is used in the decision making process. Possible Adaptation Operations are maintained by the CA and updated to the AM. For example, knowing that the device has been categorised as Video_Resolution_Limited and Battery_Limited device by the Context Reasoner, the ADE calculates the maximum resolution that all video files shall not exceed. The value should be less than the maximum resolution supported by the device display and should consume battery resources according to the deduced battery limitation. Such decisions including the parameter set will be handed over to the Content Adaptor to guide the actual actions for adaptation.

Content Adaptor Content Adaptor is the entity within the AMF to manage the actual adaptation process. In brief, it takes the part of the content requiring adaptation and its description as the input and outputs the adapted version of the content, also with its description, for delivery to the end-device. The adaptation process carries out generally a number of operations, referred to as Adaptation Operations, to change the content to its final form suitable for delivery. Examples of Adaptation Operations include video compression, language translation etc. Each Adaptation Operation could have many variations of implementations with each varying

A Knowledge-Based Multimedia Adaptation Management Framework for Ubiquitous Services

by input and output parameters for example. The actual implementations of the Adaptation Operations are referred to Adaptation Mechanisms (ADMEs) in this work. For example, to do a BMP image compression operation, either a BMP to JPG image transcoder or scaling software may be used. CA should pass the list of discovered ADMEs as well as any changes in status of the ADMEs to the AM in order to calculate the capability of the AMF overall and thus make adaptation decisions when receiving user’s request.

Figure 3. Content Adaptor Architecture: main components

CA Architecture CA is responsible for performing the adaptation tasks dictated by the AM. It maintains an extensive ADME Profiles Repository that includes ADME information discovered from all parts in the delivery network, including third parties. The CA architecture is depicted in Figure 3. As it can be seen, the CA’s inputs are original content and adaptation decisions, which include the parameters needing to be met during the adaptation process. The CA’s output would be the desired adapted content. CA is composed of the following modules: •

•

CA Interface: the interaction interfaces with the AM via this module to, for example, receive adaptation decisions, add or update the AM with ADMEs information. CA Decision Logic: is responsible for associating adaptation decisions, received from the AM, with Adaptation Operations and assigning an appropriate ADME to each operation. CA Decision Logic performs three main tasks. The Adaptation Context Inference Engine further refines the adaptation parameters given by the AM into parameters consisting mainly of the input format and the desired output format along with a number of auxiliary contexts such as user’s preferences towards time and cost etc. Based on this context, the Adaptation Plan Formulation is performed which is re-

•

•

sponsible for determining the sequence of Adaptation Operations, to produce the output format of the content. The final task is ADME Selection to select the most suitable ADME for each Adaptation Operation dictated by the Adaptation Plan Formulation from those included in the ADME Profiles Repository. ADME Profile Repository: stores the ADMEs available in the environment including their profiles. Plan Execution: is responsible for coordinating the execution of the adaptation plan such as forwarding the content to the chosen ADMEs in sequence for performing adaptation.

Adaptation Mechanisms Selection Strategy The most important functionality of the CA resides in the CA Decision Logic. That is to determine a sequence of Adaptation Operations that produces the desired output format of the content. Since original content may have to be converted into several intermediate forms before being transformed into its final desired form, content adaptation becomes a multi-step process involving a number of ADMEs each performing a specific Adaptation

157

A Knowledge-Based Multimedia Adaptation Management Framework for Ubiquitous Services

Table 1. ADMEs, input parameters and output parameters Video to Audio Converter

Input Parameter

Output Parameter

ADME1

Video

Audio MP3

ADME5

Audio MP3/MP4

Sum. Audio MP3/ MP4

ADME2

Video

Audio MP4

ADME6

Audio AMR

Sum. Audio AMR

ADME3

Video

Audio AMR

-

ADM4

Video

Audio WAV

-

Operation. Therefore it is very important to have a kind of selection strategy to choose the most suitable set of ADMEs out of many. The suitability of ADMEs is defined by the quality of service that satisfies the user preferences, for example, with regard to time and cost of adaptation. A simple and straightforward selection strategy is Bubble Sorting Algorithm15 when there are not a large number of ADMEs involved. A set of ADMEs which, for example, takes shortest time and least cost will be chosen to do the entrusted adaptation task. However, when any input of an ADME from one Adaptation Operation can not accept the output of the ADME from the Adaptation Operation in previous step, meaning there is a need for checking whether input and output parameters match or not between two consecutive ADMEs corresponding two consecutive Adaptation Operations, a more sophisticated algorithm is needed. The approach we use is to build an adaptation graph from all the available ADMEs for the required Adaptation Operations and then find the shortest path out of all possible paths from original content to adapted content. Each path consists of one or many ADMEs. One of the main advantages of building adaptation graph is to avoid inefficient use of some of ADMEs, i.e. the ADMEs which cannot lead us from original content to adapted content will not be part of the graph. Each ADME has unique input parameters and output parameters, therefore the paths in the graph is be created by matching the input and output parameters of ADMEs candidates until

158

Audio Summarization

Input Parameter

Output Parameter

the output parameters are equal to those of the adapted content. For example, in a scenario where it is decided that a video needs to be converted to an audio and then further summarised because, for example, user’s device battery is running low and can not play a long audio file. Suppose we have total number of 6 ADMEs and we have the following input and output parameters for each ADME, as shown in Table 1. Assuming the audio formats accepted by user’s mobile phone are MP3, MP4 and AMR only, the final summarised audio format should be in one of those formats only. The adaptation graph for this example is drawn in Figure 4 which contains 3 different paths. ADME4 has to be removed from the graph since the output of which would not

Figure 4. The adaptation graph

A Knowledge-Based Multimedia Adaptation Management Framework for Ubiquitous Services

match the final adapted form or any other ADMEs’ input.

Figure 5. MPEG-21 DIA vs. Adaptation Manager

IMPLEMENTATIONS So far, the architecture and the functionality of the AMF were described; the actual implementation of the AMF and its functionalities is described in this section, including the AM, CA and also providing an overview of how the AMF integrates and uses these functions in an adaptation process.

Adaptation Manager Implementations The implementation of the AM conforms closely to the specification of the MPEG-21 DIA standard with the major difference being that the MPEG21 DIA is mainly an XML-based framework whereas the AM uses OWL-based technologies. In MPEG-21 DIA framework, a decision framework is proposed which consists of the Adaptation Decision Taking Engine (ADTE) and the Bitstream Adaptation Engine (BAE) that executes the decisions made by the ADTE. The ADTE takes as input Usage Environment Descriptions (UED), Universal Constraints Descriptions (UCD) and Adaptation Quality of Service descriptions (AdaptationQoS). The output is the decisions which are the settings of the parameters such as bit-rate, resolution and frame rate. UED, UCD and AdaptationQoS are all defined in DIA. The AM implementation complies with those definitions. UED is implemented as the Adaptation Ontology KB. It provides description tools for multimedia usage environments including devices, networks and users. UCD provides description tools for restrictions which are considered when making adaptation decisions, for example, an adaptation is needed to achieve a certain quality level, a resolution less than 50% of the device resolution, and a match to the device colour capabilities. UCD is an output of the Context Reasoner and input to the

Adaptation Decision Engine of the AM. AdaptationQoS is an input to the Adaptation Decision Engine to assist the decision making process by describing the relationships between Adaptation Operations, content parameters and the resulting quality metrics. The output of the Adaptation Decision Engine would be service parameters such as exact resolutions and bit-rates. The AM’s conformance to the MPEG-21 DIA framework is depicted in Figure 5. However, BAE is not implemented here to do the actual adaptation to the content. Instead, ADMEs implemented as Web Services are used to do the actual adaptation, which will be described in next section. We use the MPEG-7 Audio Encoder16, which provides a Java API to extract MPEG-7 descriptions from audio content, as one context extractor. For videos, we have not found any tools amenable to our needs. Instead, limited information, such as bit-rate, frame-rate are manually attached when demonstrating the AMF using defined scenarios as described in Section 5. Currently, two context formatters are implemented in the Context Provider of the AM, one for MPEG-7 Multimedia

159

A Knowledge-Based Multimedia Adaptation Management Framework for Ubiquitous Services

Description Schemes (MDS) and the other for MPEG-21 DIA. The formatters are implemented in XSLT as a set of rules, and can be used to convert context profiles (MPEG-7 MDS and MPEG-21 DIA XML file instances) to Adaptation Ontology instances that are based on OWL-DL. Formatters for other description schemes, such as CC/PP and UAProf, can be similarly implemented by defining corresponding rules in XSLT.

Content Adaptor Implementations As described earlier, prior to any content request from the user and at the framework start-up, the available ADMEs are discovered by the CA and are made known to the AM. Once the CA has received an adaptation decision from the AM, it refines the decision and then formulates a practical adaptation plan containing a series of Adaptation Operations. A series of ADMEs will then be selected from ADME Profile Repository with each ADME for one Adaptation Operation in the formulated plan. In each step of the plan execution, the content is transferred to the location of the designated ADME and that ADME is invoked. When the last ADME is finished, the content is sent back to the original content server or any intermediate servers when appropriate and the URL is sent to the request originator for user’s retrieval. In reality, it is very unlikely for one single provider to provide all the ADMEs that may be required by ubiquitous multimedia services due to the heterogeneous nature of multimedia. An adaptation process may be composed of multiple steps of Adaptation Operations with each step involving an ADME that may come from different providers. Dynamic discovery, selection, composition and invocation of those ADMEs are the tasks of CA. Meanwhile, there are hardly any comprehensive ADMEs providers exposing their services in a way that favours such a dynamic process. For example open-source audio and video transcoding software FFMPEG17 is a command line tool to convert multimedia files between formats,

160

and image conversion software ImageMagick18 is typically utilised from the command line too. Though the latter can be accessed via a variety of program languages including C++, Java, PHP etc., they still lack the capabilities of exposing them for dynamic discovery, in particular, over the Internet. Web Services emerge as the de facto standard for the provision of services over the Internet. Therefore, in our implementation, we have re-written part of the FFMPEG library as ADME candidates and hosted them on Apache Tomcat server for proof-of-concept purpose. The ADME web services are described using Web Services Description Language (WSDL). We enhance them with descriptions of inputs, outputs, preconditions and effects (IOPEs) using OWL-S and thus enable the selection of successive ADMEs by matching the input of the current candidate service with the output of the previous chosen one. Bubble Sorting Algorithm is also implemented to select the least expensive or most efficient ADME combinations according to user’s preferences. This intrinsic execution of chosen ADMEs has been achieved with the help of the Apache Web Service Invocation Framework (WSIF)19 Similar process can be repeated for all of the FFMPEG transcoders and a FFMPEG Web Service repository can be established as a result.

AMF Adaptation Process We have addressed the advantages of using Web Services in handling heterogeneity when providing services over networks. Providing such capability is crucial to assure the AMF’s interoperability with existing and future systems. We develop the AMF as a whole, which consists of AM and CA, as a Web Service accessible to any user who wishes to utilise adaptation services. The user can be a content provider who wants his content to reach a wider range of devices, a network operator who wants to provide ubiquitous multimedia experience to his customers when the content provider is of no help, or an end user if neither

A Knowledge-Based Multimedia Adaptation Management Framework for Ubiquitous Services

content provider nor network operator is doing that for him. The ADMEs, though centrally managed by the CA, can actually be implemented by respective experts in their specific areas, such as language translation, video conversion etc. and plugged into the framework provided they are in the form of Web Service. The AMF runs on an Apache Tomcat server and the ADMEs also run on an Apache Tomcat server but different from the one hosting the AMF. This is to prove the concept that ADMEs can be provided over the Internet separately in a flexible manner as services. This is based on the consideration that new content formats emerge regularly and supports for these new formats especially their adaptations generally require expert knowledge and thus special services. When adaptation services for these new formats become available somewhere (by someone) in the future, they can be discovered by the AMF and be used to get content adapted to and from these new formats when needed. The AMF exposes itself to the outside world via a gateway component. The gateway creates the Web Service interface to receive requests from Web Service clients and the message translator to read the clients’ requests into adaptation requests and vice-versa write adaptation results to clients’ responses. As to the interactions of the AM with the CA, the AM creates the ADMEs discovery engine and triggers ADMEs’ discovery. The information of the discovered ADMEs is returned to the AM to associate ADMEs with supported Adaptation Operations. The AMF can take two types of request, inquire request and adapt request. The purpose of inquire request is to give user the chance to know more information about adaptation services before actually being processed to the actual adaptation, such information includes the service availability, the service cost and a rough guidance of how long it takes for the service to get the job done. The AMF can handle adapt request with or without a prior inquire request. Note that an instance of the AMF Web Service is created and initialization of the AMF is then

triggered only when the first request is received. The initialization tasks are executed only once and might result in a rather longer response time for the first request. For this reason, when deploying the AMF, it is advised to send a dummy request (or a start-up request) to start the AMF and get the initialization task executed prior to any actual use. The initialization tasks are described as follows: 1. Loading Adaptation Ontology as described in Section 2. 2. Loading context formatters and extractors as described in Section 3.1. 3. Triggering ADME discovery: at start-up, the AM triggers the CA to discover possible ADMEs. The list of available ADMEs is maintained by the CA. 4. L o a d i n g s u p p o r t e d A d a p t a t i o n Operations: all Adaptation Operations that are supported by the AMF and used in adaptation rules to make adaptation decisions are described into a XML file that is loaded at system start up. The file contains information about each Adaptation Operation needs or supports, such as language, modality, format of both input and output as well as other parameters such as bit-rate, resolution etc. 5. Maintaining a list of active Adaptation Operations: as said, one Adaptation Operation can have many versions of concrete implementations called ADMEs though the ADMEs may vary in their input/output parameters, such as bit-rate, frame-rate etc. In our implementation, an active Adaptation Operation is an operation of which an ADME implementation has been discovered. If an ADME is discovered, the AM checks whether an active operation corresponding to that ADME exists in the active Adaptation Operation list. If an active operation exists, its input and output parameters are updated according to the discovered ADME input and output capabilities. Otherwise, if a cor-

161

A Knowledge-Based Multimedia Adaptation Management Framework for Ubiquitous Services

responding operation is found in the inactive operations list which contains operations that are not actively used in current process, an instance of the operation is copied into the active operation list, and its input and output parameters are updated according to what the ADME supports. If a corresponding adaptation operation is found neither in the active operation list nor in the inactive operation list, then the ADME is not supported, it is logged for a developer to examine and consider adding its corresponding operation to the XML file containing supported Adaptation Operations. ADMEs and Adaptation Operations are matched by comparing two strings formed by combining their input-modality, input-format, inputlanguage, output-modality, output-format, and output-language. 6. Loading Reasoner: at start up, an instance of a DL Reasoner is created. The AM uses an instance of the Pellet Reasoner. Both the Adaptation Ontology and the Rules Ontology, as described in Section 3.1 which contains defined rules to be used when making adaptation decisions, are loaded and classified to reduce reasoning response time for reasoning requests. When running the AMF, the Reasoner shall be invoked by the Context Provider to refine context profiles, and the Adaptation Decision Engine to run DL-based rules. 7. Preparation for handling request: in our implementation, we associated each user request with an adaptation request, an adaptation session, and an adaptation process object. The adaptation session holds all requests from a client to allow multiple and recursive requests per user, an adaptation process is created to invoke AM functions to process the request. At start-up, the AM creates facilities to manage such requests, sessions and processes.

162

DEMOSTRATION OF THE AMF The AMF, once deployed, serves as a service enabler for ubiquitous services and is not expected to have any visible interfaces. In ubiquitous services, apart from the complexities being managed by the AMF proposed here, there are some other userspecific complexities, such as user context detection. Those complexities are particularly looked into by WP1: user perspective of the “ubiquitous services” program (MVCE-U, 2005) in which the work presented here belongs to WP3: content/service perspective. The WP1: user perspective of the work aims to develop a Personal Assistant Agent (PAA) (Bush et al., 2006) which manages user’s context, including definition, detection, discovery and modeling etc., on user’s behalf. The PAA is developed on Agent platform JADE20 to cater for the distributed and dynamic characteristics of the user environment. It can not communicate with the AMF Web Services directly. Thankfully, it is bridgeable with the help of a Web Services Integration Gateway (WSIG)21 whose typical function is to translate Agent Communication Language (ACL) in Agent Platform to Simple Object Access Protocol (SOAP), a Web Service standard communication language, and vice versa. This bridging mechanism, together with some other functionality, such as negotiating among many AMFs if there are more than one AMF services available to the user, has been implemented in an entity called Dispatcher (Tarus et al., 2007). While Dispatcher is designed to bridge the communication between the AMF and user’s environment, it has extended to the investigation of mediation and negotiation strategies among distributed agents as well as agent management over distributed network. The detailed work of Dispatcher is given in (Li et al., 2010).

A Knowledge-Based Multimedia Adaptation Management Framework for Ubiquitous Services

Adaptation Scenarios Though, in our context, the PAA is indispensable to enable a completely automated adaptation process without user’s explicit involvement or even notice, it is beyond the scope of the work being focused in this chapter. To demonstrate the AMF, a PAA user interface is particularly designed without its full-fledged functionality. It also has no intention to reflect the PAA’s ultimate look. In this case, the expectant output of the PAA, which contains user’s request together with relevant context information and works as a trigger to the AMF, has to be deliberately fed. Therefore, we define three scenarios and simplify them into context-related terms as small screen context, voice only context and no video/voice context scenario to show the Video-to-Video, Video to Audio and Video to Image capabilities of the AMF respectively. Such scenarios are chosen because they have been seen as typical scenarios of ubiquitous multimedia services, and they provide good perspectives to demonstrate the capability of the AMF. The scenarios are extracted from the following storyboard. Dave, an avid football fan, starts his day at home early in the morning. His schedules for the day involve going to the office for a meeting and to a stadium to watch a match later in the day. To demonstrate the AMF, the scenarios start when the football match starts.

Scenario 1: Small Screen Context (Video to Video Adaptation) Dave’s meeting in the office runs longer than expected. However, at the stadium the game has already begun. Not the one to miss his favorite team’s play, he sets his Nokia N70 to receive video content (by installing PAA) from the game’s web portal. Since the content is meant for large screens but small ones like his Nokia N70, AMF is then called by his PAA to adapt video content to a suitable size and format for his Nokia N70.

Scenario 2: Voice Only Context (Video to Audio adaptation) Dave dashes to his car to take the 20 minutes drive to the football stadium. As he drives, and since he needs to concentrate, he instructs his PAA to request for a video to audio adaptation of the game so that he can still listen to the game.

Scenario 3: No Video/Voice Context (Video to Image Adaptation) Meanwhile, Dave has set his PAA to continually capture important scenes of the football game for later review and archiving. Because of the small memory capacity of the phone combined with the low battery life, Dave’s PAA decides to capture only images of the important scenes of the football match. This images and some other contextual information such as time of capture give Dave a precise picture of the match progression later. His PAA, after recognizing a goal or something else interesting, sets to automatically capture that and then asks the AMF to carry out a video to image adaptation

Illustrations A number of screenshots are shown in Figure 6 to illustrate the adaptation process of the first scenario described in above section. We can see on the first screen that the user’s device is equipped with the PAA agent. In the first Video-to-Video adaptation scenario, the small screen context is selected and user’s preferences towards cost and speed performance are all taken into account. User then can choose inquire to get more information about the AMF or AMFs available. In this case, he got responses from two AMFs with one saying “can adapt for £0.54 in 22ms” and another saying “cannot adapt”. When the user goes ahead with the first AMF by clicking adapt, the actual adaptation operations are triggered and the URL of the adapted content is returned, as

163

A Knowledge-Based Multimedia Adaptation Management Framework for Ubiquitous Services

Figure 6. Screen-shots of demonstration process

shown in the 5th shot. The user can then retrieve the video which now is playable on his device, captured in the 6th shot.

DISCUSSIONS AND FUTURE DIRECTIONS This chapter has its particular focus on describing the design and development of an Adaptation Management Framework and investigates issues related to the management of dynamic content adaptation in ubiquitous environment. The work described in this chapter looked into challenges

164

from one perspective, i.e. content/service perspective, envisioned by the program “removing the barriers to ubiquitous services”22 which was initiated by the Mobile Virtual Center of Excellence (MVCE)23 and collated the views and visions from network operators, content providers and device manufactures. There are two other major perspectives envisioned by this programme, that is User Perspective and Network Perspective, which also require extensive research efforts repectively. Each perspective has a centric view of the challenges and problems in their respective domain. From the User Perspective, the challenges are to manage the dynamically configured devices in a way

A Knowledge-Based Multimedia Adaptation Management Framework for Ubiquitous Services

that they work as a whole to provide users with ubiquitous service experience. From the Network Perspective, the focuses are mobility, security and Quality of Service (QoS) management across heterogeneous networks. More importantly when they are integrated, their individual performances are not discounted. The ubiquitous services vision can not be easily realised without close collaborations among these three perspectives, particularly when best performance is sought. For example, the delivery context undoubtedly comes from user’s environment as well as network providers which are specifically looked at from the User Perspective and Network Perspective respectively. Vice versa, having the adaptation system in place to adapt content, for example converting video to audio, prior to its delivery to the user will help improving the network performances and user device’s accessibility. There are some works in literature which aim to tackle all challenges and pursuit a holistic context-aware service adaptation and delivery solution or architecture. However, they are under a variety of themes other than ubiquitous services, such as smart homes (Gu et al., 2004; Yu et al., 2006), pervasive computing (Ranganathan & Campbell, 2003), and therefore did not provide the same level of focus and depth as what has been described here, particularly on using knowledge manipulating method to approach adaptation of multimedia. Moreover, incorporating knowledge, such as availability, capability etc., of adaptation mechanisms, which carry out the actual adaptation, into adaptation decision making process has not been investigated in the context of ubiquitous services. The first significant deployment of dynamic adaptation mechanism into major service delivery network was achieved by Vodafone UK24 when they launched Mobile Internet in 2007. It was deemed to bring the richness of the PC-internet to mobile phones. The core technology deployed is a Web Translation Engine (WTE)25 placed in its core network between content providers and

end users to carry out dynamic adaptation when needed. This is a significant milestone in the direction towards ubiquitous services in reality though it is under the theme of “Mobile Internet” and achieves only a small portion of targets envisioned for ubiquitous services. Here, we summarise a few of, but not limited to those targets, some of which have been addressed in this chapter while others remain as major challenges and require further research efforts and close collabrations among various deciplines. First of all, in terms of adaptation targets, web contents or contents delivered via the web are not the only contents that can be targeted for adaptation though they are the most obvious ones. Other types of contents, such as multimedia as well as their delivery mechanisms, such as streaming in real time, should also be considered because they have played and continue to play very import roles in the way users access services. Secondly, in terms of access networks and devices, a large percentage of end users are often mobile from one location to another and switching from one network to another such as from a home area network to a vehicle area network. As a consequence, the devices a user can access also change with some being out of reach while others becoming available. One of most important features of ubiquitous services is to autonomously select the optimal and seamless handover among the available access networks and devices for uninterrupted service while maintaining the same or improving user experience. Thirdly, with regards to context accquision and representation, there are many types of context that can be considered to guide the content adaptation. The context can be accquired via devices or sensors, such as video camera, GPS etc., GUIs for user to input inforation like preferences, schedules etc., or discovery protocals such as UPnP (Universal Plug and Play) to discover device hardware and software capabilities. These information are hetergenous in both syntax and semantics and an effective modelling and representation machanism can greatly facilitate the processing of such information and help

165

A Knowledge-Based Multimedia Adaptation Management Framework for Ubiquitous Services

their interactions with any applications or services that depend upon them. Recent research progress in enhancing Sensor Netowork with semantic descriptions (Huang & Javed, 2008 etc.), Semantic Sensor Web (Shesh et al., 2008), Smart Products leveraging on Semantic Web technologies (Sabou et al., 2009), if applied, could propel the further development of this field. Finally, but by no means the last, semantic modelling of web services, to enable their automatic discovery, composition and orchestration, has been a research hotspot in recent years (Pedrinaci et al., 2010 etc.). In the context of content adaption in ubiquitous environment, such a requirement arises from adapation mechanisms which carry out the actual content adaptation. We have manually annotated a number of mechanisms in this work for proof-of-concept purpose. Such a process can become automatic when semantic annotation tools for web services, such as the one in (Maleshkova et al., 2010), become available.

CONCLUSION This chapter presents the design of a content/service Adaptation Management Framework (AMF) capable to manage the complexity of dynamic and autonomous content/service adaptation processes. The AMF serves as an invisible service enabler in content/service delivery network that provides the user and content/service provider the flexibility to dynamically manage contents across multiple devices and heterogeneous networks. The implementation details of our demonstrator are given and do show the validity of the concepts. Deployment scenarios of the AMF are described and an example of how to interact with the AMF is given. The deployment of a service-oriented design for the AMF assures the interoperability and extensibility of this work with existing and future service provision/delivery models and standardised works in respective domains, and thus increases its chances of being widely de-

166

ployed by different players in the content/service delivery chain. The AMF has been defined and a showcase demonstrator has been implemented, the demonstrator has shown the feasibility of automatic adaptation of contents without adding complexity to the usability of services and devices, nor loading the content providers with having to provide their contents in many different media formats. For the demonstrator implementation, requests and accompanying context were manually done. The steps and functions involved in the formation of a request include real-time context sensing and reasoning, embedding the PAA’s functionality into a user’s normal multimedia browsers or applications etc.

ACKNOWLEDGMENT The work reported in this chapter has formed part of the Ubiquitous Services Core Research Programme of the Virtual Centre of Excellence in Mobile & Personal Communications, Mobile VCE, www.mobilevce.com. This research has been funded by the Industrial Companies who are Members of Mobile VCE, with additional financial support from the UK Government’s Technology Strategy Board (previously DTI).

REFERENCES Bush, J., Irvine, J., & Dunlop, J. (2006). Removing the Barriers to Ubiquitous Services: A User Perspective. In Proceedings of the 3rd Annual International Conference on Mobile and Ubiquitous Systems - Workshops, 1-5. Garcia, R., & Celma, O. (2005). Semantic Integration and Retrieval of Multimedia Metadata. In Proceedings of the 5th International Workshop on Knowledge Mark-up and Semantic, Annotation at the 4th International Semantic Web Conference.

A Knowledge-Based Multimedia Adaptation Management Framework for Ubiquitous Services

Gu, T., Pung, H. K., & Zhang, D. (2004). Towards an OSGi-Based Infrastructure for Context-Aware Applications in Smart Homes. IEEE Pervasive Computing / IEEE Computer Society [and] IEEE Communications Society, 3(4), 66–74. doi:10.1109/MPRV.2004.19 Hanrahan, R., & Merrick, R. (2004). Authoring Techniques for Device Independence. W3C Working Group Note, http://www.w3.org/TR/2004/ NOTE-di-atdi-20040218/. Huang, V., & Javed, M. K. (2008). Semantic Sensor Information Description and Processing. In Proceedings of the 2nd International Conference on Sensor Technologies and Applications. Hunter, J. (2001). Adding Multimedia to the Semantic Web-building and MPEG-7 Ontology. In Proceedings of the 1st International Semantic Web Working Symposium. Imai, M., Hirota, Y., Satake, S., & Kawashima, H. (2006). Semantic Sensor Network for Physically Grounded Applications. In Proceedings of the 9th International Conference on Control, Automation, Robotics and Vision, (ICARCV’06), pp. 1–6. IST. (2004). 37 COMMITTEE Information Technology — Multimedia Framework (MPEG21) — Part 7: Digital Item Adaptation (pp. 21000–21007). ISO/IEC. Jannach, D., Leopold, K., Timmerer, C., & Hellwagner, H. (2006). Knowledge-based Framework for Multimedia Adaptation. Applied Intelligence, The Inter. Journal of Artificial Intelligence, 24(2), Special Issue on Innovations in Applied Artificial Intelligence, 109-125, Springer. Li, N., Attou, A., & Moessner, K. (2007). A MPEGbased Ontology for Ubiquitous Content/Service Adaptation. In Proceedings Of the 4th Context Awareness for Proactive Systems (CAPS) conf., Guildford, UK.

Li, N., & Moessner, K. (2007). The MVCE Management Framework for Context-aware Content and Service Adaptation. In Proceedings of the 7th International Workshop on Applications and Services in Wireless Networks, Spain. Li, N., Tarus, H., Moessner, K., & Irvine, J. (2010). (in press). A Communication Middleware for Ubiquitous Multimedia Adaptation Services. Journal of Computing and Informatics, 29(4). Maleshkova, M., Pedrinaci, C., & Domingue, J. (2010). Semantic Annotation of Web APIs with SWEET. In Workshop: 6th Workshop on Scripting and Development for the Semantic Web at 7th Extended Semantic Web Conference. Martinez, J. M. (Ed.). (2004). MPEG-7 Overview (version 10). International Organization for Standardization ISO/IEC JTC1/SC29/WG11 N6828. MVCE-U COMMITTEE. (2005). Removing the Barriers to Ubiquitous Services. Mobile VCE Core 4 Programmehttp://www.mobilevce.com/frames. htm?core4frame.htm. Pedrinaci, C., Liu, D., Maleshkova, M., Lambert, D., Kopecky, J., & Domingue, J. (2010). iServe: a Linked Services Publishing Platform. In Workshop: Ontology Repositories and Editors for the Semantic Web at 7th Extended Semantic Web Conference. Ranganathan, A., & Campbell, R. H. (2003). A Middleware for Context-Aware Agents in Ubiquitous Computing Environments. In ACM/IFIP/ Usenix Int’l Middleware Conf., Springer-Verlag. Sabou, M., Kantorovitch, J., Nikolov, A., Tokmakoff, A., Zhou, X., & Motta, E. (2009) Position Paper on Realizing Smart Products: Challenges for Semantic Web Technologies. In Workshop: The 2nd International Workshop on Semantic Sensor Networks

167

A Knowledge-Based Multimedia Adaptation Management Framework for Ubiquitous Services

Sheth, A., Henson, C., & Sahoo, S. (2008). Semantic Sensor Web. IEEE Internet Computing, 12(4), 78–83. doi:10.1109/MIC.2008.87

De, S., & Moessner, K. (2009). Rule-based Reasoning Mechsanims for Context-aware Service Presentation. eChallenges 2009, Istanbul, Turkey.

Soetens, P., Geyter, M. D., & Decneut, S. (2004). Multi-step Media Adaptation with Semantic Web Services. In Proceedings of the 3rd International Semantic Web conference (ISWC).

Kopecký, J., Vitvar, T., Bournez, C., & Farrell, J. (2007). SAWSDL: Semantic An- notations for WSDL and XML Schema. IEEE Internet Computing, 11(6), 60–67. doi:10.1109/MIC.2007.134

Tarus, H., Bush, J., Irvine, J., & Dunlop, J. (2007). Ubiquitous Entity Interactions and Dispatcher Architecture. In Proceedings of the 66th IEEE Vehicular Technology Conference (VCT) Fall, 106-110.

Stamou, G., van Ossenbruggen, J., Pan, J. Z., Schreiber, G., & Smith, J. R. (2006). Multimedia annotations on the semantic Web. IEEE MultiMedia, 13, 86–90. doi:10.1109/MMUL.2006.15

Tsinaraki, C., Polydoros, P., Moumoutzis, N., & Christodoulakis, S. (2004). Coupling OWL with MPEG-7 and TV-Anytime for Domain-specific Multimedia Information Integration and Retrieval. In Proceedings of the RIAO, 783-792. Yu, Z., Zhang, D., Zhou, X., Chin, C., & Yu, Z. (2006). An OSGi-Based Infrastructure for Context-Aware Multimedia Services. IEEE Communications Magazine, 44(10), 136–142. doi:10.1109/MCOM.2006.1710425

ADDITIONAL READING Attou, A., & Moessner, K. (2008) Context for Multimedia Service Adaptation, In Proceedings of the 4th International Mobile Multimedia Communications Conference, Oulu, Finland. De, S., Meissner, S., Kernchen, R., & Moessner, K., A Semantic Device and Service Description Framework for Ubiquitous Environments, ICT Mobile Summit, Stockholm, Sweden. De, S., & Moessner, K. (2008). Ontology-based Context Inference and Query for Mobile Devices. In Proceedings of the 19th Annual IEEE International Symposium on Personal, Indoor and Mobile Radio, Cannes, France.

168

Tsinaraki, C. (2007). Interoperability support between MPEG-7MPEG-7 /21 and OWL in DSMIRF. Knowledge and Data Engineering. IEEE Transactions on, 19, 219–232. Yu, Z., Zhou, X., Zhang, D., Chin, C., Wang, X., & Men, J. (2006). Supporting Context-Aware Media Recommendations for Smart Phones. IEEE Pervasive Computing / IEEE Computer Society [and] IEEE Communications Society, 5(3), 68–75. doi:10.1109/MPRV.2006.61 Zhdanova, A., Li, N., & Moessner, K. (2008). Semantic Web in Ubiquitous Mobile Communications, in eds. Ma, Z., The Semantic Web for Knowledge and Data Management. Hershey, PA: IGI Global

KEY TERMS AND DEFINITIONS Context-Aware: The idea that computers can both sense, and react based on their environment. MPEG-21 DIA: The Part 7: Digital Item Adaptation, of MPEG-21 multimedia framework which overall aims at defining a normative open framework for multimedia delivery and consumption for use by all the players in the delivery and consumption chain. MPEG-7: A multimedia content description standard. It was standardized in ISO/IEC 15938 (Multimedia content description interface).

A Knowledge-Based Multimedia Adaptation Management Framework for Ubiquitous Services

Multimedia: Media and content that uses a combination of different content forms, such as text, image, video, audio etc. Ontology: A formal specification of a shared conceptualization. Semantic Web: Methods and technologies to allow machines to understand the meaning, i.e. semantics, of information on the World Wide Web. Ubiquitous Services: The delivery of a mix of contents and services across a mix of heterogeneous network and device technologies as well as administrative domains. Web Services: Application programming interfaces (API) or Web APIs that are accessed via Hypertext Transfer Protocol (HTTP) and executed on a remote system that hosts the requested services.

6

ENDNOTES

19

1 2



4 3

5



CC/PP: http://www.w3.org/Mobile/CCPP/ UAProf: http://www.wapforum.org/profiles/ UAPROF/ccppschema-20000405 WSDL: http://www.w3.org/TR/wsdl WSDL-S: http://www.w3.org/Submission/ WSDL-S/ OWL-S: http://www.daml.org/services/



9 7 8

10 11



12



13



16 17 18 14 15

21 20

22



25 23 24

SA-WSDL: http://www.w3.org/2002/ws/ sawsdl/ OWL: http://www.w3.org/2004/OWL/ XSLT: http://www.w3.org/TR/xslt DIG: http://dl-web.man.ac.uk/dig/2003/02/ interface.pdf Fact: http://dl-web.man.ac.uk/dig/fact.shtml Pellet: http://www.mindswap.org/2003/pellet/ OWL 1.1: http://www.w3.org/Submission/ owl11-overview/ SWRL: http://www.w3.org/Submission/ SWRL/ JESS: http://www.jessrules.com/ http://en.wikipedia.org/wiki/Bubble_sort http://mpeg7audioenc.sourceforge.net/ FFMPEG: http://ffmpeg.sourceforge.net ImageMagick: http://www.imagemagick. org WSIF: http://ws.apache.org/wsif/ JADE: http://jade.tilab.com/ WSIG: http://jade.tilab.com/doc/tutorials/ WSIG_Guide.pdf http://www.mobilevce.com/frames. htm?core4frame.htm http://www.mobilevce.com http://www.vodafone.co.uk http://www.betavine.net/bvportal/resources/ vodafone/mics

169

Section 3

Novel User-Interfaces, Emerging Forms of Interaction and Media Theories

171

Chapter 9

Interactive Visualization and Exploration of Video Spaces through Colors in Motion Teresa Chambel University of Lisbon, Portugal João Martinho University of Lisbon, Portugal

ABSTRACT Video is nowadays largely used in everyday life. It is becoming pervasive in richer and broader widely accessed media spaces, and is also a very rich medium by itself, including pictures, text and audio that change in time. This richness makes these media spaces very interesting, allowing the communication of huge amounts of information and an excellent means for creativity to be expressed and explored, but it comes with a challenging complexity to handle. Visualization techniques can help to handle the complexity and to express the richness in these information spaces. This chapter identifies challenges and concepts inherent to the visualization and exploration of video spaces and presents an approach through ColorsInMotion, an interactive environment to process, visualize and explore a video space with a semantic focus on cultural aspects like music and dance, and stressing features such as their color dominance, rhythm and movement, at the level of the video space and the individual videos. It provides means to capture, experience, and express videos’ properties and relations, allowing to gain insights into our culture and to influence the expression of its intrinsic aesthetics in creative ways. DOI: 10.4018/978-1-60960-774-6.ch009

Copyright © 2012, IGI Global. Copying or distributing in print or electronic forms without written permission of IGI Global is prohibited.

Interactive Visualization and Exploration of Video Spaces through Colors in Motion

INTRODUCTION We have been witnessing a rapid explosion of video data in richer and broader widely accessed media spaces. It is becoming very easy to access video in a digital manner in private and public scenarios, due to the propagation of digital cameras, high bandwidth Internet connections and video sharing websites. Furthermore, video is a very rich medium combining image and sound, thus providing huge amounts of information and an excellent platform for creativity to be expressed and explored. The pioneers of the Video Art, for instance, have been exploring this creativity, using installation and performance art in conjunction with video in order to create an immersive experience for the spectator. However, all the richness that makes video based information spaces so interesting, inside each video and outside in the information realm where in many ways they relate to each other, comes with a challenging complexity to handle. One of the problems is the fact that video is not structured data-wise, and so, accessing all the data that a video can provide is often not an easy task. Semantic descriptors can be used to tag some information of the video: by the users, a common approach in video repositories like YouTube; or through information segmentation and understanding, a more complex task. Low level data, like color and shot information, duration of the video, and scene information, provide additional information that can be retrieved by calculations and time consuming tasks. But once this information is collected, we can try to use it to our advantage, for a better organization of the individual and collective video spaces, to search, to assist with the process of editing videos and even to provide new forms of visualization and interaction. Visualization techniques (Card et al, 1999; Tufte, 2006) can actually help to handle the complexity and express the richness in these information spaces. Video visualization can be an intuitive and effective way to convey meaningful

172

information in video. However, there is still a lack of effective techniques to convey complex information intuitively through automatic video processing. An alternative approach is to provide an overview by extracting interesting information and present it in a meaningful way (Daniel & Chen, 2003). Summarization for e.g. allows to visualize video in concise ways, based on properties like movement, rhythm or scene change. Also in the Video Art movement, some of the works make use of visualization - as a tool to convey some kind of meaning to the viewer. Advances in data visualization have emerged from research rooted primarily on visual perception and cognition. Trends are to evolve towards scenarios where tools become invisible while we become immersed in the exploration of data (Few, 2007). In our approach, we explore visualization techniques and interaction modalities in the direction of making the user immersed in the video spaces, while stressing significant properties. This chapter identifies challenges and concepts inherent to the visualization and explorations of video spaces and presents an approach through ColorsInMotion, an interactive environment to support the interactive and creative visualization and exploration of videos with a strong emphasis on color and motion properties, at the crossroads of information access, culture and digital art. It provides video analysis support along with the means to help with the visualization and browsing of the collective video space and the individual videos, stressing features such as their color dominance, rhythm and movement. Interaction can be done through a traditional setting based on a screen, keyboard and mouse, or a touch screen, but we are also developing more natural and deviceless approaches, especially adequate for installation settings and ambient interaction contexts. This work is based on our earlier work: “ColorsInMotion: Interactive Visualization and Exploration of Video Spaces”, in (Martinho & Chambel, 2009) © ACM, 2009.

Interactive Visualization and Exploration of Video Spaces through Colors in Motion

Next section makes a review of most relevant background. Then, ColorsInMotion is presented based on the description of its two modules, the VideoAnalyzer and the Viewer. The chapter ends with a discussion of future research directions and some concluding remarks.

BACKGROUND Through the website The Best Tools for Visualization (Perez, 2008), many visualization tools and applications can be found. Most of which are used to visualize social networks and the internet, a few to visualize music, the Amazon and Flickr. Some of them are simply intended for artwork, others for performing exploratory data analysis. But most of them do not address video visualization. Exceptions include the visualization of videos in YouTube and Videosphere (Bestiario, 2008). From each video on YouTube, the user can access a 2D view that represents videos as circular scattered still images, giving access to the traditional page to watch the video. It allows for visual neighborhood navigation, but provides limited functionality and information about the videos or the video space. Videosphere (Bestiario, 2008) represents a video space around a 3D sphere, with links among the videos, reflecting semantic compatibility, and allowing navigating around and inside and out the sphere. The visualization is restricted to the videos on that sphere, with the focus on exploring semantic relations, and no special support for the visualization of the videos other then still keyframes and traditional video play. Some research works explore new ways to view an individual video. In (Fels & Mase, 1999), the authors show a tridimensional view of the video by considering the video data to be a volume, in which the third dimension is the time, which is similar to what Daniel and Chen present in (2003). This kind of three-dimensional visualization provides the user with a way to view the evolution of a given area of the video throughout its duration

by using the traditional x and y-axis as the width and height of the frame and the z-axis as the time, giving depth to the volume. Slit scan imaging techniques capture time-based phenomena into static images, being an interesting candidate to represent videos. (Levin et al, 2005-2008) collects information about slit scan video artwork. Timeline (Nunes et al., 2007) adopts slit scanning to allow easy and rapid exploration of a video history to view participants in collaborative scenarios. In (Boreczky et al., 2000) the authors make use of a comic book like presentation in order to summarize an individual video. The frames are clustered using the hierarchical agglomerative clustering technique in order to produce segments that can be represented by a keyframe, which is then presented. Comparable to this visualization, in (Irani et al., 1995), the authors make use of the mosaic, or tile, based representation in order to present significant frames in a video sequence. This representation can be used to make a visual summarization of the video. In previous work, we explored this among other types of representation to index video and help the navigation in hypervideo (Chambel & Guimarães, 2002). Other related works like (Finkelstein & Range, 1998)0 or (Klein et al., 2002) use searching and content-based image retrieval techniques in order to do the matching process necessary to create video mosaics, as an arrangement of small videos that suggests a larger unified video sequence. In (Martin, 2004), the author used some of these techniques to try and recreate an entire video with mosaics. Mosaic based works always make use of color histograms and diverse techniques of image analysis, in order to search and retrieve images that are similar to the regions they are going to replace in the mosaic. In previous work (Rocha & Chambel, 2008), we provided interactive 3D visualization and navigation of video, as an art installation, to explore cultural properties and links among different videos: at the global level where a collection of videos represented by loops was organized by semantic

173

Interactive Visualization and Exploration of Video Spaces through Colors in Motion

categories like countries, themes and authors; and at the micro and more syntactic level of color and brightness of each video, experienced when the user was watching and interacting with the 3D representation of each video while it was playing. With the aim to help video editing, other works explore different ways to browse a collection of videos. Among these, we find (Girgensohn et al., 2000), where the Hitchcock editing system forms stacks of videos organized by color histogram similarity, providing a visually perceptible way of organizing large collections of videos. In order to have sufficient data to perform this kind of organization, there are also some works that focus on analyzing the images and the data contained in videos, like (Kerminen & Gabbouj, 1999), where the authors designed a system to retrieve images based on their color histogram. In order to create a histogram they analyze the images pixel by pixel and quantize the color space. In (Saykol et al., 2004) the authors also use a similar technique in their work to search images based on color, shape and textures, and especially the color space quantization required to create the color histograms. The Multicolr Search Lab (Idée, 2008) finds images based on selected colors. Users may select up to ten colors, and may repeat the ones they want to make more prominent. They extract the colors from around three million pre-selected photos, the ones they find more interesting. So far, it is available for Flickr and Alamy Stock Photography. This type of photo sharing sites usually allows searching mainly based on tags, titles and keywords, not colors. The same is true for video sharing sites like YouTube. Creativity is also an important aspect in our work. There are a number of projects that use different and original approaches to visualization and organization of videos and other types of information. For instance, in (Sims, 1991), Sims used genetic algorithms and the evolution theory in order to create graphics. In a related perspective, we also explored creative ways of

174

editing videos, using evolutionary algorithms in our previous work (Chambel et al., 2007). New video sequences were combined and selected, based on their characteristics represented as video annotations, either by defining criteria or by interactively performing selections in the evolving population of video clips. One of the objectives included exploring and discovering architectonic and cultural relations in videos from Brazil and Portugal, but we felt then the need for richer and more flexible ways to visualize and navigate video spaces. Soft Cinema (Manovich & Kratky, 2010) is a dynamic computer-driven media installation, based on an ambient narrative approach addressing ways to represent the subjective experience of a person living in a global information society. The viewers are presented with an infinite series of narrative films constructed on the fly, using author defined rules and a media database containing a few hours of video, animation, voice over narration, and music. In the area of Video Jockeying (VJing) (Makela, 2006), performance artists create moving visual art, usually based on video on large displays, at events such as concerts, nightclubs, sometimes in conjunction with other performance art. Computational support for creative editing and visualization of video provide tools for these artists. Creative processes usually have some degree of randomness and uncertainty - as Pepperell in (Pepperell, 2002) states, creativity is a response by the organism to the unpredictability of the environment. Taking this into account, most of the previous approaches try to apply the metaphor of the dynamic and evolutionary system to a collection of videos or graphics and, by allowing the combination of their intrinsic properties and rules with the user’s interaction and context, aim to induce and support creative processes in innovative ways.

Interactive Visualization and Exploration of Video Spaces through Colors in Motion

Figure 1. ColorsInMotion architecture

COLORS IN MOTION The interactive application ColorsInMotion was designed and developed to visualize and explore video spaces, based mainly on videos’ color and motion properties. It has two main components: the Video Analyzer, to process videos in order to extract their main properties and create video views based on these properties; and the Viewer, to visualize and interact with video spaces at global and individual video levels through different perspectives. ColorsInMotion architecture (Figure 1) is based on two separate modules that share information about the videos. Videos can enter the system from video files or a webcam through Video Analyzer. Video information extracted by this module along with the created video views are then accessed by the Viewer, where users can navigate, search and interact with the video space, through more traditional interfaces using a screen, keyboard and mouse, or a touch screen, or more natural and deviceless approaches towards more immersive interactions. ColorsInMotion modules were developed in Processing open source programming language and environment (Reas, 2007). In the next sections, we introduce the main

concepts, options and interaction modalities of ColorsInMotion Video Analyser and Viewer.

COLORS IN MOTION VIDEO ANALYZER Video Analyzer is the module responsible for the analysis of the video files, in order to get some information about its colors and motion, the production of metadata, and some of the visualizations for the videos, creating a base of pre-processed videos for ColorsInMotion. Videos can be accessed from a public repository like YouTube, any location provided by the user, or captured from the real world by a webcam - providing the users with the option to create their own videos on the fly. This section presents Video Analyzer’s main features. Figure 2 exemplifies its interface.

Color Analysis To analyze the colors of the videos, VideoAnalyzer relies mainly on color histograms. The color histogram is created with a number of bins specified by the user. The more the bins, the more individual colors we have - a tradeoff between performance

175

Interactive Visualization and Exploration of Video Spaces through Colors in Motion

Figure 2. Video Analyzer interface: a) full screen; b) zoom into parameters; c) slit scan of another video; d) zoom into color histograms

and the need to have more colors. Histograms are used in particular to find the average color of the entire video, the most dominant colors on the entire video, the most dominant colors in each frame and the percentage of dominance of these colors (little purple square and following charts in top right region of Figure 2a, shown in more detail in d). The evolution of the average and dominant colors on each frame are presented in real time, on a timeline like visualization (first two horizontal stripes in Figure 2), as the video is playing (top left corner in Figure 2) and analyzed. These are some of the metrics that are used to index the collection of videos as well as to produce some of the visualizations of the videos in ColorsInMotion Viewer. Next subsections discuss in more detail aspects related with these metrics and the color spaces used in the video analysis.

Color Spaces Video Analyzer supports two color spaces: RGB and HSB. In the RGB space, a uniform quantization is adopted, i.e. using the same number of bins (specified by the user) of the same size in each

176

color component, all contributing with the same weight in the perceived color. This is a common approach for dealing with colors, but it does not account well for perception. The color quantization based on the HSB color space, on the other hand, takes human perception into account, and was defined as a non uniform quantization space, to distinguish more colors in the ranges where colors are better perceived. It focuses on the color (hue), saturation and brightness dimensions – easier to master by humans, and allowing taking into account the fact that humans are more sensible to brightness than color. In order to reduce the amount of colors used in the areas where colors are poorly discernible by the human eye, when the Brightness value is too low we count it as black, and if the Saturation value is too low we just use the Brightness value, rounding it to grey, thus having 4 colors on the Brightness scale: black, dark grey, light grey and white. This way, low saturated colors can be well perceived by their brightness (in grey scale), and very dark colors require less distinct classes. We used 18 bins for Hue, which is a 20º interval for each one; 4 bins for Saturation and 4 bins for Brightness.

Interactive Visualization and Exploration of Video Spaces through Colors in Motion

By providing the two color spaces in the Video Analyzer, the user can compare the results of analyzing the same video, and decide for the best results, in terms of perception. In our experiences, HSB did better most of the times in color search, to create color histograms and finding dominant colors. It provides better color space organization and makes easier the task of grouping similar colors when creating color histograms. These results are also aligned with those described in (Kerminen & Gabbouj, 1999; Saykol et al., 2004).

Color Metrics The different color metrics can also be compared, to help to choose best approaches in each situation, dependent on video properties, especially its homogeneity in color dominance. In videos where some colors stand up as dominant at the frame level, the dominant colors by frame count seems to be a good metric, because the most dominant colors at this level are more perceptually accepted as dominant. On heterogeneous videos with many colors, the frame dominance could lose important information - a color could appear very often but not really dominating in individual frames. In this case, by counting every pixel on every frame of the video we have more accurate result of the dominant colors of the video, with some reservations. These include usually having too many colors on display, too close to each other in terms of proportion (%) to call any one of them the most dominant color. The metric that we use in order to perceive if a video has many dominant colors is the percentage of dominance of the most dominant colors. In Figure 2a and 2d, the first chart from the left gives us the dominant colors by counting all the pixels. The results are very similar for a few colors, stating that these are the ones that dominate. Complementing that information, the second chart gives us the most dominant colors by counting them only when they dominate in the frames. Here, the brownish on the left is around

45% dominant, but the third graphic gives us the percentage of dominance of each one of these colors in individual frames, where again it gets a dominance value closer to the other dominant colors, and also closer to our perception of dominance. To allow the selection of good perceptive representations for color dominance, Video Analyzer presents different metrics in a visual form, supporting users’ perceptual expertise in selecting best or complementary choices. We are working towards the tuning of automatic suggestions based on video properties and users’ feedback.

Motion Analysis and Visualization Another significant property in the visualization of a video is its motion. A technique that produces interesting results in the visualization of motion is slit scanning (Levin et al, 2005-2008; Tang et al, 2009), creating static images of time-based phenomena. It takes the middle column of pixels in each frame of the video and concatenates it to an image, so as to show a temporal view of the entire video in one image, capturing video motion, color, and length, since the width of the final image is proportional to the video length. Video Analyzer creates a slit scan for each video it analyzes, in real time. The user can watch the slit scan evolving at the bottom of the interface (Figure 2a and 2c). Slit scans allow the perception of video motion also along the different scenes, identifiable when color patterns change. For example, in Figure 2c, the first scene features a ‘talking head’ making some movements, and the second scene is a pan across a sitting audience. Besides slit scans, another type of image is created to capture motion in video scenes – we call them average scene views. These capture the evolution of the video frames by accumulating the average color of each pixel, in its position, along time (image below the video on the upper left corner in Figure 2, and Figure 9). If the video has a lot of movement, the average image captures it through significant blurring; otherwise a more

177

Interactive Visualization and Exploration of Video Spaces through Colors in Motion

stable image will persist throughout the entire length. Using a dissimilarity threshold to decide when there are radical changes in the successive video frame colors, the analyzer performs a simple form of scene detection on the video. Best results are obtained when there is not too much movement in the scene, resulting in images that resemble impressionist paintings. The user can also adjust the threshold parameter to define the sensitiveness to color change in the detection of scenes, to improve the results. The average scene loop is created with the final average frame of each detected scene.

Video Loops as Visual Summaries VideoAnalyzer performs different forms of image extraction to represent the videos along time. These images are then presented in a loop to provide visual summaries of the videos. Traditional loops only take into account the duration of the video. They are created through the extraction of a desired number of frames on a regular pace. After having analyzed the video and presenting the results, we can extract loops based on color measurements. These include the average color loops, the dominant color loops, and a loop based on a color picked by the user. The average color loop is created comparing the average color of each frame to the average color of the video, and we can control the similarity threshold and number of pictures we want in the loop. The dominant color loop is created comparing the color histogram of each frame to the video color histogram and comparing the dominant colors of each. We can control the number of pictures we want in the loop, the similarity threshold between the colors and the minimum percentage dominance of the dominant color. The user can also pick any color from a color palette or from the charts and video evolution visualizations in order to extract a loop based on that color. It uses the same settings as the dominant color loop.

178

All the loops are saved and referenced to each video for later access. Also, all the metadata information extracted from each video is saved so that we can use it later to search and visualize the video collection. Note that visualizations such as the dominant or average color circles and the dominant color stripes are built by the ColorsInMotion Viewer module, based on this metadata.

Configuring the Video Analysis Before each video analysis, the user can configure some parameters that influence the way the analysis is performed. Main parameters include (Figure 2a and 2b): SPEED controls the speed of the video playing and analysis. If too high, information may be lost when there is not enough time to process all the incoming frames. The default is the normal video playing speed; HISTOGRAM_BINS controls the number of bins used to create the color histograms in the previously selected color model (RGB or HSB); PICTURES refers to the maximum number of images to be included in the video loops; THRESHOLD, the higher its value, the more tolerant the color comparisons in scene detection and in the selection of images for the color loops. Lower values lead to more accuracy and result in fewer selections. The best value for each video, or video segment, depends on the video properties, and should be adjusted accordingly; DOMINATION_PCT refers to the minimum value, in percentage, that a color has to dominate in a frame n order for that frame to be selected for a dominant color loop; NUMBER_COLORS controls the number of colors to be represented in the color histogram charts. Bellow . the user can select the type of color loop to extract: based on a dominant color, average color or chosen color. Through these parameters, the user can control the video analysis, influencing the information extracted and, as a consequence, the created views.

Interactive Visualization and Exploration of Video Spaces through Colors in Motion

COLORS IN MOTION VIEWER

Color Based Views

ColorsInMotion Viewer allows the visualization and interaction with video spaces at global and individual video levels in different perspectives. At the video space level, videos are presented in the form of a physical particle system where a collection of video icons, or views, move on the screen in accordance to their color similarity and user interaction, allowing for video search, comparison, browsing and creative visualization. From this level, the user can select and navigate to each one of these videos, to watch them and to further explore their individual features. In this section, we introduce the main concepts, options and interaction modalities of ColorsInMotion Viewer.

For a stronger emphasis in the colors, videos can be represented by circles filled with their average or dominant color; or by striped rectangles, where the stripes feature video dominant colors with their width reflecting the proportion in which they dominate in the video. These views are presented in Figures 3b and 4a. Color stripes, in Figure 4b, represent the evolution of the average or dominant colors of the videos along time. This stripes are summarized in the video space. When the mouse is over one stripe, the extended version appears, as exemplified for the stripe in the middle of the screen. Like in the other video space views, all these video representations attract or repel each other based on their dominant color similarity.

Color and Motion Views in the Video Space At the Video Space level, the user can cycle through different views, to get an overview of the videos’ contents and their color and motion properties. The following subsections present the main video views, all of which created with the data extracted in the video analysis phase.

Visual Summaries through Video Loops Traditional video loops include frames taken at constant time intervals, presented in indefinite cycles, for a content overview or summary (Figure 3a). Average color loops and dominant color loops, are also video loops, but present selected frames based on their proximity to these color properties, providing a content overview that is more representative of the video colors. The background color in ColorsInMotion is configurable. For printing with optimal contrast, we chose a light color for the figures in this chapter. In an installation, a darker background can be more adequate to highlight the videos.

Color and Motion Based Views For a stronger emphasis on the motion aspects, videos are represented by average scene loops or by slit scans. In the loops (Figure 5a), each image captures the movement through the average pixel colors occurring in each scene in the video. Slit scans (Levin et al, 2005-2008) capture, in a still image and in sequence, the action happening in the central area of the video. Instead of blurring the movement into frame sized images for the scenes that are presented in sequence, like in the average process, it focuses in the central activity and expands across the duration of the video. In this view, scene changes can be noticed by the user through the discontinuities or abrupt changes in the horizontal sequence. In the video space (Figure 5b), slit scans are presented in a condensed or summarized form that can be expanded and scrolled through user interaction over the image, as exemplified in the middle for the purple Sevillanas’ dance video.

179

Interactive Visualization and Exploration of Video Spaces through Colors in Motion

Figure 3. Video space views: a) (left) video loops; b) (right) video average or dominant color

At the Video Level From the video space, each video can be accessed individually, through user selection. This view allows watching the video, accessing details and the different views of the video. Figure 6 presents the individual view of the purple Sevillanas’s dance video. The video is playing in the center. In the top row, the color aspects are highlighted through the average color loop and circle (purple), followed by the dominant colors’ rectangle, and the dominant color circle (a brownish dark red) and loop. Note that the dominant color prevails in the striped rectangle, as expected. To the left

of the video, the traditional loop, to the right, the average scene loop – stressing movement aspects – complemented by the scrollable slit scan in the bottom. From this view, users get a complementary overview of the video from the different perspectives, and can access any of these views at the video space (through direct selection), where they can compare this video to the others, in the chosen perspective. For example, they can perceive different colors and rhythms in the dance videos from different authors and countries, in the slit scan view.

Figure 4. Video space views based on color: a) (left) video dominant colors; b) (right) video average or dominant colors along time

180

Interactive Visualization and Exploration of Video Spaces through Colors in Motion

Figure 5. Video space views based on color and motion: a) (left) average scene video loops; b) (right) slit scans

Searching the Video Space The video space we are demonstrating presents a selection of videos in a cultural context, featuring themes like music and dance from different authors of different countries. The user can further search and select videos based on the dominant or average colors. This can be done by selecting one or more colors from a palette, or from the real world, through a webcam (Figure 7). In the case of more than one color, percentages of dominance

for each color can be specified. In any case, a threshold can be defined for the search precision. This search can be issued from any one of the video space views. Results are presented in the same view where the search was issued. Video space views are defined as physical particles gliding on the screen, where the forces among the videos are dependent on their relative colors, attracting like and repelling different colors. Therefore, these views allow searching by color similarity, either by just watching particles

Figure 6. Individual video view

181

Interactive Visualization and Exploration of Video Spaces through Colors in Motion

Figure 7. Search by color interface

finding their way, either by interacting to reposition or change speed of selected particles, changing the focus and influencing their path. The system has the particles starting on random positions and then evolving as the forces between them act. Figure 3a presents the video space in its initial state, where the videos take a random position. In Figure 3b, the circles have already moved for a while, attracting those of similar color. There is one specific view in the video space that can only be obtained by a particular search the video mosaic. Mosaics are based on layered imagery, composed of individually recognizable elements that suggest a larger overall picture (Martin, 2004). This view is composed by small video loops of dominant color that best match the color of each small region of a given search picture, creating the mosaic effect. For this search, the user drags an image into the application, or he can opt to capture one from the real world with the webcam. The search methods used to create the mosaics are similar to those used to select the videos in the video space based on color dominance. In future developments, through image content recognition or just metadata access, if the image was already annotated for instance in an image repository, the search can be reinforced with semantic properties. For example, if the user presents an Eiffel tower picture, all the videos selected to create the tower in the mosaic would

182

themselves include the Eiffel tower or, depending on the scope, would be from Paris, or France. In the cultural context of our demo, for example, it would select music or dance videos from France. Video mosaics can contain multiple layers of meaningful images, interesting for e.g. in commercial, cultural and artistic domains, and be the expression of creativity aided by an automatized search process.

Interactive Creativity with Colors in Motion Chance is often considered an important element in the creative process, and can be seen as a complement to user control in the authoring of digital artwork (Kneller, 1965; Paul, 2003). In ColorsInMotion, users can take the chance on the video properties, color search and the forces in the video space, and interfere by selecting the view that is presented, the videos that are on the set, and then influencing the movements and zooming to create the visual outcome. They can also take the chance on individual video content, and control parameters like threshold to influence the aesthetics in the creation of views that capture color or motion, like average scene frames, video mosaics or slit scans. In the video space, defined as a physical world, all the particles have the same mass and random speed at the start of the evolution. Once the par-

Interactive Visualization and Exploration of Video Spaces through Colors in Motion

Figure 8. Tracing particles in video space average color view

ticles are deployed, forces are created between all of them. As mentioned before, these can be forces of attraction or repulsion, depending on the similarity of colors among them. If the similarity is high, the particles will attract each other, and if they are completely different, particles will repel one another creating an evolution that will organize the world by color similarity. Thus, similar colors tend to group and follow each other, while colors that are dissimilar will react and run away from each other, colliding on the screen boundaries and coming back. Since the world does not have any gravity, particles will bounce on any direction. This interaction among the particles eventually fades off after a while, because of a small drag force introduced and the tendency to get the repelling forces apart. This way, the movement does not become perpetual. Users can interact with the system by dragging the particles around, changing their speed and direction and then see the interactions produced among them. They can also zoom in and out on the particle system, changing the size and position of the particles, and choose to capture the trace the particles leave behind. This way, in a mixture of chance and user action, dynamic visual effects can be created and captured. Figure 8 exemplifies one of these creations based on the average color view of the video space, resembling an abstract painting. Note how some similar colors already tended to approach, and remember that the user can interfere with this tendency at any time. Users can take the chance on individual video content, and optionally control parameters to influence the aesthetics in the creation of some of the views. Figure 9 presents average scenes

capturing the color and motion of individual scenes on a music video, on the left, and a dance video, on the right, with an impressionist aesthetics. The threshold parameter can influence the result. These views took part on the average scene video loop, presented in Figure 5a. Different parameters can also interfere with the selection of the loops for video mosaics and the selection of slits in the slit scans, influencing the creation of the visual outcome.

Interacting through Color and Motion All the interactions can be done with mouse and keyboard, and most of them can be done just through point, click and drag - even values can be introduced with sliders - also adequate for touch screen interaction. Besides the capturing of colors and images from the real world, for search by color and video mosaic purposes, we have also made some mappings for interaction based on the detection of predefined colors in specific regions in front of the screen; and we are also working on a mapping for gestures. These modalities based on color and motion could provide a more natural and deviceless interaction in an installation setting, in this experience of colors in motion. Interaction through keyboard and mouse is adequate for most situations. Keyboard in particular can be the preferred mode for experienced users, for example in a VJing scenario, allowing better performance and accuracy by selecting the right keys right away. The alternate interactions provide the flexibility to accommodate different types of users and scenarios of usage, towards more natural and potentially immersive experiences.

183

Interactive Visualization and Exploration of Video Spaces through Colors in Motion

Figure 9. Average scene views of music and dance

FUTURE RESEARCH DIRECTIONS As future perspectives, we intend to refine video processing and visualizations based on more systematic evaluations, explore new video properties and views for different contexts, and also evolve farther towards more immersive visualization and interaction techniques. Semantic search can also evolve to new levels in ColorsInMotion, or again in cooperation with media repositories, accessing videos available and shared in wider contexts, relying for e.g. on Web 2.0 services and metadata standards. As an example, video mosaics could be built by searches based on image content recognition or previous metadata access. Complementing visual with semantic properties will narrow down the search results and give the whole mosaic a more meaningful presentation, in the sense that the image would be recreated by videos with a common theme. For example, if the user presents an Eiffel tower picture, all the videos selected to create the tower mosaic would themselves include the Eiffel tower, be from Paris, or France, depending on scope and context. Complementing the possibility to capture information from the real physical environment, searches can also be location aware, allowing to filter information for local contexts. Another perspective includes extending the interactive creative browsing to support video and

184

media editing, also building on our previous experience with evolutionary video editing (Chambel et al., 2007). Digital video editing is often a time consuming task. Search and visualization methods can be used to aid users in selecting the video segments and views, or even support them in the editing, allowing for example, the specification of color and motion, in addition to semantic properties, in a timeline script as requirements to create the videos. These options could help to make the task of editing easier, more creative and even more pleasant.

CONCLUSION We presented ColorsInMotion, an application that supports the interactive and creative visualization and exploration of videos with a strong emphasis on color and motion properties, at the crossroads of information access, culture and digital art. Users can visualize the video space through different views, to search, compare and interact with a selection of videos in a cultural context, featuring areas such as music and dance from different authors and countries. Complementary views, based on video’s content, their dominant colors and motion, like video loops, colored shapes, and slit scans, allow visualizing and emphasizing videos’ properties, helping to deal with their

Interactive Visualization and Exploration of Video Spaces through Colors in Motion

inherent complexity in more perceptive ways. In terms of perception, color space choice and the underlying quantization strategy were aspects that had important significance in classification and search. Videos are preprocessed and indexed into the system, according to different dimensions, but may come from several sources in the virtual or real world. This type of approach is becoming increasingly important as video information becomes increasingly pervasive. Tag based search is usually supported by video repositories like YouTube, but visual properties like color and motion are usually not taken into account. After getting the right context through tag search, visualization techniques allow to present information in meaningful ways to ease the perception, comparison and search through browsing. In ColorsInMotion, visualizations can, for example: allow the user to perceive and compare the different colors and motion patterns in Fandango and Sevillanas, or folk dances from Portugal and Spain; help to find a video from a singer with a dominance of indigo blue, along with other videos with similar color dominance, to include in a specific publicity campaign; allow a video jockey to perform a selection and composition of videos and video collections taking visual properties into account; or allow the interactive creation of views with aesthetics influenced by color and motion properties, like those presented in the Interactive Creativity subsection. The application can be used in a traditional setting with a screen, keyboard and mouse, or a touch screen, adequate in many contexts. But we are also developing more natural and deviceless interfaces, adequate for installation settings and ambient interaction contexts. Along with the capturing of videos, colors and images from the real world, for data capture and search purposes, color and gesture based interactions are being tuned to allow for a more involving and richer experience of colors in motion in the real and virtual spaces.

REFERENCES Bestiario (2008). Videosphere. Retrieved from http://www.bestiario.org/ research/videosphere/ Boreczky, J., Girgensohn, A., Golovchinsky, G., & Uchihashi, S. (2000). An interactive comic book presentation for exploring video In Proceedings of the SIGCHI CHI’00 Conference on Human Factors in Computing Systems, The Hague, The Netherlands, April 1-6 (pp. 185-192). New York: ACM. Card, S., MacKinlay, J., & Shneiderman, B. (1999). Readings in Information Visualization: Using Vision to Think. San Francisco, CA: Morgan Kaufmann Publishers. Chambel, T., Correia, L., Manzolli, J., Miguel, G. D., Henriques, N. A. C., & Correia, N. (2007). Creating Video Art with Evolutionary Algorithms. Special Issue on “Technology and Digital Art” [Elsevier.]. Computer & Graphics Journal, 31(6), 837–847. doi:10.1016/j.cag.2007.08.004 Chambel, T., & Guimarães, N. (2002). Context perception in video-based hypermedia spaces. In Proceedings of ACM Hypertext’02, (pp. 85-94), College Park, MD., USA. Daniel, G., & Chen, M. (2003). Video Visualization. In Proceedings of Vis’03, the 14th IEEE Visualization 2003, October 22-24. IEEE Computer Society, (p.54), Washington, DC. Fels, S., & Mase, K. (1999). Interactive video cubism. In Proceedings of NPIVM ‘99, Workshop on New Paradigms in Information Visualization and Manipulation, Kansas City, Missouri, USA, Nov 2-6, (pp.78-82), New York, NY: ACM. Few, S. (2007). Data Visualization: Past, Present, and Future. IBM Cognos Innovation Center.

185

Interactive Visualization and Exploration of Video Spaces through Colors in Motion

Finkelstein, A., & Range, M. (1998). Image Mosaics. In R. D. Hersch, J. André, and H. Brown (Eds.) In Proceedings of the 7th International Conference on Electronic Publishing. Lecture Notes In Computer Science, vol. 1375 (pp.11-22). London, UK: Springer-Verlag. Girgensohn, A., Boreczky, J., Chiu, P., Doherty, J., Foote, J., Golovchinsky, G., et al. (2000). A semi-automatic approach to home video editing. In Proceedings of UIST’00 the 13th Annual ACM Symposium on User Interface Software and Technology, San Diego, CA, USA, Nov 6-8, (pp.81-89). New York: ACM. Idée’s. Multicolr Search Lab, (2008). Retrieved from http://labs.ideeinc.com/ multicolr/ Irani, M., Anandan, P., & Hsu, S. (1995). Mosaic based representations of video sequences and their applications. In Proceedings of ICCV’95 the Fifth International Conference on Computer Vision, June 20-23 (p.605). Washington, DC: IEEE Computer Society. Kerminen, P., & Gabbouj, M. (1999). Image Retrieval Based on Color Matching. In Proceedings of FINSIG’99 Finnish Signal Processing Symposium, Oulu, Finland, (pp.89-93). Klein, A. W., Grant, T., Finkelstein, A., & Cohen, M. F. (2002). Video mosaics. In Proceedings of NPAR’02 the 2nd international Symposium on Non-Photorealistic Animation and Rendering, Annecy, France, June 3-5, (pp.21-28). New York, NY: ACM. Kneller, G. F. (1965). Art and Science of Creativity. New York, NY: International Thomson Publishing. Levin, G. and Collaborators. (2005-2008). An Informal Catalogue of Slit-Scan Video Artworks and Research. http://www.flong.com/ texts/lists/ slit_scan/

186

Makela, M. (2006). LIVE CINEMA: Language and Elements. Unpublished MA dissertation in New Media, Media Lab, Helsinki University of Art and Design. Manovich, L., & Kratky, A. (2010). Soft Cinema: Ambient Narrative. Retrieved February 14, 2010, from http://softcinema.net Martin, S. (2004). Video Mosaics, Berkeley, Final Project for CS283 Graduate Graphics with Prof James O’Brien, http://stevezero.com/eecs/ mosaic/index.htm Martinho, J., & Chambel, T. (2009). ColorsInMotion: Interactive Visualization and Exploration of Video Spaces. In Proceedings of the 13th International MindTrek Conference: Everyday Life in the Ubiquitous Era. Tampere, Finland, Sep 30th-Oct 2nd, (pp.190-197). New York: ACM. Nunes, M., Greenberg, S., Carpendale, S., & Gutwin, C. (2007). What did I miss? Visualizing the past through video traces. In Proceedings of ECSCW’07 European Conference on Computer Supported Cooperative Work (pp.1-20). London: Springer-Verlag. Paul, C. (2003). Digital Art. London, UK: Thames & Hudson. Pepperell, R. (2002). Computer aided creativity: practical experience and theoretical concerns. In Proceedings of C&C’02 the 4th Conference on Creativity & Cognition, Loughborough, UK, October 13-16, (pp.50-56). New York, NY: ACM. Perez, S. (2008). The Best Tools for Visualization. http://www.readwriteweb.com/ archives/ the_best_tools_for _visualization.php Reas, C., & Fry, B. (2007). Processing: A Programming Handbook for Visual Designers and Artists. Cambridge, MA: MIT Press.

Interactive Visualization and Exploration of Video Spaces through Colors in Motion

Rocha, T., & Chambel, T. (2008). VideoSpace: a 3D Video Experience. Installation. In Proceedings of Artech’2008 the 4th International Conference on Digital Arts, Portuguese Catholic University, Porto, Portugal, November 7-8. (pp.305-310). Saykol, E., Güdükbay, U., & Ulusoy, O. (2004). Integrated Querying of Images by Color, Shape, and Texture Content of Salient Objects. In Tatyana Yakhno (Ed.), Proceedings of ADVIS’2004 Advances in Information Sciences. Lecture Notes in Computer Science (LNCS), Vol. 3261 (pp.363371). Izmir, Turkey: Springer-Verlag. Sims, K. (1991). Artificial evolution for computer graphics. In Proceedings of SIGGRAPH ‘91 the 18th Annual Conference on Computer Graphics and Interactive Techniques (pp.319-328). New York, NY: ACM. Tang, A., Greenberg, S., & Fels, S. (2009). Exploring video streams using slit-tear visualizations. In Proceedings of CHI EA ‘09 the 27th international Conference Extended Abstracts on Human Factors in Computing Systems, Boston, MA, USA, April 4-9 (pp.3509-3510). New York, NY: ACM. Tufte, E. R. (2006). Beautiful Evidence. New York, NY: Graphics Press.

KEY TERMS AND DEFINITIONS Color Histogram: Represents the distribution of the colors in an image. In digital images, it counts the number of pixels that have each one of the colors in the underlying color space. RGB and HSB color spaces are often used in color histograms. HSB Color Space: HSB stands for Hue, Saturation and Brightness. It is a cylindricalcoordinate representation of colors in the RGB color space. This rearrangement of geometry in the model allows for the definition of more perceptual dimensions. It is therefore referred to

as a human-oriented color model. Its dimensions are closer to those used by artists: Hue stands for the actual color in the color spectrum, Saturation refers to how strong or grayish the color is, and Brightness distinguishes darker and lighter colors. This color space is also known as HSV, where V stands for Value. Interactive Visualization: Refers to a process where humans interact with a computer in the creation of visual representations of information. Humans have some control over some aspects of the visual representations in a timely manner, often in real time. RGB Color Space: Is a color space based on the RGB, Red Green Blue, primitive light colors. It is an additive model because all components add to the light that is transmitted. Colors in this space are defined as a combination of these primary colors, being black the color that as null values in the three components and white (day light) the color with maximum values in these components. It is often referred to as a hardware oriented color model, because it maps directly to pixel intensities in the three components. Unlike subtractive models, it is adequate for light based devices like screens and scanners. Slit Scan: Is an imaging technique that creates static images from time-based phenomena. It allows to capture motion in a single image. In traditional film photography, where the concept was created, slit scans are obtained by exposing film through a slit-shaped aperture while sliding along. In digital video, the thin slices are extracted from the subsequent frames in a sequence and concatenated into an image. It has been used mostly in the cinema and art projects. Video Space: Is a collection of videos that can somehow be related and accessed from a common interface. Video Visualization: Is a computational process that conveys to users, in appropriate visual representations, meaningful information extracted from video.

187

188

Chapter 10

Issues on Acting in Digital Dramas Thomas Schmieder University of Applied Sciences Mittweida, Germany Robert J. Wierzbicki University of Applied Sciences Mittweida, Germany

ABSTRACT With advanced technology there are new possibilities to interact in virtual environments. Game players are being given more and more new opportunities to intervene as avatars in what is happening in the game, take on roles, and alter the flow of the stories. Through the interaction of many users new storylines and plot constructs are developed, which demonstrate many typical characteristics of modern dramas which are performed in real theatres – the plot is, for example, non-linear and attention is no longer paid to uniting time, place, and plot. These digital “performances” differ greatly from plays performed on real stages, however they are programmed as computer games with the result that the plot must fit into a pre-defined interaction pattern. The players are not casted like real actors. They step out onto the virtual stage as non-trained avatar actors and apart from the usual help options there is initially no director to instruct them. Also, the actions of the virtual actors are not foreseeable and the stories told have no distinct dramatic composition. One of the challenging problems of tomorrow’s iTV is how to generate a digital drama that looks like a real movie but which emerges out of the interaction of many users. The problem of actors’ credibility has been widely discussed in the relevant literature, however only in the context of the traditional theatre play. This chapter describes the concept of a future digital drama and investigates some fundamental aspects of acting in digital environments. The focus is put on the “competitive acting”, a new paradigm for digital stage plays of the future which combine drama with interaction-driven dialogue and action elements in converged media.

DOI: 10.4018/978-1-60960-774-6.ch010

Copyright © 2012, IGI Global. Copying or distributing in print or electronic forms without written permission of IGI Global is prohibited.

Issues on Acting in Digital Dramas

INTRODUCTION As media convergence evolves, it becomes more and more evident that not only digital environments but also the roles of the users therein are changing. Digital environments become digital theatres letting the players augment their physical existence through their digital representations in virtual spaces (Flintoff, 2004; Laurel, 1993). As the new possibilities to interact with the “old” TV medium open up viewers become active users and gamers. Players within interactive game environments become digital actors and performers; the users’ actions not only push the game forward but also let dramatic plot structures emerge and unfold. The users’ avatars orchestrate a digital stage play. Drama is typically described as a work portraying life intended for performance by actors on stage. Traditional dramas strongly focus on roles to be played and on the verbal and visual expression of human emotions. They build on a conflict with the “me“, with the fellow human beings and with the environment. They are often concerned with the heteronomy of the individual, his constant search for identity and his instinct. The physical nature of dramatic forms arises out of the physical existence of a real theatrical stage where the action takes place. Even if the fixed structure is removed in modern dramatic approaches and the audience is confronted with non-causal plots and non-connected, strung-together “snapshots”, the implemented storylines remain linear and do not let viewers intervene in or influence the play. With technology and interactivity becoming more and more distinct and predominant in our lifestyles, culture and society, a demand for interactive experiences and the delivery of more personalized entertainment content becomes noticeable. In addition, in the era of virtual worlds the human imagination on dramatic content design and delivery is tendentially changing since “the boundaries between the real and the virtual are becoming increasingly confused and the interface

is becoming increasingly important in our experience” (Flintoff, 2004). It is very probable that in future approaches to drama a virtual environment will serve as a stage. The actors will even play their roles distributed across different physical locations or as models who only lend their appearances, mimics, gestures and speech to virtual AI actors (Flintoff, 2004; Waxman, 2006). The recipients in turn will gain the possibility to actively participate in the play and thus rise to become active performers. Theatre might, in this case, become just a metaphor for an interface where the real and the virtual converge. Future digital dramas in such environments will tell stories involving conflicts and emotions through action and dialogue, as we know it from traditional theatre stage plays. The substantial difference will be the technology-enabled possibility to be involved in the action as a future “digital me”. We intentionally avoid at this stage using the word “avatar” since the future will almost certainly enable us to construct things which go far beyond what we associate with the avatars of today (2009), e.g. simple human representations in SecondLifelike environments (www.secondlife.com, 2010). Today, we are already approaching the concept of digital drama through MMORPGs - Massive Multiplayer Online Role Playing Games - which feature a large base of players interacting with one another in a virtual world (www.amd.com, 2010). Stanislavski (1863-1938), one of the pioneering theoreticians in theatre acting, demanded from theatre an exact reconstruction of reality. According to Stanislavski and his “creative if” concept (Stanislavski, 2008) actors should personify the characters. In an ordinary life our bodies automatically manifest our feelings through mimic and gestures for example. These have to be intentionally used on stage to make it possible to convey the intentions of the performance, to indicate and to empathize with the inner feelings of the characters played and demonstrate the “authenticity” of the action on stage and the truthfulness of the plot. Actors have to treat the set as if it were reality.

189

Issues on Acting in Digital Dramas

They should not only understand the individual goals of the roles played but also keep in mind the individual inner line of the characters by pursuing their individual goals throughout the play. Actors need to remain in the frame of mind of the characters off-stage so that the aforementioned inner line does not get broken: “if the inner line is broken an actor no longer understands what is being said or done and he ceases to have any desires or emotions” (Stanislavski, 2008). Lee Strasberg, an American director, actor, and producer (1901-1982) held a slightly different view, although based on the theory of Stanislavski. Lee Strasberg’s teachings are associated with the term “method acting” (Adams, 1980). According to Strasberg’s explanations a realistic and lifelike manner of acting is supposedly achieved through impersonation, and by letting personal experiences interact with a given situation and thereby influence the play. An actor should recall emotions or sensations from his or her own life and use them to identify with the character being portrayed (Strasberg, 1988). Thus, the actor’s dramatic interpretation inevitably emerges out of the actor’s own personality and his or her psychological structure. Strasberg actually derived his Method Acting from Stanislavski’s “creative if” concept but he demanded from actors that they behave in a particular way since “the circumstances of the scene indicate that the character must behave in a particular way” (Strasberg, 1988). The actual situation and plot unfolding should motivate the actors to bring their own feelings, emotional responses and experiences into the play and largely translate their human experiences through unique and spontaneous behaviour. “To tell any story - from a fantastic, surreal story to a very naturalistic, realistic story - the key ingredient is always humanity” (Tehrani, 2007). The “Method Acting” training by Strasberg stands for successful “Hollywood Style” movies and is still the basis for actor training in many institutions worldwide.

190

“PARTICIPATION-OF-MANY” IN DIGITAL THEATRES OF THE FUTURE Approaching game-like settings in converged media, actors need to not only embody the characters played but also develop an understanding of the concept of a virtual stage, where the digital representation of actors needs to co-exist with digital representations of game players, which, in this can be treated as untrained actors. Schmieder, Wierzbicki and Lugmayr already reported on a GAMECAST system, which represents such a setting and is currently under development (Schmieder, Wierzbicki, & Lugmayr, 2008). The basic idea behind GAMECAST is to create a television series, the course of which can become interactive. From today’s point of view, one of the significant problems with digital dramas of the future will emerge from the interaction of a large number of untrained actors represented by game players. These players need to be granted the right to settle conflicts on their own initiative and interact with the play according to their own strategies so that their particular goals can be reached. The conflict a player is facing is typically binary in its nature and it can build up on a win-or-lose basis. The action follows a typical branching structure which represents the set of possible paths the user can select from and walk through. One of the most important goals within the GAMECAST storytelling concept is to create a feeling of interactive freedom for the player while maintaining affordable production costs. To reach this goal it is important that the player does not notice the interactive storytelling structure (e.g. a branching structure, or a structure that uses scenes according to their dramatic/narrative function) beneath it, with variable scenes, triggered when particular conditions are met. An “old-fashioned” computer game is based on an “interaction-of-one” principle - only one player is involved in playing the game and only one player needs to be given options to choose

Issues on Acting in Digital Dramas

from. In MMORPG-like “interaction-of-many” settings, where players not only appear as virtual actors in the digital drama but also contribute towards the plot unfolding therein, each player needs a specific goal to be achieved throughout the interaction process. In converged, “participationof-many” environments like GAMECAST (we are thinking here primarily about the fusion of games with television series as described by Schmieder et al. (2008) the situation becomes much more complex. Virtual actors (AI characters; see. Magerko, Laird, Assanie, Kerfoot, & Stokes, 2004), active players (lean-forward users, untrained actors), passive, lean-backward audiences and human game masters (directors of the play) create a unique sphere of agitators, prosumers, consumers and “superusers”. AI characters and human game masters regulate the flow of the interaction. Appearing as roles from the story they convey goals for the interaction scenes to the untrained actors (players). AI characters and human game masters also operate as agent provocateurs. They foster conflicts and intervene, if needed, using dramaturgical methods that originate from the story. The overall goal is to create a dramaturgically fruitful unfolding story, which is required to satisfy the passive recipients of a series. This approach aims at creating authentic and credible dramatic lines and believable appearances in a TV series, not just game-based performances in terms of on-screen-as-if-on-stage performances as described by Lowood (2005). One of the challenges of this approach is to motivate the players to play the game, let them have fun with the “role” they create and enjoy the feeling of improvisation they are experiencing. It is not important to make them perfectly impersonate the role but instead to try to generate competitive situations on a digital stage where the players concentrate on gaming and winning the game rather than on playing roles. The super-objectives (Stanislavski, 2008) of a player within GAMECAST are shaped when he or she decides which game option to choose. A simple decision like this might be to join and

support a specific group of protagonists in the series (GAMECAST reference). The player’s social relations within the online world, his way of playing as well as his behaviour and the sense of achievement within different quests the player has started shape his objectives, his virtual character and personality. Considering storytelling the most appropriate way to construct interactive narratives for digital theatres would be to take into account the individual actions of every player and also the player’s digital me accordingly. This would, however, raise the costs to the extent that the production would soon become unaffordable. Therefore, for new players in GAMECAST the goals are set according to the party they joined. This approach incorporates special guided scenes in which supporting characters (non-player characters or the characters which are controlled by human game masters) explain the goal of the mission and encourage the players accordingly. The function of the scenes is to provide each player or each group of players with a certain goal, which might be achieved during the interaction scene which follows. The goals of the quests are set to foster dramaturgic conflicts, as described above. The guided scenes are not shown to the audience which does not interact with the game part of the series, as long as they do not become game players themselves and take part interactively. The individual players can define their goals in the game by finding a position amongst what is available in the interaction space / storyworld, for example by deciding to start out on a particular mission, react to a situation in the interaction space or support or oppose a group of players or NPCs (non-player characters). A player therefore chooses his goals in the game not autonomously, but dependant on the situations (characters, stories, game stimulations) he finds in the game. This makes the player’s behaviour in the game more predictable. In addition, the development of a goal in the game provides the interaction of the player with a specific purpose (Costikyan, 2002) since the player interacts in order to reach this goal.

191

Issues on Acting in Digital Dramas

Due the possible large number of players in GAMECAST not everyone can be granted the same ability to influence the play. To differentiate between levels of influence, a concept of interaction levels has been introduced which accounts for the players’ performance and their reputation within the game community which is run parallel to the series. Dependent on these, a player can even become and impersonate the main character of the series (Schmieder et al., 2008). Players have the chance to advance within the story and impersonate this role, which will then first appear within the story. Under the conditions described above, acting theories that use the remembrance of similar emotional experiences are of almost no use for digital theatres. The “method acting” by Strasberg can certainly help trained actors to put themselves into a certain mood on a real stage, but it can hardly be adopted by untrained actors who are playing a game. Certain extreme emotions like suicidal despair are just too harsh as that an untrained actor would and should experience it. Those interacting would have to be out of their mind “if they would want to submit themselves to the fate of a heroine who commits suicide as the result of a love affair turned bad” and that “... any attempt to turn empathy […] into first-person, genuinely felt emotion would […] trespass on the fragile boundary that separates pleasure from pain” (Ryan, 2001). Furthermore it would simply take too long for untrained actors to recall all the aspects and feelings of a certain situation.

COMPETITIVE ACTING Thanks to training and acting studies, professional actors, are able to (un)intentionally orchestrate and convey their feelings with their bodies, speech and faces, whilst still staying focused on the scene and the other actors within it. Untrained actors lack this training and often focus on the process of conveying emotions during their per-

192

formance, which can likely lead to a point where they focus more on demonstrating than on really immersing themselves in the situation. That is why untrained actors are always threatened with becoming emotionally blocked, which makes them choose strategies based upon what they believe to be appropriate in a given situation. For this reason, performances by untrained actors often seem unnatural. The design of GAMECAST, a convergedmedia “theatre”, is based on a principle we call “competitive acting”. The German noun for acting, “Schauspiel”, which could literally be translated as “Show-game”, reflects relatively well the two elements of acting – the show and the game (play). GAMECAST focuses on the latter; it concentrates on the ‘game’ rather than on the mimetic aspect of acting. The authors suggest the following three principles to the players: 1. “Do not concentrate on acting!” 2. “It is not a staged play - it is a game experience, and every game has its rules!” 3. “Try to win by using the game’s strategies and you will be a believable actor” In environments like GAMECAST where many players actually face the same conflict, it is obvious that each player needs to be confronted with a situation which demands a win. In order to win and reach their goal in the game, players can employ “strategies” which are possible in the virtual world. The term strategy is used here to mean goal-oriented actions which can be utilised by the player so as to reach a particular goal. According to this the strategies are determined by the possible actions the player’s avatar can carry out. These possible actions can, for example, include actions such as running, using objects, fighting or communicating via (voice) chat. What at first sight appears to be a more or less significant limitation is in fact a great relief for an inexperienced performer and player. If the individual possible actions of an avatar represent

Issues on Acting in Digital Dramas

authentic plots in the respective game world then the players who are carrying out these actions in order to reach goals in the game world function as an organic component of the storyworld and as genuine protagonists. The goals of the different players are devised to oppose each other so that they will lead to a conflict in the interaction scenes. Striving for success and concentrating on the mission is one of the main driving forces as regards the formation of involvement and emotion. It implies behaviour that is free from worrying about whether one is doing the right things or not. In addition, it lets one forget the fear of being watched – a perfect condition for the creation of a candid actor performance through playing a game (see Tehrani, 2007). Real actors need years of training to gain such a level of acting performance credibility, honesty and authenticity on stage. This is easier in a virtual stage situation such as GAMECAST: The reason for this is that all factors which are necessary on stage and in film to produce and transmit the scenes are not visible in virtual worlds. There are no visible cameras or spotlights and no director to tear the actors away from their emotions with his stage directions. Because these distracting effects are not present it is easier for the actors to get involved with the scenes. Also missing is the direct confrontation with situations of observation by cameras or an audience. Stage fright, which occurs above all because the actor asks himself whether he can fulfil the expectations of his audience (www.movie-college.de, 2010), is thereby significantly lessened. The fact that there is no stage situation with an audience and no cameras means that the interaction in virtual words resembles more a game than a play. The playful aspect of this situation is additionally promoted by the programmed game rules and the fellow players. The rules ensure that causal effects definitely occur, whereas on a real stage or in improvisation they must be imagined by the actors (Hollburg, 2010). This can easily lead to a situation where the

actors fail to imagine these rules because they are excited or lack concentration and begin to assert emotional effects and theatrical reactions which they deem to be appropriate. In the case of improvised plays with inexperienced actors this often leads to a situation where the scenes appear to be unnatural and staged. The playing situation in a virtual world with its “rules of the game” maintains consistency in the game on the one hand and on the other hand this system sanctions itself. In a competitive game situation in which players are trying to reach a particular goal, players who do not really play seriously (for example because they try to come across in a particular way through their behaviour) lose against players who above all aim to win and achieve the goal. Based on these aspects, the conflicts within each scene in a GAMECAST online world are not staged; they naturally arise out of the fact that different players pursue different opposing goals. Rather than choosing methods of acting, players choose what they believe and feel to be the best strategies necessary to prevail. Their dialogue partners choose what they believe to be the appropriate response to the current action if they don’t want let their opponent be victorious in the scene. Due to the competitive approach the players don’t need to feign an appropriate emotion. Just like in a game, emotions develop automatically as a result of success of failure depending on whether or not the chosen strategy was successful. Therefore, every scene emerges out of the actions and reactions of the performers and unfolds in an improvisation-like manner. Even professional actors work according to this “step-by-step-principle” (www.schauspielhausgraz.com, 2010) which is common in improvisational theatre. The approach taken by an actor is analogous with the approach taken by a player in a game. The goals in a game and the intention to win, i.e. to reach these goals, can, according to game designers such as Costikyan lead and direct the

193

Issues on Acting in Digital Dramas

Table 1. Comparison of cognitive processes in video games and plays (Schmieder, 2009). 1.

2.

3.

4.

Player

“At every point, he or she considers the game state. That might be what he sees on the screen.”

“He tries to decide on the best course of action.”

“He considers his Objectives and the resources available to him; he considers his opposition, the forces he must struggle against.”

“And he responds as best he can to achieve his objectives – his goals.”

Actor

Acquire

Evaluate

Decide

Act

The top line in Table 1 shows the approach taken by a player in a game. They are citations from game designer Greg Costikyan’s greatly respected essay “I have no words, but I must design” (Costikyan, 2002). The bottom line shows the train of thought of an actor in a play situation as taught at one of Germany’s most influential acting schools “Felix Mendelssohn Bartholdy” (Neubauer, 2000).

players’ interaction: “The basic transaction we make with games is to agree to behave as if achieving victory is important, to let the objective guide our behavior in the game. There’s little point, after all, in playing a game without making that basic commitment” (Costikyan, 2002). Thus a game with its rules represents a perfect basis for the creation of an authentic and vivid acting achievement: “Rules might not seem like much fun. But once players set the system of a game into motion, play emerges. And play is the opposite of rules. Rules are fixed, rigid, closed, and unambiguous. Play, on the other hand, is uncertain, creative, improvisational, and open-ended. The strange coupling of rules and play is one of the fascinating paradoxes of games.” (Zimmerman, 2005) A major problem for actors comes about when they suffer an emotional block and can no longer involve themselves in the scene’s situation in order to feel emotions (Richter, 2006). A competitive acting setting can break through this blockade and theatrically awaken emotions even in non-experienced actors. The resulting emotional feelings can then be picked up by corresponding interfaces, which can recognise emotions (Küblbeck, 2010; Waxman, 2006) and transferred to the controlling of mimic animations, the language and the theatrical expression of the avatar’s animations. In this way inexperienced actors get the chance

194

to act and thereby effectively become credible actors in a film: “Just like a person who suddenly falls because of an unconscious fear that he may come to some harm, who instinctively takes up a position which will safeguard him; a position he would come up with without thinking; in the same way this affects every nerve in the body. The actor who knows how to immerse himself in a true emotion will be able to act naturally and correctly, whatever the expression.” (Sulzer, 2010) As the fun of human players would be greatly diminished if they had to follow the orders of a director, their actions cannot really be controlled. They can only be manipulated by the dramaturgical methods through game masters and NPCs, like new strong enemies, obstacles, agent provocateurs etc. appearing in the game. It is most important that the action remains believable and that it fits into the dramatic content. Problems with players who might intentionally want to disturb the interaction scenes and threaten the success of a mission would have to be solved through the appropriate intervention of a game master (director) or through a reaction carried out automatically by the system. In the play situation the system itself must also be able to identify interferences and react autonomously. To this end the actions of the players must be analysed by the game logic and organised according to the occurrence of particular

Issues on Acting in Digital Dramas

behavioural patterns. To achieve this, the game logic must analyse the actions of the players and recognise the occurrence of specific predefined patterns. If the action pattern originating from the player corresponds to the “negative patterns“, i.e. those which are not desired, the player will be confronted with an appropriate message from the system on his client interface. The expectations and the immersion of player as well as the empathy of the viewer must as far as possible not be disrupted by these rules (Engels, 2008). Rules of the game must be presented to the players as coherences which in turn must originate from the logic of the story (storyworld). For example, notifications to individual players must be presented as messages which fit into the storyworld. NPCs take on a decisive role in this respect. Players who disrupt a scene can be advised by an NPC by way of a logical argument in the sense of the storyworld which fits the story situation that they have to be withdrawn from this mission because they are, for example in an action scene, endangering the chance of victory for their own troop.

STORYTELLING IN THE “PARTICIPATION-OFMANY”-SCENARIOS In the media industry interactive storytelling has, up until now, mainly been achieved by branching off strands of a sub-plot, which can easily be realised when there are only very few possible story strands (Spierling, 2006). This does, however, only result in limited theatrical and narrative freedom (Crawford, 1993). As opposed to this, the attempts made in the area of “emergent interactive storytelling” have up until now essentially been limited to one or very few players and even the production of very short scenes is extremely complex (Spierling, 2007; Mateas, & Stern, 2008). Using emergent interactive storytelling on the commercial mass market is therefore not appropriate.

Up until now authors have been used to telling stories without any interactive influence on the part of the user (Spierling, 2007). The complexity of emergent systems makes a new approach to storytelling a prerequisite. GAMECAST unites both approaches – the linear and the emergent. At the core of every GAMECAST episode there is a specific situation/problem in the plot which the players must solve in their favour by taking on divided-up roles. This “player generated content” is the emergent part. It comes in the form of logfiles with which scenes from the online game world are recorded. The emergent part is attached to “author narrated content“: at the beginning by way of a predefined introduction which describes the initial situation and at the end by way of one of several possible endings which can potentially contain a cliff-hanger for the forthcoming episode. Instead of a continuous series plot or episodes which follow on from each other there is an episode network – a vital parallel universe (Schmieder, Straßburger, Schubert, & Nowacki, 2010), which reflects the wishes of the users and the trends within the community. This leads to extensive stories as tendencies and developments are picked up which are relevant for the community and the audience. Each episode can be available in many different variations. Users can work cinematically and theatrically with GAMECAST as actors and decision-makers in an interactively and creatively involved community and thereby semi-automatically influence and create narrative works. Through user appraisal the created works (and also the behaviour of the players’ characters and the resulting scenes) are evaluated. By linking these social appraisals with data-mining processes and semantic analysis of the logfiles the director gets a picture of how things are developing in the online world and in the community. With the help of the system he is then able to find out which scene created by the players’ interaction he can select so as to cinematically edit this development and integrate it in the series plot. In order to implement a scene which

195

Issues on Acting in Digital Dramas

was created in this way in a media-convergent series or film not all verbal utterances of a player can be included. This is due to the fact that many players speak with a particular dialect and in contrast to an actor have not usually undergone any voice and speech training. This means that professional post-production of the scenes is necessary in order to bring them up to a high-quality cinematic standard. This post-production involves re-dubbing, cutting, setting camera angles and adding further animations which are not present in the game. Also the 3D models from the game must be re-rendered using more detailed versions. At the same time the director and his team can steer the community, provoke theatrical actions and loosely link the game experience in terms of time to the production and broadcast of the format. This happens, for example, in that at then end of an episode corresponding missions are made available in which the players can continue and solve plot strands. The player-generated scenes which originate in the online world are then again recorded, evaluated, selected and integrated in the next episode (Schmieder, 2009). This results in a TV format which makes clear just how films can be created from actions and social trends emerge from minute details on the web.

DIRECTING DIGITAL DRAMAS Converged media brings us to the point where the way in which dramas and stage plays are rehashed or conceived for games, cinema and television needs rethinking. Directing digital dramas will require a balanced approach to dealing with dramatic content and interactivity. It has been claimed that interactivity and dramaturgy are fundamentally antagonistic concepts which rule each other out (Wages & Hornung, 2005; Adams, 1999). This is why the interactivity in narrative remains a challenge to critics (Rieser,1997) and many approaches to interactive films and dramas have failed due to a lack of joy in playing the

196

game (Sluganski, 2010). Granted more interactive freedom a consistent dramaturgy becomes problematic because users / players who impact on an interactive story often do not care about creating a good dramaturgy. “After all, this would require a kind of thinking that contradicts the goal of immersion. No hero in a conflict would think about which actions would lead to the best dramaturgy - she cares only about how to solve this conflict.” That is why: “Depending on the level of power over the story, the user can more or less destroy it.” (Sander, 2007) Competitive Acting attempts to incorporate the player in the process of storytelling as an agent driven by self-centred wants, so that the actions that originate from these wants foster the unfolding of the dramaturgy. This corresponds to works by Tanenbaum, among others (Tanenbaum, 2007). Tanenbaum demands a rethinking of the user’s role within interactive media: “... responsibility for the quality of an interactive narrative experience must perforce be shared across the system designer and the player … As long as we continue to design interactive narrative artefacts around the assumption that the player’s role is simply as a problematising source of error that must be corrected for, then we are denying half of the equation.” (Tanenbaum, 2007). For an interactive player a film produced in accordance with the principles of competitive acting is a game with rules and the goal of winning. For the viewers of this interaction an improvised virtual drama emerges from the actions and reactions of the players which by adopting cinematic methods can most certainly convey a plot, a story and emotions. As such, competitive acting is a “renunciation” to those ludologists (www.researchquest.blogspot.com, 2010) who are of the view that games could not convey stories. David Strasberg made very interesting statements on diverse topics within the area of acting in an interview with Bijan Tehrani (Tehrani, 2007):

Issues on Acting in Digital Dramas

“What directors need to find is how to watch an actor and how to look for what they need. One actor needs a pat on the back, the other needs a shout, and the other needs a whisper in the ear. Understanding the actor gives a huge advantage to any director. If you’re working with untrained people I think it it’s doubly important because you can’t fall back on the standard vocabulary that you use to guide a professional actor. You really have to understand what you want and what that actor needs at that moment.” The above seems to be generally proved true in any theatre, classical or digital, contemporary or future. In a situation where the play remains fully uncontrolled and the plot unfolds completely dynamically it is very probable that the action will eventually end in chaos. Directing digital interactive dramas in the future will require bringing in dialogues between multiple characters (trained and untrained actors) and structuring them in such a way that the action makes sense. This is a task which does not differ much from the traditional directing of films and television series, or better an improvised scene.

CONCLUSION The linking of the stage, film and television worlds marks a further step towards media convergence. Although many elements of traditional theories of acting may still apply in different situations within virtual environments the digital form of reciprocity which users (untrained actors, players) fulfil in different game-like settings makes digital play different in its nature from what we associate with traditional theatre. In this chapter a new model of competitive acting in converged environments was introduced and contraposed to traditional acting theories by Stanislavski and Strasberg. The “Competitive Acting Theory” aims to foster two aspects within interactive dialogue and drama. The first aspect is authenticity; the second and most important is fun and the joy of competing. It has been claimed that the authentic-

ity of acting in digital media emerges out of the competitive settings in which the player concentrates on working out a win situation for himself or for a group he joined in the play.

ACKNOWLEDGMENT This work is based on an earlier work: Competitive Acting - Issues on Action, Interaction and Acting in Converged Media, by Thomas Schmieder and Robert J. Wierzbicki, in Proceedings of the Academic MindTrek Conference 2009, Tampere, Finland, 30 - 2 Oct 2009, ACM Press.

REFERENCES Adams, C. H. (1980). Lee Strasberg: The Imperfect Genius of the Actors Studio. Garden City, NY: Doubleday. Adams, E. (1999). Three Problems for Interactive Storytellers, Designer’s Notebook. CMP Game Group, CMP Media, LLC, San Francisco, CA. Retrieved March 30, 2010, from http://www. gamasutra.com /view/feature/3414/ the_designers_notebook _three_.php AMD. (2010). Alternate Reality: The History of Massively Multi-player Online Games. Retrieved March 30, 2010, from http://www.amd. com/us-en/ Processors/ProductInformation /0,30_118_9485_9488% 5E9563%5E9599% 5E9793,00.html Costikyan, G. (2002). I have no words, but I must design. In: Mäyrä, F. (Ed.), Proceedings of Computer Games and Digital Cultures Conference, Tampere, Finland, June 6-8, 2020, Tampere University Press. Retrieved March 30, 2010, from http://www.costik.com/ nowords2002.pdf Crawford, C. (1993). Flawed Methods for Interactive Storytelling. In: The Journal of Computer Game Design Volume 7/1993a. Retrieved March 30, 2010, from http://www.erasmatazz.com/ library/JCGD_Volume_7/ Flawed_Methods.html 197

Issues on Acting in Digital Dramas

Engels, S. (2008). Character & Level Design. Retrieved March 30, 2010, from http:// www.cs.utoronto.ca/ ~sengels/csc404/lectures/ CharactersLevels_4up.pdf

Neubauer, G. (2000). Grundsätze des Schauspiels. Unpublished lecture material. Hochschule für Musik und Theater “Felix Mendelssohn Bartholdy”.

Flintoff, K. (2004). Interfacing: Drama, The Arts and I.C.T. Retrieved March 30, 2010, from http:// www.ecawa.asn.au/login /2004/vol1/ Kim_Flintoff.pdf

Researchquest blogspot (2010). Game Studies: Narratology v. Ludology. Retrieved March 30, 2010, from http://researchquest.blogspot.com /2008/04/game-studies-narratology -v-ludology. html

Hollburg, L. (2010). Die Arbeit des Schauspielers an der Rolle. Retrieved March 30, 2010, from http://www.ludwig-hollburg.de/ uploads/ media/die_arbeit _des%20schauspielers_an_ der_rolle2.pdf Küblbeck, C. (2010). Mood Detection. Retrieved March 30, 2010, from http://www.iis.fraunhofer. de/ EN/bf/bv/kognitiv/ mood_detection.jsp Laurel, B. (1993). Computers as Theatre. Boston, MA: Addison-Wesley Longman Publishing Co., Inc. Lowood, H. (2005). Real-time Performance: Machinima and Game Studies. In Conrad Gleber (Ed.), The International Digital Media & Arts Association Journal 2 (1) (pp. 10–17). Retrieved March 30, 2010, from http://www.idmaa.org/journal /pdf/iDMAa_Journal_Vol _2_No_1_screen. pdf Magerko, B., Laird, J. E., Assanie, M., Kerfoot, A., & Stokes, D. (2004). AI Characters and Directors for Interactive Computer Games, In Proceedings of the 2004 Innovative Applications of Artificial Intelligence Conference, 2004. Retrieved March 30, 2010, from http://ai.eecs.umich.edu/ people/ laird/papers/ magerko-2004-IAAI-Haunt.pdf Mateas, M., & Stern, A. (2008). Writing Façade: A Case Study in Procedural Authorship. Retrieved March 30, 2010, from http://www.electronicbookreview.com /thread/firstperson/ coderead Movie-college. (2010). Lampenfieber. Retrieved March 30, 2010, from http://www.movie-college. de/ filmschule/schauspiel/ lampenfieber.htm

198

Richter, D. (2006). Gedanken zu Improvisation und Improtheater (2006). Retrieved March 30, 2010, from http://www.danrichter.de/ impro/06/ improgedankenmai06.htm Rieser, M. (1997). Interactive Narratives: A Form of Fiction? In Convergence: The International Journal of Research into New Media Technologies, 3(1). doi:10.1177/135485659700300102 Ryan, M.-L. (2001). Beyond Myth and Metaphor - The Case of Narrative in Digital Media. Paper presented at Computer Games & Digital Textualities conference, Copenhagen. Retrieved March 30, 2010, from http://gamestudies.org/ 0101/ryan Sander, F. (2007). The Fight for Control – User Influence vs. Dramaturgy in Interactive Story Telling. Retrieved March 30, 2010, from http:// www.kreativrauschen.com /blog/2007/12/09/ thefight-for-control- %E2%80%93-user -influencevs-dramaturgy-in- interactive-story-telling/ Schauspielhaus-graz. (2010). SCHAUSPIEL AKTIV! am Schauspielhaus Graz stellt sich vor. Retrieved March 30, 2010, from http://www. schauspielhaus-graz.com/ schauspielhaus/schauspiel _aktiv/schauspiel_aktiv.php. Schmieder, T. (2009). Der Videospielfilm - Konzeption eines medienkonvergenten Unterhaltungsformats. Unpublished bachelor thesis, University of Applied Sciences Mittweida, Mittweida, Germany. Schmieder, T., Straßburger, T., Schubert C., & Nowacki, O. (2010). Konzeptpapier GAMECAST. Unpublished concept paper.

Issues on Acting in Digital Dramas

Schmieder, T., Wierzbicki, R. J., & Lugmayr, A. R. (2008). GAMECAST®: A Cross-Media Game And Entertainment System, In: Lugmayr, A. et al. (eds.), TICSP Adjunct Proceedings of EuroITV 2008, Changing Television Environments, Salzburg, Austria, 3-4 July 2008 (pp. 157-161). Retrieved March 30, 2010, from http:// www.fgam.de/papers/ 2008/GAMECAST(R)_A_ Cross-Media_Game_and_ Entertainment_System _(2008).pdf Secondlife (2010). SecondLife. Retrieved March 30, 2010, from http://secondlife.com. Sluganski, R. (2010). LucasArts. Speaks Out - Part 1: A Discussion with Hal Barwood. Retrieved March 30, 2010, from http://www.justadventure. com /Interviews/Hal_Barwood/ Hal_Barwood2. shtm Spierling, U. (2006). Nonlineare Dramaturgie. Neue Konzeptionsanforderungen für das Erzählen durch die Einflüsse interaktiver Möglichkeiten In Rebensburg, K. (Ed.), NMI2005, Neue Medien in der Informationsgesellschaft – “Film & Computer“. Aachen: Shaker Verlag. Spierling, U. (2007). Adding Aspects of “Implicit Creation” to the Authoring Process in Interactive Storytelling. In: Cavazza, M., Dinikian, S. (Ed.), Virtual Storytelling. Using Virtual Reality Technologies for Storytelling. 4th International Conference, ICVS 2007, Lecture Notes in Computer Science. Heidelberg, Germany: Springer. Stanislavski, C. (2008). An Actor Prepares. Methuen Drama, A&C Black Publishers Ltd. Reprint.

Strasberg, L. (1988). A dream of passion: the development of the method. New York, NY: New American Library. Sulzer, J. G. (2010). Ausdruck in der Schauspielkunst. Retrieved March 30, 2010, from http:// www.textlog.de/ 3728.html Tanenbaum, J. (2007). Placing the blame for a quality interactive narrative experience. Retrieved March 30, 2010, from http://www.thegeekmovement.com /blog/?p=29 Tehrani, B. (2007). An Interview with David Strasberg about Lee Strasberg Theater Institute, at Cinema Without Borders. Retrieved March 30, 2010, from http://www.cinemawithoutborders. com/ news/127/ARTICLE/1319 /2007-07-14. html Wages, R., & Hornung, A. (2005). The Virtual Real-Time Dramaturgy: Formalisation of Dramaturgic Principles, In: Virtual Reality at Work in the 21st Century – Proceedings of the VSMM 2005: Eleventh International, Conference on Virtual Systems and Multimedia, Ghent, Belgium, October 3 – 7, 2005. Waxman, S. (2006). Cyberface: New Technology That Captures the Soul. New York Times, October 15th, 2006, Retrieved March 30, 2010, from http://www.nytimes.com/ 2006/10/15/movies/ 15waxm.html Zimmerman, E. (2005). Narrative, Interactivity, Play, and Games: Four Naughty Concepts in Need of Discipline In Bushoff, B. (Ed.), Developing Interactive Narrative Content. Munich, Germany: HighText-Verlag.

199

200

Chapter 11

Re-Coding the Algorithm: Purposeful and Appropriated Play Alison Gazzard University of Bedfordshire, UK

ABSTRACT This chapter highlights the different types of play possible within videogames, developing Roger Caillois’ (1958) original categories of “paidia” and “ludus”. The game is examined in terms of Lev Manovich’s (2001) concept of the “algorithm”, in order to see how different syntagms of play are possible within the greater game paradigm. Play is categorised in terms of purposeful play, defined as the play intended by the designer, as opposed to appropriated play, which is discovered by players seeking more from the system. It is through these new terms that different types of motivation for play are discussed, leading to an analysis of how playing outside of the intended rules of the game can be considered through new terminology beyond the often negative connotations of cheating.

INTRODUCTION This chapter examines different play types possible within the videogame. Ludology (Frasca, 2003) recognises the act of playing as important to the study of videogames, yet there is often an emphasis on how the game should be played. Instead, this chapter examines the game through defining a DOI: 10.4018/978-1-60960-774-6.ch011

set of playable syntagms enabling various play types to be explored outside of the intended rules of the game. By discussing play in this way, we can see it as being appropriated for the player’s own style of playing and the various motivations behind playing in such a way. This recognises the aberrant player, the player seeking to decode the game system in a new way to which the designer intended it to be played. Defining players in this way further aims to remove the sometimes nega-

Copyright © 2012, IGI Global. Copying or distributing in print or electronic forms without written permission of IGI Global is prohibited.

Re-Coding the Algorithm

tive connotations surrounding players previously termed as “spoilsports”(Salen and Zimmerman, 2004) or “cheats “(Consalvo, 2007). In order to understand aberrant play, there is a strong emphasis on recognising how play occurs through the acknowledgement of a community of like-minded gamers. Although it is possible for players to play in isolation through single player games, it is still possible for them to acknowledge a wider community before, during and after play. The use of walkthroughs, cheat codes, hacking or modding games for the player’s own means adds to the greater social and cultural understanding of how games are discussed, used and sometimes modified. Therefore, this chapter seeks to define a new terminology to discuss this growing collective of gamers and game players, not only appropriating games for themselves, but also through furthering this knowledge in an attempt to extend the playable syntagms of the gameworld.

DEFINING THE GAME Manovich (2001) defines the “algorithm” as the underlying structure of the videogame, the structure in which players strive to uncover in hope of mastering the game. The algorithm is not only the underlying code of how the game functions, but how this code reveals itself to the player, through the various worlds and puzzles it generates. In seeking to discover the algorithm, the player “gradually discovers the rules that operate in the universe constructed by the game” (Manovich, 2001). It is through this chapter that the concept of the ‘algorithm’ will be discussed by understanding the game as a paradigm. Through an analysis of the game as a paradigm, it can be seen how the game is constructed of various syntagmatic relationships. Syntagmatic relationships are defined by Chandler (2007) as “the various ways in which elements within the same text may be related to each other”. By discussing videogames in this way, it is possible to see how

various syntagms of play can arise within the game world. Each syntagm can be related to one another but consist of various motivations by the player. It is through defining these motivations that two types of play are defined, that of ‘purposeful play’ as intended by the game designer, in contrast to ‘appropriated play’, created by those players seeking to decode the algorithm in a different way to which it was originally intended. These new play types rework and build upon Caillois’ (1958) definitions of ruled and free form play, or “ludus” and “paidia”, thus renewing their uses within videogame environments. Finally, this discussion recognises that videogames are played within a social setting, an idea I will term the ‘cultural logic’ of the game. In understanding how players are part of a wider gaming culture, both in single player and multiplayer gaming scenarios, the motivations for various types of play and their outcomes become situated within the wider context of game play itself. It can be seen that players may play in isolation, with their own agenda for playing a game, whether that be completing the game in the fastest time, collecting all the items thought to be found within the game, or being able to open levels to explore the game through player-defined rules. Similarly it is important to note that the majority of these acts exist within a framework of other similar players and the games they play. As Newman (2008) notes in the opening chapter of Playing with Videogames, “…playing videogames is not an activity undertaken in a vacuum but rather is one that is informed by and situated within the contexts of other players and their analyses and playing…”. It is these statements that help to define cultural logic. Games can be social systems without having to exist within a multiplayer domain. Single player games also exist within a context that there is the acknowledgement of other players, playing the same game, striving to complete the same missions, and competing against other player’s high-scores. It is an understanding of the cultural logic that will underlie much of this

201

Re-Coding the Algorithm

chapter, in developing a way of discussing aberrant play beyond the connotations of cheating. As Johnson (2005) writes, “if you don’t think about the underlying mechanics of the simulation – even if that thinking happens in a semiconscious way –you won’t last very long in the game. You have to probe to progress”.

PLAYING THE GAME On starting up a new game, the player is often faced with various decisions to be made and actions to be learnt. This includes how to make their character move through the gameworld by pressing up, down, left or right on the joypad, using different keypresses or moving a mouse. The player learns how to pick up objects, move objects and how objects may be linked to the opening of new paths. Each successful understanding of a combination of actions, a way of jumping or running, or picking up an object, can lead to a reward of further gameplay experiences. These signs, deliberately installed within the game system, show the player that they are on the right path to succeeding (or even failing) at the game, and that the player is in fact experiencing the ‘purposeful play’ of the system. Purposeful play can be defined as the play intended by the game designer, and is linked to the goals and rules associated with the game by the outside designed source. Many videogames offer tutorial levels, or text-based tutorial guides at the start of the game, that act as a way of guiding the player along the intended game play of the game’s world. The purposeful play of the gameworld reveals itself to the player through these guided levels such as the preliminary level to We Love Katamari (2005). During this pre-level, the controls of the game are explained to the player through images depicting the controller. Instructions include how to complete certain moves using various control combinations to pick up objects in the game. There are no scores or time limits

202

within the level, it is instead filled with text-based rewards for using the right sequences in order to guide the Katamari and collect objects. This level is designed as an uncomplicated version of subsequent levels in order to act as a guide for what the player may come to expect of the rest of the game. The level expands as new control feedback loops are grasped, acting as a teaser for the timed, score-rewarded version of the game yet to come. Feedback loops familiarise the player with the purposeful play syntagm and reveal what is seen by many players to be the ‘norm’ of the game. Learning the skills needed in order to play Prince of Persia (2008), are also detailed at the beginning of the game. Once the introductory movie has ended, the player’s avatar is directed to follow a non-player character along the beginning paths of the gameworld. This shows the player how to move through the gamespace, and what keys to press in order to jump across platforms or run along walls. These actions are learnt through onscreen text detailing how to produce each manoeuvre and at what moment the action should start in order for the player to be successful. It is during these first few moments that any initial exploration of the gameworld appears to be temporarily stripped from the player as they are guided to learn the rules of moving in and playing through the game. Once again, the purposeful play of game is revealed through this preliminary level. The need to explore the game differently can be seen to fade into the background as the player is focused on the tasks that have been set for them. Initial levels such as these lead to the player familiarising themselves with the mechanics of the game and players learn what may be expected of them. Although these tutorials seek to reward the player for understanding the correct set of actions, videogames often contain deliberately installed fail signs for the player to recognise in connection with the learning process. Signs such as character life deterioration through missing heart icons, the loss of points, or the score counter decreasing and coins spilling out into the gameworld show

Re-Coding the Algorithm

the player how to build on and potentially correct these mistakes. The learning process starts again, allowing the player to learn how to keep their points and avatar’s life span. These reward/ failure feedback loops are about normalising the game’s “algorithm”. They consolidate the code of legitimate play through the player being rewarded by the purposeful play syntagms of the game paradigm. Both of these feedback loops permeate the syntagm of playing within the rules. This can be seen to constitute Caillois’ (1958) category of “ludus”, which he defines as “…rule-bound, regulated, formalised play”. It is through the learning process that the player starts to understand this particular syntagmatic of the game. The purposeful player is content to stick to the rules of the gameworld as revealed to them, to beat their high scores, to solves the hidden puzzles, unlock the next level or mission and hopefully complete the game. It is through these definitions that the purposeful player can be seen in terms of Perron’s (2003) category of the “gamer”, the player who is “…bound to the rules and limits of the game universe and gameplay”. This type of player only sees the one syntagm within the greater game paradigm. They feel no need for further exploration, or discovery of further syntagms, the pre-defined game world is enough to satisfy their experience. By submitting themselves to the rules of the gameworld, the purposeful player is further defined in terms of the invisible contract entered into on deciding to stick to the rules. In discussing play occurring within Huizinga’s “magic circle”, Salen and Zimmerman (2004) question the attitude of the player entering into the gamespace. In doing so, they raise the issue of Bernard Suit’s concept of the “lusory attitude” defined as “…a shared attitude towards the act of game-playing, an openness to the possibility of taking such indirect means to accomplish a goal”. Therefore, although there may be easier, more direct means of completing a game, the players of each individual

game submit themselves to the rules defined by the game they are playing. Salen and Zimmerman (2004) reiterate this point by claiming that Suit’s notion of the lusory attitude can be “…extended to say that a game is a kind of social contract”. It is this “social contract” that is adhered to by the purposeful player. To play outside of this contract would, for them, to be in breach of the rules of the game. It is this scenario that this chapter now seeks to explore, the possibilities of further syntagms of play that may be revealed to the player, through what may be seen by other players as breaking the rules.

BREAKING THE RULES In contrast to the purposeful player are those players that can be seen to intentionally disrupt the play of the game. In understanding the rules of the game, some players may decide to deliberately interfere with the game, by turning the “algorithm” on itself. This can be defined as what I term appropriated play. Appropriated play is acted out by aberrant players in an attempt to appropriate the play of the game’s world to their own individual means. The use of the term aberrant is defined through Eco’s (1965) concept of “aberrant decoding”. Here the text is decoded in a different way from that it was originally encoded. Therefore, aberrant play is not to be seen as a negative form of play, but as a way for the player to extend the syntagms of play not intended by the designer. It is not to say that all aberrant play can be seen in a positive light, as the syntagm of cheating still exists. However, the terms open up further motivations leading to the definition of two types of aberrant player, those that disrupt the game and those that try to contribute to the game. For the disruptive aberrant players purposeful play has become ‘perverse play’ as they seek to disrupt the state of the game inappropriately by implementing the rules perfectly. This is compa-

203

Re-Coding the Algorithm

rable to Salen and Zimmerman’s (2004) player type of the “spoil sport”, who acts as “…a player that refuses to acknowledge the authority of a game in any way. This nihilistic players do not hesitate to destroy the magic circle of the game”. This form of aberrant player is distinctly ludic as it is founded in a paradoxical denial of the rules while playing within them. In seeking to win the game, the player may cheat, turn off the game system, pause the game at a crucial point, or walk away mid-competition. Instead of playing strictly within the rules of the system, this player is disrupting the game, by breaking the rules, with no benefit to anyone but themselves. The disruptive aberrant player deliberately seeks to sabotage the game, which can be seen by purposeful players as not being playful. Although the player is recognising other rules outside of the clearer signs of the game world, they are not using them to any advantage in their exploration of the game’s system. In many ways, the disruptive aberrant player can be seen as the stereotypical cheat, the game player who hates to lose, and uses any means possible to win the competition when it is not going their way. Disruptive aberrant play can be seen within the cultural logic of the game. It can be seen amongst that groups of players, all seeking to turn the algorithm on itself, posting of YouTube clips showing off their disruptive play techniques in an attempt to become more accepted within the wider community. The knowledge surrounding how to play disruptively can be spread amongst the gaming community through online forums and word of mouth. In many instances all of these players are subsequently labelled as cheats, but are all cheats disruptive? Is it possible to see the aberrant player in a different light? It is for this reason that another category of aberrant play can emerge. Newman (2008) and Consalvo (2007) both discuss various characteristics that define a cheat. Consalvo in particular analyses how different players define cheating, ranging from whether it is acceptable to use a walkthrough as opposed to

204

cheat codes, and how cheating may differ between multiplayer and single player games. It is through the concept of the aberrant player, that the cheat can be defined once again. Alongside the disruptive aberrant player, there are also contributive aberrant players. These can be defined as players deliberately seeking to contribute to the game, by using techniques and gameplay styles that go beyond of what is usually seen as the norm of the game. Whereas cheats are often discussed with negative connotations, through their disruption of the game’s system, it is possible to see how some terms often associated with cheating may be better understood in terms of the contributive aberrant player. Maurer and Maurer (2007) state, “the act of playing is very close to exploring and redefining existing boundaries”. Therefore, instead of seeing one syntagm of the rules or goals within the game paradigm, it is possible for the player to start to recognise further syntagms within the game to explore. The player in this instance may start to move away from purposeful or ludic play to a form of exploration, which can, initially at least, be seen as paidic (from Caillois (1958) concept of “paidia”). Appropriated play is the act of discovery and exploration of the “algorithm” with the player trying to uncover more than the designed rules of the purposefully played game. Appropriated play is then acted out by both contributive and disruptive aberrant players. In analysing the motivations for appropriated play, a set of conditions or wishes can be identified. These include boredom or anxiety, a need to beat the system, or to improve gameplay. This may be through a response to poor design or a need to amplify fun or reward within the game. It is through these motivations that different types of contributive aberrant player can be discussed. Players may wish to demonstrate originality among their peer group through hacking or modding, and in doing so, try to gain the respect of the gaming community and recognition

Re-Coding the Algorithm

Figure 1. Motivations for different syntagms of play

for their achievements. It is these motivations that are outlined in figure 1 where it can be seen that all motivations of play can be connected to one another in various ways. One motivation may be linked in certain play experiences or as time progresses. Each motivation does not have to be seen as a single entity but instead can be seen as collective ideas working together as a way of discussing how players can experience the same game setting as an individual or group. Furthermore, types of motivation may evolve through discussions within the gaming community and an understanding of the exchange of ideas through the culture of gaming, strengthening the idea of the cultural logic surrounding game play. One of the first motivations outlined in figure 1 is based around Csikszentmihalyi’s theory of “flow”. “Flow” is defined by Csikszentmihalyi (1992) as the “way people describe their state of mind when consciousness is harmoniously ordered, and they want to pursue whatever they are doing for its own sake”. Although not originally developed as a theory for the videogame experience, it has been proposed by theorists such as Salen and Zimmerman (2004) and Juul (2005) that the concept of “flow” can be used as one way of understanding the player’s experience within

videogames. Salen and Zimmerman (2004) state that, “the connection between game design and the flow experience clearly appears in Csikszentmihalyi’s description of the components of flow, the conditions that make flow possible”. Whilst in the “flow state” the player is neither too anxious nor too bored. For the time they are in the state, the player is seemingly immersed within the gameplay and the quotidian reality fades into the background until the player decides to end the game, or encounters a level of difficulty above or below that which they have mastered. This break in “flow” may cause the once purposeful player to explore the game system further, leading them into the realm of appropriated play. Salen and Zimmerman recognise that not every player and or every game will reach or determine a state of “flow”. However, it is through an understanding of the flow condition that is it possible to see how players may start to explore further syntagms within the game’s paradigm. In their chapter Defining Play, Salen and Zimmerman (2004) state how “from a formal point of view, the rules of a game indeed constitute the inner ‘essence’ of a game. But there is a danger in limiting the consideration of a game solely to its formal system”. The authors continue to de-

205

Re-Coding the Algorithm

fine what they term “transformative play”, which can be linked to the idea of appropriated play currently unfolding within this chapter. “When play occurs, it can overflow and overwhelm the more rigid structure in which it is taking place, generating emergent, unpredictable results”. It is through ideas of “transformative” and appropriated play that it is possible to see how play can shift from the ludic to the paidic as the player starts to search for further unpredictability within the games system. This unpredictability may be as a result exploration due to boredom or through an unexpected glitch in the system.

BEYOND CHEATING As Newman (2008) notes, “a glitch, or ‘bug’ as it is sometimes known, is a generic term of the result of a programming error”. Whilst playing Lego Indiana Jones (2008) the player’s avatar can become stuck in the sand of the gameworld behind other objects. It may also be the case that buttons usually allowing entry to vehicles may in some scenarios stop working. It is instances such as these that can cause a forced restart of the game, for the player to resume and see the glitches disappear. Any effect of the glitch remains temporary for the purposeful player as they continue to try and beat the rules of the intended game play. On the other hand, the contributive aberrant player strives for these fortuitous discoveries and these are seen as opportunities for further exploration of the game’s system. It is through discovering new possibilities of play that a game can change and emerge into a new type of appropriated play for the player. It could be seen that a motivation for this is that although the player may have fallen out of a state of flow within the intended gameplay they paradoxically remain engaged with the system as a whole. Players often have an invested interest in the games they are playing. This can be seen in terms of the cultural logic of the game, through players not wanting to appear

206

to be foolish amongst their peers by not seeking to get the most out of the gameworld, and/or players wanting to purely get value for money through their game purchase. Returning to the game of Lego Indiana Jones, players can have various game states they wish to achieve. They may wish to unlock all the levels in a short space of time, and feel some sense of completion even though they may have only ‘officially’ completed 25% of the game. On having to replay parts of the game in order to complete 100% of the game’s missions, the intended play may start to become uninteresting for the player. The monotony of having to replay similar sequences of events in order to get all the artefacts and coins for each level may start to set in. In an attempt to make the level more appealing, and re-reach a state of ‘flow’, other new ways of playing may be found. The player may start attacking the other computer bot character working with them, or in a multiplayer scenario, the players may attack one another, and play mini-games of tag, or hide and seek. In some instances, finding the glitches hidden within the game may turn into a type of play in itself. Standing at certain places in the game world can cause players to get stuck in walls, or objects to become trapped between pieces of gameworld geometry. The other computer bot character can also become forced to animate on a loop as it becomes trapped due to the positioning of the player controlling the central character. What I term glitch exploration can take over as the player’s new task within the gameworld but to some extent this is often short-lived as the player then has to succumb to the rules of the level in order to complete it again. These smaller, playful objectives created by the player allow them to reach a new and different state of flow and gain reward for personal goals found and completed by these new types of appropriated play. Glitch exploration can lead to players becoming what Newman (2008) terms “glitch-hunters”. These are players that deliberately expose glitches commonly found within

Re-Coding the Algorithm

particular games. This type of play provides the player with a status for seeking out glitches, and recording these finds on online forums to gain reward and recognition for doing so, adding to the cultural logic of the game experience. Here the play of the game becomes distinctly ludic through recognition of the ‘glitch-hunting’ syntagm within the gameworld. Newman (2008) also links the finding of glitches to other types of play recognised within the gaming community, including that of “speed running”. Speed running is a type of play associated with players working out how to complete game levels in the quickest time possible and often posting the results online. This type of play can be linked to the player motivations in figure 1 of both trying to ‘beat the system (rules)’ as well as ‘amplifying gameplay/reward’. Speed-running is the resultant action of players seeking to define their own rules and create their own type of play through the discovery of this new syntagm. It can be found in YouTube videos of players recording their gameplay achievements, for example complementing laps of Mario Kart in the quickest time possible using short-cuts and exploiting weapons often used to harm other players. In discussing speed-running Newman (2008) notes how, “glitches, inconsistencies, and the undocumented features of the game are explored and exploited”. Here, glitch exploration in the early stages helps to work out the quickest path for the player and is vital in playing the game more quickly. It is through this shift from the paidic discovery of the glitch to the ludic confirmation of the resultant new game that players can also be seen to shift from the disruptive to the contributive. Once intentional disruption of the game may then lead to a new type of recognised play that contributes to the social understanding of the game community. Therefore in discussing player motivations for various types of play fund within the gameworld, it is possible to see how both Caillois’ categories of “paidia” and “ludus” can be discussed within the same game through a discovery of different syntagms.

In trying to separate ludic games into those played by “gamers” and more paidic games played by “players”, Perron suggest the player type of the ‘gameplayers’. According to Perron, the “gameplayer” is able to generate forms of paidic play within the ludic ruled games system. He discusses Grand Theft Auto III (2001) in relation to the gameplayer stating that, “what makes the success of such a driving-shooting-action-missionsimulation game is there is as much for the gamer that has to accomplish specific mission to do as there is for the player who wants to wander the city and just go on committing various criminal acts” (Perron, 2003). GTAIII is frequently used in the discussion of exploratory gameplay due to its seemingly expansive gameworld allowing for in-game missions along with player driven exploration. Exploring larger gameworlds, alongside the creation of in-game missions, can lead to the creation of other player-designed objectives. Indeed Bateman (2005) states in his essay The Anarchy of Paidia, that “Play is arguably always on a journey from paidia to ludus, although it would be wrong to think that it cannot also travel back towards paidia”. Players starting out within the game system may in many ways experience “paidia” during the early stages while trying to work out the rules of the game. Play can then oscillate between states of “paidia” and “ludus” during a cycle of learning. This oscillation can keep happening in various other explorations of the system. Therefore play in videogames does not have to be limited to either one of these states, but can balance somewhere between the two, or oscillate between varying degrees of each sate. Once again returning to Lego Indiana Jones, it is possible for players to wander around in ‘free play mode’ (or even ‘story mode’ if they so wish) in order to find personal missions. Players can experience these levels of paidic play within the ruled game, linking together Perrons’ “gameplayer” with the concept of the aberrant player. The player has to co-operate with the game’s system to some extent in order to experience this, and unlock

207

Re-Coding the Algorithm

further syntagms to experience the extended game paradigm. This is where sometimes the game system doesn’t open up to the player fully through an initial exploration, so new motivations call for them to explore it further, other than through the use of cheat codes. After the first inclusion of an ‘easter egg’ (a hidden, secret feature) in the game Adventure, the act of seeking this extra piece of code and what it may reveal to the player, turned into a type of play for some players. Consalvo (2007) details the histories of the ‘easter egg’, explaining that Warren Robinett, the creator of Adventure, decided to create an ‘easter egg’ in the game in order to give game designers and programmers the recognition he thought they deserved. Consalvo goes on to explain how players had to find a grey pixel hidden within one of the rooms of the game, and carry it to another location, to unlock a new room displaying Robinett’s name in flashing colours. Consalvo (2007) states: The first easter egg was a useless hidden bonus. It didn’t give you an extra life or allow you to change your appearance. It was just here, waiting to be found; nothing bad would happen if you never found it (p. 18). Although the first easter egg didn’t give the player any reward in terms of increasing high-cores or opening up a larger area to explore further, it still rewarded the player in terms of finding it, and opened up the notion of being able to explore extra hidden code within the game system. Nowadays, with more advanced technologies and larger game worlds to be explored, seeking the hidden easter egg has in many ways turned into finding out what else the game code will allow the player to do. This enables the player to move beyond the confines of purposeful play and situate themselves further within the gaming community. “Emergent gameplay” has often been discussed as a new way for the game system to be explored by players. As Kücklich (2007) states, “One of the most famous examples of emergent gameplay is

208

the possibility of using mines to climb walls in Deus Ex…”. This type of play arises within the original game system and shows the exploration of players trying to find new ways of manipulating objects for different means. Emergent gameplay can be seen as a form of contributive aberrant play, and highlights once again the oscillation between “paidia” and “ludus”. Out of free-form experimentation in the gamespace emerges a new syntagm, which contains its own sets of rules as defined by the player. It is then possible for this new ruled game to leak out in the social sphere of the gaming community, encouraging others to play this new game, or in fact seek other types of game/play syntagm within the same gameworld. Through types of “emergent gameplay” and a general exploration of the game’s system, other motivations of the player may lead to new areas of game creation or the creation of artworks from what they may find occurring in the game outside of the intended gameplay. Players may seek to use this knowledge, found through new types of appropriated play in other ways to expose the game as art as well as purely play (or indeed both). It can be seen that artists such as JODI and Brody Condon use the exploration of the algorithm as a way of exposing the system as a piece of art. According to Galloway (2006) these artworks cease to be game, as he writes, “…rather than a gesture of fandom – as Counter-Strike was to Half-Life – then, more often than not, the game loses its rule set completely and ceases to be a game after all”. Although the final outcome of the artwork may longer be a playable game by the community, the process of constructing the artwork emerges through the artist’s desires of aberrant play. The final product, although not playable, still adds to the gaming community as a way of promoting aberrant play and other processes possible through an exploration of the game’s underlying system. Condon’s own contributive aberrant play transforms the original game into an electronic art piece, such as in his work Suicide Solutions (2004). The work displays in-game characters

Re-Coding the Algorithm

using weapons also found within the game to turn on themselves in order to commit suicide. This is Condon’s new syntagm within the system; his own experience of using the game system to expose it’s other uses. In an interview with Andy Clarke (2007), discussing how he is inspired to create new artworks, Condon replies “Usually, I work with these visions from the netherworld that hit me during states of relaxed concentration, especially while playing games. I’m playing the game, I get bored, and I start experimenting, and some kind of image of action that resonates and makes itself clear”. It could be seen that once out of his ‘state of flow’ with the game Condon looks for other uses of the game system. It is here that experimenting with what the system can offer besides the intended rules of the game that artworks then start to form. Although this is not the intended game or play of the system, it highlights the game systems other uses and what the contributive aberrant player can then form out of discovering these possibilities during types of appropriated play. In contrast to creating artworks, other motivations for appropriated play can also be drawn upon, such as developments in hacking or modding games. What Kücklich (2007) refers to as “cheating” in his essay Wallhacks and Aimbots, is actually a way for the game system to be explored by players. He comments on codes such as “noclip”, used within first-person shooters which render walls obsolete as the player can now move through them, and “…thus be regarded as a means of laying bare the technological foundations of gamespce and of denaturalizing its representational aspects”. Once again, this is a way of players exploring the game paradigm, and re-constructing a new syntagm exposing the “algorithm” that gameplayers are not always used to seeing. This poses the question of where cheating stops and appropriated play begins or vice versa. Cheating can manifest itself in various ways as shown by aberrant players, but the connotation of cheating implies deceit and unfair advantage, using hidden codes in order to gain a

better score or open up areas of the game that the player could not solve by themselves. Therefore it can be seen that cheats and disruptive aberrant players are not interested in the workings of the algorithm or greater game paradigm. They are blinded by its syntagms of purposeful play and how to overcome them, whether in isolation such as a player using a code to gain infinite lives, or within a social situation disrupting the game to end the competition or using codes such as character invincibility to gain unfair advantage over the other gameplayers. However, cheating can be linked to appropriated play through the passing of knowledge between a contributive aberrant player to a disruptive aberrant player, especially in light of the understanding of the cultural logic of games. In order for game cheats to be found there needs to be aberrant players willing to deconstruct the algorithm to expose these further game traits. These players are contributive, adding new knowledge to the foundations of the system. By placing the knowledge online for other players, it is possible for these new syntagms of play to be picked up on by a disruptive aberrant player in the search to destroy the game in a social setting, for example, using a wallhack to their advantage in a multiplayer game scenario. Disruption may become contribution with this passing of knowledge. Game hacks can also be used for other means than cheating, using the resulting game patch for different reasons. Within the release of Doom III, players are able to use a flashlight in order to guide their avatar through dark corridors, where the zombies await them. However, the player’s avatar cannot carry both a flashlight and a gun at the same time, making the shooting of enemies, and the ability to see in the dark, an on-going problem in the swapping between each object or weapon. This activity disrupts the gameplay and exploration of many players and therefore a solution was found and posted online for other players to use. As Rehak (2007) notes, “…to many players, it is a game-disabling error on the level of a bug. There soon appeared a software 209

Re-Coding the Algorithm

patch…that allowed flashlight and weapon to be used simultaneously”. This modification to the game is not seen as a cheat, but as a welcome extra to players of Doom III to enable them to play the game without constantly changing controls. The player experience was fragmented by the original game conditions, therefore players felt motivated to make a patch for the game. The bug was recognised and then adjusted to suit the needs of the player and the Doom III playing community. Appropriated play, motivated by the need to improve gameplay, helped to address a problem concerning many gamers in a similar situation. The sense of community that has evolved through the use of the Internet in online gaming forums and online play itself has meant that the community surrounding individual games and their gamers are more accessible than ever before. The recognition of new types of appropriated play can be logged more easily through online gaming forums, social networking and websites such as YouTube that detail player’s speedrunning attempts or detailed walkthroughs of levels. This sense of community can also be linked to multiplayer games, and networked games such as Doom and Quake that allow for multiple players to play on the same game map at the same time. In a multiplayer environment, any appropriated play found by the contributive aberrant player can be easily shared and the naming of this new play syntagm can become apparent more easily through its recognition by all players. Discovery of what the game system can offer can also be recognised through the creation of maps for online games (or offline games that can be added to by Internet download). This can be seen through the game Half Life and the creation of the Counter-Strike map. Counter-Strike’s success amongst the Half Life and general gaming community meant that it is now available as a commercial release separate to its first origins as an added extra. The popularity if the game has since seen it being developed as a set of further sequels. Through exploring the algorithm, and gaining a greater understanding of the game system and its mechanics, players have 210

generated content for other players away from the original game designer’s content. The Internet has allowed for ‘player as designer’ in many instances with the growing availability of toolkits and world editors. Appropriated play therefore works by contributive aberrant players wanting to add further recognition of the “algorithm” for others to see. This itself generates new types of gameplay and rules for the players, for this process to then occur again. Appropriated play recognises the growing community of game players, and how modifying games to individual or group needs allows for different types of gameplay to emerge. Emergent gameplay is not something that is newly associated with videogames as a medium. Traditional card games show how the change of play, and variations in how the game can be played, has resulted in different instances of the same game. Games have also changed from their arcade beginning of beating high-scores such as in Pac-Man, or reaching the third level of Donkey Kong. The shift in the communication within gaming communities has allowed the player to become a designer in more ways through the growth in Internet communications, modding communities, and hints as to how it may be possible to play a game in a different way. As digital games are constantly growing and becoming embedded within our everyday lives, is there still a role for appropriated play or will more types of purposeful play be deliberately built into systems to meet to growing needs of the player?

CAN APPROPRIATED PLAY SURVIVE? Although the boundaries of games are seemingly changing through ubiquitous technologies, appropriated play will probably always play a role in how player’s can experience games. Locationbased games are increasing, through the use of GPS and mobile phone technologies, with artists such as Blast Theory creating social game scenarios existing in both virtual and real world spaces. Ian

Re-Coding the Algorithm

Russell, details his experiences of participating in Blast Theory’s Can You See Me Now? (2005) location-based game and how he tried to push the boundaries of his play encounters through the associated media. Can You See Me Now? involves the movement of players through a mixture of both a virtual and real life cityscape. In moving their avatar onscreen across the virtual landscape, players are attempting to dodge the ‘runners’, a team of opposition players, whose mission is to track down the other player’s avatar while running through real life city streets. The virtual world is accessible to the players, free from obstacles such as human and vehicle traffic. However, the runners have to dodge through the real life streets in order to find the virtual player in the same mapping to their real world positioning. Russell logs his own experiences within the game world and how he deliberately tried to push the boundaries between the real life and virtual world scenarios. Players can send text messages about the game to other participants that are also viewable by the runners. Russell (2007) decided to tease the runners through taunting them with his location through text messages, confusing them as to whether his direct messages could be believed or not. He notes, “The spirit of digital conceptual play won out as they merely assumed that I was asserting the location of my avatar – and not myself” (Russell, 2007). By being able to place himself physically as a player and his onscreen avatar in separate positions, the types of play experienced by the player may start to be mapped independently. It is in these situations that the boundaries of ludic play may start to blur, allowing for further types of play to emerge. In these instances the definitions of the aberrant player may need to be reworked further to encompass these new game scenarios. Russell’s experiment also raises the question of the boundaries of location-based gaming and whether pushing the boundaries of how to play these games is built in to the intended gameplay more freely than in purely screen-based videogames. As players are not as familiar with the

inner workings and rules of these emerging games, compared to other game scenarios, the learning process and initial probing of the “algorithm” may require more time. As Arsenault and Perron (2009) note, “It goes without saying that mastering new game mechanics is a learning process leading to better analytical and implementation skills”. This is especially true in location-based games, where the player does not have access to a large community base for information on previous games, in comparison to mass-produced screen-based videogames. Although these games are predominantly played in social settings, they are often limited to smaller groups that have access to them, therefore the need for walkthroughs or cheat codes are in many ways rendered obsolete. It is this quality of location-based games that allows for each game played to differ from the one before. In many ways the same can be said for the new wave of augmented reality games, such as Invizimals (2009) on the PSP. The game is based around finding creatures (the invizimals) in the player’s own environment, and collecting them in order to fight them against other creatures found within the game world. Even though much of the game will be shared amongst different players in terms of the creatures they find, and how to use each creature’s individual strengths, the location of the creatures will differ amongst players. This starts to make it harder to produce walkthroughs and share object-based information amongst a similar community of gamers, therefore some of the aspects that allow for appropriated play may start to disappear. Publishers are increasingly trying to find new ways for players to play games and tell stories through the medium of games. This can be seen through adverts for games such as Heavy Rain (2010) that promote it as having individual storylines for each player. Promises of personalised play experiences may start to draw some aspects of appropriated play out of games in the future but the commonality of types of shared experiences will no doubt still remain. It is this aspect of games, their social nature, and

211

Re-Coding the Algorithm

the exchange of ideas amongst a community that recognises similar cultural values associated with games that will always allow for some type of experimentation and appropriated play.

CONCLUSION Whereas designers map out how games can be played, through an understanding of the rules and associated feedback loops of progression, the purposeful play of videogames may not always be a straightforward experience. Unexpected glitches, boredom, and frustration of not being able to play purposefully, allows for play to be appropriated by the players for their own means. Each new, serendipitous discovery by the player allows play to become exploration and a sense of paidia to be experienced, however short-lived that may be. Instead of enforcing negative connotations of cheating onto certain players, understanding new types of play as being aberrant and contributive allows for a more positive understanding of different routes players may take through the gameworld. Growing communities of gamers with shared interests and understandings of cultural values allow for the development of new play types. These can then be added to other experiences and expanded upon in order to enrich new play experiences further. Whereas some players may see the disruptive aberrant player as a nuisance and a stereotypical spoilsport, others may be in awe of how they are seen to manipulate the game world for their own means. Here the disruptive player can be seen as contributive as they are adding a new dimension to the gamespace and new challenges for other players to improve upon. It can be seen that both purposeful play and play appropriated by aberrant players can only be seen in light of an understanding of the communities surrounding the games. To play purposefully is to commit to an understanding of how the game is acceptably played amongst a group with

212

shared values surrounding that particular game. This is also true of play that is appropriated by those wishing to re-encode the algorithm for their own devices. This knowledge is usually shared as a way of showing others what else the game’s “algorithm” is capable of, and giving the player their own sense of place within a community of similar players. The discovery of glitches, speedruns, cheat codes, and game hacks are all part of what Consalvo (2007) states to be “gaming capital”. This is information to be traded, discussed, expanded on and shown amongst each player’s peers as they strive to reach new levels of play and experience within their own understanding of the game. Gaming communities are growing, through the use of the Internet and social media. Alternate reality games rely on these communities for knowledge to be shared amongst players in different countries, across different time zones. Now that these communities are growing, it may be possible to see how in the contributive aesthetic of probing the game’s “algorithm” is starting to evolve, further questioning whether the aberrant player may need to be redefined once again in years to come.

ACKNOWLEDGMENT This work is based on an earlier work: “Grand Theft Algorithm: purposeful play, appropriated play and aberrant players”, in MindTrek’08, October 6-9, 2008, Tampere, Finland. Copyright 2008 ACM 978-1-60558-197-2/08/10,

REFERENCES Arsenault, D., & Perron, B. (2009). In the Frame of the Magic Cycle: The Circle(s) of Gameplay In Perron, B., & Wolf, M. J. P. (Eds.), The Video Game Theory Reader 2 (pp. 109–132). New York: Routledge.

Re-Coding the Algorithm

Bateman, C. (2005). Only a Game: The Anarchy of Paidia. Retrieved 14th May 2007, from http:// onlyagame.typepad.com/ only_a_game/2005/12/ the_anarchy_of__1.html Caillois, R. (1958). Man, Play, Games. Chicago, IL: Illinois University Press. Chandler, D. (2002). Semiotics: The Basics (2nd ed.). London, UK: Routledge. doi:10.4324/9780203166277 Clarke, A. (2007). An Interview with Brody Condon In Clarke, A., & Mitchell, G. (Eds.), Videogames and Art (pp. 85–93). Bristol: Intellect Books. Consalvo, M. (2007). Cheating. Cambridge, MA: MIT Press. Csikszentmihalyi, M. (1992). Flow. London, UK: Rider. Eco, U. (1980). Towards a Semiotic Enquiry into the Television Message In Corner, J., & Hawthorn, J. (Eds.), Communication Studies (pp. 131–149). London, UK: Arnold. Frasca, G. (2003). Simulation versus Narrative: Introduction to Ludology In Perron, B., & Wolf, M. J. P. (Eds.), The Video Game Theory Reader (pp. 221–236). London: Routledge. Galloway, A. (2006). Gaming: Essays on Algorithmic Culture. Minneapolis, MN: University of Minnesota Press. Johnson, S. (2005). Everything Bad is Good for You. London, UK: Penguin Books. Juul, J. (2005). Half-Real: Video Games between Real Rules and Fictional Worlds. Cambridge, MA: MIT Press. Kücklich, J. (2007). Wallhacks and Aimbots In von Borries, F., Walz, S., & Böttger, M. (Eds.), Space, Time, Play, Computer Games, Architecture and Urbansim: The Next Level (pp. 118–131). Basel: Birkhauser.

Manovich, L. (2001). The Language of New Media. Cambridge, MA: MIT Press. Maurer, M., & Maurer, N. (2007) The Uninhibited Freedom of Playfulness. In von Borries, F, Walz, S, & Böttger, M. (Eds), Space, Time, Play, Computer Games, Architecture and Urbansim: The Next Level. (pp.352-353). Basel, Switzerland: Birkhauser. Newman, J. (2008). Playing with Videogames. London, UK: Routledge. Perron, B. (2003). From Gamers to Players and GamePlayers: The Example of Interactive Movies In Perron, B., & Wolf, M. J. P. (Eds.), The Video Game Theory Reader (pp. 237–258). London, UK: Routledge. Rehak, B. (2007). Of Eye Candy and ID: The Terrors and Pleasures of Doom 3 In Atkins, B., & Krzywinska, T. (Eds.), Videogame, Player, Text (pp. 139–157). Manchester, UK: Manchester University Press. Russell, I. (2007). Critical Studies in New Media: Now, I can see you. Retrieved 22nd October 2009, from http://humanitieslab. stanford.edu/44/278 Salen, K., & Zimmerman, E. (2003). Rules of Play: Game Design Fundamentals. Cambridge, MA: MIT Press.

ADDITIONAL READING Bogost, I. (2007). Persuasive Games. Cambridge, MA: MIT Press. Brown, H. J., & Oren, M. (2005). Living Art: Commercial Modding and Code-Illiterate Gamers In Garrelts, N. (Ed.), Digital Gameplay (pp. 146–159). London, UK: McFarland & Company Inc.

213

Re-Coding the Algorithm

Huisman, J., & Marckmann, H. (2005). I Am What I Play: Participation and Reality as Content In Rasessens, J., & Goldstein, J. (Eds.), Handbook of Computer Game Studies (pp. 389–404). Cambridge, MA: MIT Press.

Ryan, M. L. (2007). Beyond Ludus: Narrative, Videogames and the Split Condition of Digital Textuality In Atkins, B., & Krzywinska, T. (Eds.), Videogame, Player, Text (pp. 8–28). Manchester, UK: Manchester University Press.

Juul, J. (2007). Without a Goal: Open and Expressive Games In Atkins, B., & Krzywinska, T. (Eds.), Videogame, Player, Text (pp. 8–28). Manchester, UK: Manchester University Press.

Simon, B. (2006). Beyond Cyberspatial Flaneurie. Games and Culture, 1(1), 62–67. doi:10.1177/1555412005281789

King, G., & Kryzwinska, T. (2003.) Gamescapes: Exploration and Virtual Presence in Game Worlds. Paper presented at DIGRA 2003, Utrecht University. Klabbers, J. H. G. (2006). The Magic Circle: Principles of Gaming and Simulation. Rotterdam: Sense Publishers. Kücklich, J. (2007). Homo Deludens: Cheating as a Methodological Tool in Digital Games Research Convergence, 13(4), 355–367. Kücklich, J. (2009). A Techno-Semiotic Approach to Cheating in Computer Games: Or How I Learned to Stop Worrying and Love the Machine. Games and Culture, 4(2), 158–169. doi:10.1177/1555412008325486 Manovich, L. (2008). Softbook. Retrieved from www.softwarestudies.com /softbook Mäyrä, F. (2008). An Introduction to Game Studies: Games in Culture. London: Sage Publishing. Newman, J. (2002). In search of the videogame player: The lives of Mario. New Media & Society, 4(3), 405–422. Nielsen, S. (2008). Understanding Video Games: The Essential Introduction. New York, NY: Routledge. Rushkoff, D. (2005). Renaissance Now! The Gamers’ Perspective In Rasessens, J., & Goldstein, J. (Eds.), Handbook of Computer Game Studies (pp. 415–422). Cambridge, MA: MIT Press.

214

Sykes, J. (2006). A Player-Centred Approach to Digital Game Design. J. Rutter & J. Bryce (Eds.) Understanding Digital Games (pp.75-92). London: Sage Publishing. Winston, B. (1998). Media, Technology and Society. Oxon: Routledge.

KEY TERMS AND DEFINITIONS Aberrant Player: A player who seeks to appropriate play. Algorithm: The inner workings of the game, seen at surface and/or code level. Appropriated Play: Play that arises outside of the ruled play intended by the designer. Contributive Aberrant Player: A player who seeks to add extra elements to the game through finding new types of appropriated play. Cultural Logic: Recognition of the wider culture of games and gameplayers. Disruptive Aberrant Player: A player who is seen by others to deliberately disrupt the rules of the game. Glitch: A bug found within the game system, usually a temporary fault in the code. Purposeful Play: Play intended by the game designer.

Section 4

Rising Principles in Virtual Communities, Mediated Social Interaction, and Digital Community Networking

216

Chapter 12

Exploring the Ecosystems and Principles of Community Innovation Andrea Botero Aalto University, Finland Kimmo Karhu Aalto University, Finland Sami Vihavainen Aalto University, Finland

ABSTRACT In this chapter, we explore some of the contemporary configurations of what we will refer to as community innovation. We probe the relevance of the phenomena by illustrating and comparing the digital ecosystems that surround some communities that innovate together in a world of social media and Web 2.0 tools. In particular, two cases are used to illustrate the arguments: a collective venture for designing electric car conversion kits (eCars – Now!) and a looser collective representing the development ties of LEGO® user groups with the firm. These cases are presented through their existing ecosystem and communication tools and the ways in which their stories challenge linear and individualistic models of innovation. We argue that, for these communities, configuring and constructing an appropriate set of communication tools and new media seem critical in negotiating a place for themselves between grassroots cultural innovation and corporate control. In doing this, we also suggest some social principles that drive community innovation practices, as they are present through our examples. DOI: 10.4018/978-1-60960-774-6.ch012

Copyright © 2012, IGI Global. Copying or distributing in print or electronic forms without written permission of IGI Global is prohibited.

Exploring the Ecosystems and Principles of Community Innovation

BEYOND THE FIRM: AN ALTERNATIVE LOCUS OF INNOVATION1 The classical, manufacturing-centred model of innovation that has prevailed for most of the past several decades assumes that an innovation starts – usually – from insights created within a firm’s research and development unit (Godin, 2006). These innovations are then developed into a product (offering), marketed, and later ‘diffused’ to end users (Rogers, 2003). From a managerial point of view, the classical manufacturing-centred model has had several salient implications. When it comes to the activities of those involved with new product and service development, the main concern has been to keep the process strictly controlled between the boundaries of the firm (closed) and assert it as the realm of experts. When it comes to the rest of us, one immediate consequence has been to believe that it is mostly lonely experts, usually within firms, who hold a monopoly in the innovation and design processes of new technology. This happens despite the fact that most of our everyday life experiences can account for innovation as, mostly, a collective endeavor of making sense of both technology and practices at the same time (Tuomi, 2003). In this sense, the aforementioned classical producer or manufacturing-centred model has shown its limits in accurately describing how innovations actually unfold. In recent decades, research in such diverse fields as science and technology studies (Bijker, Hughes, & Pinch, 1989), innovation management (Von Hippel, 1998), marketing (Prahalad & Ramaswamy, 2004), design (Greenbaum & Kyng, 1991), and media studies (Jenkins, 2006) have shed light on new understandings of innovation as a distributed, non-linear, and dynamic process. It has also become increasingly clear that these processes involve changes at different stages (not only in technology) and that there are more active roles for stakeholders, such as audiences, users, sup-

pliers, and customers, who previously have been assumed to embody mainly reactive roles. While the field of media studies can be recognized as an early mover in recognizing this shift in grassroots innovation culture, particularly when it refers to new ways of content creation, in this chapter, our focus will be on similar discussions that are ongoing in business and innovation management. One interest is to challenge the pervasive belief in the position of companies as the places where innovation takes place. This is evident, for example, in current management approaches that advocate distributed processes of innovation and seem to contain inherent biases towards the role, benefits, and policy implications as they pertain to firms and corporations (Nachira, 2007; West, 2009). It has been argued that one of the most important challenges for innovation research is to enhance collective capacities to participate in innovation processes and, even more importantly, collective capacities to participate in the governance of such processes (Callon, 2004; Thirft, 2006). The problem seems to be one of achieving cooperation between groups who share some interest but might also have conflicting and often antagonistic agendas and motivations. According to Callon (2004), the notion of community (as described and elaborated on by authors such as Knor Cettina, Laven, and Wenger) could be at the center of new forms of organization that are experimenting with these processes. In explaining his point further, Callon argues: Talking of community means giving up the myth of the brilliant individual innovator and inventor. It means recognizing that users or consumers who express their preferences are not isolated but caught up in social networks. It is collectives that invent, design, develop and use innovations. In fact, more and more often, the same collectives simultaneously take care of all these activities. In order to do so they combine the competencies of different actors. These collectives also contain technical devices and in particular systems of

217

Exploring the Ecosystems and Principles of Community Innovation

communication without which they would be ineffective. In short, these strange melting pots are a mix of humans and non-humans. I will use the words “communities” or “hybrid collectives” to denote these new actors of innovation. (Callon, 2004) Building on some on these insights, we will elaborate on the idea of community innovation (Botero, Vihavainen, & Karhu, 2009) as a viable and already existing form of organizing cultural production in order to understand better the possibilities and challenges that lie ahead. We do this by exploring two cases. The first case collects insights from a collective venture whose main aim is creating electric car conversions, grouped around the self-organized e-Car Now! Community. The second case draws from a more loosely tied assembly formed by the adult user communities of LEGO bricks and some of their development ties with the actual firm. In this chapter, we take three angles to analyze and compare the cases and structure the material. First, to get a more heterogeneous view on the phenomena, we map the context, actors, and particular communication infrastructure visible in each of them using some concepts and ideas from the perspective of Digital Business Ecosystem research (Nachira et al., 2007). Second, we look closer into their iterative configuration of particular communication infrastructures and related tools by looking at them in the light of some concepts borrowed from the Computer Supported Collaborative Work (CSCW) tradition (Baecker, 1995). Third, based on the previous insights, we suggest some of the social norms or “principles” that drive community innovation practices in a world of social media and Web 2.0 tools. This is done by contrasting the differences in motivations and attitudes that are present in the cases chosen as they relate to other alternative modes of innovation approaches that have been suggested elsewhere. We conclude our chapter with some reflections and suggestions for future work.

218

ALONE OR TOGETHER? A LOOK AT SOME COMMUNITIES THAT INNOVATE Networked media has drawn our attention to new forms of collective organization and collaborative practices that have made possible collective and distributed innovation. Though the phenomenon has been mostly discussed with respect to practices in free (libre) and open-source software programming (Gay, 2002; Tuomi, 2003), fandom, and media content production (Jenkins, 2006), similar examples have also been identified in more consumer-oriented fields such as sports equipment development (Franke & Shah, 2003. We decided to employ a comparative case study analysis (Yin, 2002) focusing on the multiple evolving elements and relationships of two particular communities. These communities are engaged both in collective endeavors with some shared goals and schemes of collaboration that span both face-to-face and online activities. Both communities coordinate, document, and spread the word about their projects using mostly digital media; however, their creations are not limited to it. The descriptions and visualizations that we created correspond to a snapshot in time of the infrastructures, tools, and achievements of these collectives as of November 2009. We looked at their online activities and the development of the related Web sites. We also made inventories of the tools and strategies mentioned and linked in their sites. We complement this data with accounts from the popular press and previous research; in particular, the case of LEGO has been documented before and can be considered a more established community. In the case of the eCars-Now! community, we have also had the chance to hear presentations from key members of the community. Our view is that both communities are highly distributed and interested in engaging more people, so it is in their interest to keep their dependency on the Internet itself as transparent, up-to-date, and pragmatic as possible. While we are aware of some of the

Exploring the Ecosystems and Principles of Community Innovation

limitations that might occur with this approach, our interest is in discussing the phenomena as it is made visible and possible through new media, so we consider this a good starting point. We will now briefly introduce the cases.

“Everyone is Invited to Take Part in the Future”: eCars – Now! According to the movie Who Killed the Electric Car?, during the late 1990s, many big car companies launched electric cars to the market. However, the launch flopped and electric cars were almost forgotten until recently. Today, the development of electric cars is back. For example, the biggest newspaper in Finland, in a special issue of their car pages printed in August 2009, listed at least six interesting electric car projects from big car manufacturers around the world (Lautsi, 2009). From the perspective of this chapter, however, something has changed. This time, ordinary citizens are also making contributions through a myriad of ad-hoc and organized projects. One of the most interesting ventures in this area comes from a community project started in Hikiä, Finland, called Sähköautot – Nyt! (eCars – Now!) (www. sahkoautot.fi). The initiative started as a project involving a few enthusiasts interested in getting an electronic car. Today, the community has expanded to the whole country and there has been interest in other countries as well. Local groups have been established in places such as India, Turkey, Spain, Latvia, Denmark, and the United States of America, and a new global community Web site in English recently was established (20102010Sähköautot – Nyt!, 2010!!). After many experiments, benchmarking projects around the world, and brainstorming together, they have ended up framing the project goal as follows: “How can electric cars be massproduced using modern technology conversions at an affordable price?” (20102010Sähköautot – Nyt!, 2010!!). Based on that idea, the community is engaged in creating open-source conversion

plans for some of the most popular car models. Their first working prototype has been labeled “eCorolla,” launched globally at an automobile show in November 2009. Their Web site condenses their plan “for greener, electric driving” into six simple steps: 1. Create a conversion plan for a gasoline car. 2. Find buyers for the car. 3. Organize a group purchase. 4. Get the cars on the road. 5. Share the conversion plans. 6. Start over again with another model. In general terms, their proposition is to produce e-cars by updating new or slightly used cars into zero-emission vehicles. The process includes replacing the gasoline engine with pinnacle electric technology, and the assembly consists of a high-quality permanent magnet motor, stateof-the-art lithium batteries, and a computer with an Internet connection controlling the system (20102010Sähköautot – Nyt!, 2010!!).

There is More to LEGO than Bricks: LEGO User Communities LEGO is a Danish company that has been producing toys for children’s creative play since the beginning of the last century. The most successful innovation of the company is considered the LEGO construction bricks, which have been in production since 1940. Today, LEGO bricks are considered a classic toy in the West and the epitome of creativity and modularity. Their popularity and good quality have made them a part of generations of children’s lives as they have engaged with the bricks beyond their childhood age, surrounded by an important subculture that supports many activities and by-products (Bender, 2010). Today, there are several online hubs where users and fans of all ages discuss, create, design, and do business around LEGO products. These are not necessarily isolated projects; sometimes they are collective ventures that organize and cultivate these activities through more permanent infrastructures (see e.g. Lehman, 1997).

219

Exploring the Ecosystems and Principles of Community Innovation

On the other hand, LEGO group’s experiments with user involvement have been widely discussed in the popular press and academic discourse as representative cases of such issues as co-creation (Prahalad & Ramaswamy, 2004), mass customization (Prospero, 2007; Piller et al., 2005), and lead user involvement (Koerner, 2006). Commentators seem to agree that these moves have been the basis of the company’s rebirth in the last decade (Seybold, 2006; Beloe, 2006; Koerner, 2006). These stories are usually told from the perspective of the company and relate how they have managed to tap into users’ creativity. Recent discussions have, for example, centred on the services that the firm has created in order to engage more with its users. However, a closer look at the history of the tools and practices of user groups show how user communities pioneered and actually developed some of these services decades before the company and how those ideas are chosen by the firm at distinct moments (Lehman, 1997Piller et al., 2005). LEGO has, without a doubt, been able to internalize and, sometimes, facilitate, some of the innovations developed in the user communities into their own offerings and vice versa because, around the brick, there exist not mere users but vibrant and active collectives that do with the bricks much more than what LEGO alone could ever think is possible.

DIGITAL BUSINESS ECOSYSTEMS: STAKEHOLDERS, RESOURCES, AND THEIR RELATIONSHIPS In this section, we will, first, introduce the concepts of the Digital Business Ecosystem, digital ecosystems, and the business ecosystem to serve as a framework in which we can illustrate the “ecosystems” of the two cases presented in the previous chapter. After that, we will present illustrations for both ecosystems and discuss them. We use the definition of Digital Business Ecosystem (DBE), developed by Nachira et al.

220

(2007), who saw DBE as socio-economic development catalyzed by ICTs emphasizing co-evolution between the business ecosystem and its partial digital representation: the digital ecosystem. In their view, there are two separate layers in the Digital Business Ecosystem: the digital ecosystem and the business ecosystem. The digital ecosystem is the technical infrastructure used to connect to the services and information over the Internet and to enable networked transactions. We will explore this further and analyze the set of tools and experiences enabling communication and collaboration within the community and with other parties in the forthcoming section titled Community Constructed Communication. The business ecosystem refers to the ecosystem of companies, goods, and services. The business ecosystem concept was originally developed by Moore (1993) when he suggested “that a company should be viewed not as a member of a single industry but as part of a business ecosystem that crosses a variety of industries”. Furthermore, in his book, Moore (1996) recognized individuals, in addition to organizations, as the “interacting organisms” of the business world. To get a concrete understanding of the two studied cases and to make it easier to compare the ecosystems around them, we have created a visual model of both ecosystems (Figure 1 and Figure 2). This way of illustrating the ecosystems relies on our earlier research (Karhu et al., 2009). In the following sections, we will explain the main elements of the visual model. First, for the business ecosystem side, we have included in the visual model all formally organized actors (companies or firms) that are somehow involved in the case. Users, the “interacting organisms”, are illustrated with characters that interact with companies and services. It is important to recognize that active engagement and participation is not, typically, equally distributed, and, therefore, there are several different roles for users that might change according to time and interest. For instance, Nielsen (2006) famously claimed that, “in most

Exploring the Ecosystems and Principles of Community Innovation

Figure 1. DBE for eCars – Now! community2

Figure 2. DBE for LEGO user communities and the LEGO company

221

Exploring the Ecosystems and Principles of Community Innovation

online communities, 90% of users are lurkers who never contribute, 9% of users contribute a little, and 1% of users account for almost all the action”. We have adapted this insight and illustrated some of these possible roles in the DBE as follows:

of the interaction (for example, see the “Trade models and pieces” interaction in Figure 1).

General public (“lurkers”) represented by light gray and small characters LEGO and eCars users (“contribute a little”) in a darker color but a small size LEGO and eCars creators (“account for almost all the action”) in a bright color and a large size Entrepreneurs (emerging role) are illustrated with a dark grey color and large characters to illustrate smaller actors that create business in the ecosystem by supplying materials or supporting services.

In the case of the eCars – Now! community, the core members of the project are geographically dispersed volunteers interested in addressing an environmental project through a very pragmatic approach. At least in Finland, they meet faceto-face around local garages to experiment and develop the conversion kits in what is not only a work endeavor but a social and fun encounter. A large bulk of the work, coordination and documentation, happens online when it best suits the participants. Their vision includes entrepreneurship roles for small garages that will produce the parts and offer conversion services. To support these spin-offs, they are committed to the free sharing of information and documentation. The community is also very straightforward and clear about the transparent management of new alliances and the ways in which community resources are managed and how decisions are being made (Räsänen, 2009). Figure 1 depicts the various stakeholders involved in the Digital Business Ecosystem of the eCars – Now! community. On the top right corner, we have the eCars – Now! community, consisting mainly of the core people associated with the project and the various services that the community provides. The media tools that they use to coordinate their activities articulate the community. In between these two groups, we can find a series of electric car “entrepreneurs” that represent the small garages that will take care of the actual electric car conversions and all others that could produce and sell the required parts. They might be an active part of the community, contributing to the development of the kits, but others could be only peripheral participants that take advantage of the open nature of the development. Those people interested in buying an electric car

• • •

•

In reviewing the cases, we identified three rough clusters: •

•

•

Company-controlled includes big corporations and the services that they provide (illustrated with a blue circle). Community-driven largely features the more grassroots initiatives of our interest and the services that they provide (illustrated with an orange circle). Media/Web 2.0 shows the media and Web 2.0 – companies that can be identified in the ecosystem (illustrated with a gray circle)

Second, for the digital ecosystem side, we have included in the visual model the most relevant tools and digital services we found that are used and/or developed by the users and communities. These are illustrated using server-like symbols that have a small globe emphasizing that a service or tool is available over the Internet (for example, see Wiki in the community-driven group in Figure 1). Finally, the basic interactions amongst the stakeholders are illustrated using color-coded arrows. Each arrow includes a descriptive label

222

Communities’ DBE at a Glance: eCars – Now! and LEGO Users

Exploring the Ecosystems and Principles of Community Innovation

are illustrated in the bottom-left corner as electric car “users”; they are also actual or potential members of the core community and can acquire other roles. Finally, in the bottom-right corner, we also have the general public that follows the project both through the community channels or more institutionalized ones; they are fans and supporters of the initiative. The “Media/Web 2.0” cluster consists of media services that are not a part of the core community but report about the project. Also, services such as YouTube, which are taken advantage of by the community to spread the word and document their advancements, are included to account for the many dimensions of creativity that are involved. In the top left corner, we have the traditional car manufacturers that form a part of the “companycontrolled” cluster, which are not yet in contact with this community but, nonetheless, provide the framework for their activities (original cars). In regard to their own interactions with car users, some car manufacturers have recently made some efforts to host and try nurturing platforms where users (their current car owners) can discuss and share information. This has focused more on brand-building through discussion forums and experience-sharing (pictures, videos, and stories). It is worth mentioning that regulatory bodies play an important role in this case, as safety constraints are important issues in car manufacturing and transportation, and this might frame the feasibility of their strategy in the long run. In the case of LEGO user groups, it is fair to say that, thanks to the Internet, these distributed and sometimes loosely coupled communities have become more visible and connected; as a matter of fact, one of the first USENET forums was centered on LEGO brick enthusiasts (Lehman, 1997). This scaled up the range of “sub-products” of the bricks and spread user practices even more. For example, it has been common practice in fan and user communities to create custom-made models; this is supported by ad-hoc flexible marketplaces offering secondhand and reused bricks. The marketplace

is linked to catalogues made by collective effort that invent the contents of the packages in order to locate important pieces and build collective knowledge. Another collective effort since 1995 created shareware CAD-equivalent software and extensions for LEGO brick models that supported better the construction of virtual models (by adding possibilities for generating detailed lists of parts and, sometimes, instructions3). Furthermore, user groups are also actively involved in creating media content using bricks, which is actively shared and remixed online (e.g. animations). In Figure 2, we can see some of the various stakeholders involved in the Digital Business Ecosystem for the LEGO company and LEGO user communities. In the top-right corner, we have the LEGO company and some of the online services they provide to interact and engage with their users. Those are clustered around “companycontrolled”. In the top-left corner, we have groups of LEGO “users”, people who normally buy and play with the sets but are not necessarily enthusiasts. On the left, we have the general public, those do not necessarily buy, play with, or engage with LEGO sets but know about it and, for example, may watch some fun animations created using the bricks available from video-sharing services such as YouTube. In the bottom-right corner, we have an example LEGO user community hub, consisting of engaged fans and members along with some of the tools that they use to communicate with each other.4 The “Media/Web 2.0” cluster that is illustrated in the center of the figure includes mostly media and other third-party services that are taken advantage of by the community to promote and document their creations or that otherwise serve “the general public”. In both of our DBE illustrations, we have included a blue dashed arrow from the “communitydriven” area towards the “company-controlled” area. Provocatively, we have called this arrow “Diffusion of innovation” to illustrate the transfer of ideas and innovations that also happens from communities to companies. Rogers (2003) defined

223

Exploring the Ecosystems and Principles of Community Innovation

diffusion of innovation as “the process on which an innovation is communicated through certain channels over time among the members of a social system”. Typically, the direction for such diffusions has been understood as they happen from producers (companies) to consumers (users). However, in our cases, we can see or expect that it will happen in the opposite direction as well, with some interesting consequences. For example, in the LEGO case, the community was the first to develop the idea of building digital models and even developing the LDraw design software. After almost a decade, the company picked up on this practice and made the activity a part of their offerings as the LEGO Digital Designer software, and the same can be said for most of the other services. For the eCars – Now! case, it still remains to be seen whether car companies will adopt or draw inspiration from some of the ideas developed by this community. However, it is possible to posit some scenarios in which other companies might eventually become interested in the activities and results of such tinkering communities as eCars – Now!

COMMUNITY CONSTRUCTED COMMUNICATION In this chapter, we will examine our community innovation cases from the point of view of communication technologies (e.g. email, wikis, social

networking groups). We will discuss the roles of communication tools in supporting community innovation, as well as how the community has, itself, evolved a suitable set of technologies to get organized and support the interaction among the stakeholders. We begin by presenting two taxonomies of CSCW5 systems that appeared in Ellis (1991). First, the taxonomy of time and space is the classic CSCW matrix that first appeared in Johanssen (1988) (Figure 3). The CSCW matrix presents four categories of groupware.6 To be comprehensive a groupware system should be able to fulfill all four categories. It should support face-to-face interaction, synchronous distributed interaction happening at the same time between different locations, asynchronous interaction happening at the same place but at different times, and asynchronous distributed interaction happening in different places and at different times. As Ellis (1991) pointed out, this matrix is a simplification but still demonstrates the variable demands for a groupware system. Second, the application-level taxonomy presents groupware systems from two dimensions, the common task dimension and the shared environment dimension (Figure 4). We use this taxonomy to illustrate the spectrum of groupware systems. Groupware systems can vary in the common task dimensions. This means that there are systems that support groups that do not have a tight common task, such as timesharing of indi-

Figure 3. CSCW time-space matrix (adapted from Ellis, 1991)

224

Exploring the Ecosystems and Principles of Community Innovation

viduals with their individual tasks, and there are systems that support groups that do have a tight common goal, such as a group that is working together to review a software system. Groupware systems can also vary in shared environment dimensions. This dimension shows the system’s functionality in providing information, such as information about the individual participant’s state, social contexts, or the current state of the project. Next we will walk through our cases, eCars – Now! and LEGO, to show how today’s community innovation ecosystems need a diverse set of groupware technologies to support their interaction efficiently. It is worth remembering that our capture of the eCars – Now! and LEGO cases are snapshots in time. Both communities are changing continuously. In the ecosystem figure of the eCars – Now! (Figure 1), we can see how the community has to communicate inwardly and with various stakeholders outside of the core participants. Inwardly, the core activists of the project have the common task of developing the instructions for electric car modifications. The main development is located in one place, Hikiä, Finland. Thus, the core people

are, very often, in the same physical location with each other, but a lot of activity happens online and may migrate more as new contributors are recruited. Outside the core community, there are stakeholders such as electric car users, entrepreneurs, the media, and the general public. These stakeholders have various tasks independent from others and do not necessarily have a common task with the core community. Of course, many might support their task but are not always core contributors. To address these needs, the eCars – Now! community has constructed its own toolbox for communication. There are, for example, several kinds of blogs (e.g. the project monitoring blog, the visitor blog), a wiki (an open information bank on electric cars), several mailing lists (e.g. the active list, the monthly letter), an international RSS stream of electric car information, and the #sahkoautot.fi IRC channel. The base of their public communication is centred on the Web site (www.sahkoautot.fi) and the Wikidot service. The latter is a free service that allows Web site-building, content-publishing, and document-sharing. The sahkoautot.fi site contains explicit links to certain tools, which are either hosted by the community

Figure 4. Two dimensions of groupware spectrum (adapted from Ellis, 1991)

225

Exploring the Ecosystems and Principles of Community Innovation

itself or outsourced to a free service or other provider, depending on the circumstances. Their infrastructure consists of: •

•

•

•

•

•

•

226

Forums: discussion board groups concerned with the project and about electric cars generally where members share and collect information and knowledge. Blogs: there are several blogs, the project blog (Projektiseuranta), where it is possible to monitor the progress of the project; the visiting guest blog (Vieraskynä), where participants, usually celebrities, write their thoughts about electric cars; and the activist blog (Puuhamiehet), where the core contributors of the community blog about their personal thoughts. International RSS feed: an RSS feed syndicating content from several international green car blogs, such as Plug and Cars, Jalopnik, and so on. Wiki (SähköautotWiki): an open data bank about electric cars and the official repository of the project’s results. Free for anyone to read. The document workshop (Paja): basically, a wiki page (Wikidot) where the core contributors collaborate on various aspects of the project. Activities include writing invitations for tenders, contracts, and discussions about the technological solutions used in electric car modifications. IRC channel: an Internet relay chat channel for discussing electric cars and coordinating some activities of the members. Mailing lists: several mailing lists are used, including the activity list (Aktiivilista), a mailing list for general discussion about the project; the monthly letter list (Kuukausikirje), a mailing list for getting a monthly project report; the technology list (Tekniikka), which contains information and instructions on performing the actual modification from a fuel-based car to

•

•

an electric car; and the announcement list (Tiedotus) for announcements for project activists and for public relations. Facebook group: a channel used mainly for publishing news about the project to those interested in following the developments. Severa project management software: commercial project management software, which the community has gotten through the sponsorship of Severa Corporation. Severa seems to have been used at least during the initial project planning, designing project phases, and dividing tasks into sub-categories (Severa, 2010).

Their myriad communication tools demonstrate the need for myriad ways of interacting with the community. The openness of the community and the diversity of the communication technologies used give people the possibility of being involved in the project at varied levels, and to change their level of involvement more fluidly than in traditional innovation ecosystems. In the LEGO ecosystem figure (Figure 2), we can see that the community has also a diverse internal ecosystem. In the beginning, fans started to use the Internet to communicate between them, and new activities emerged such as the creation of virtual LEGO models, an inventory of the available LEGO pieces and products, and the selling and buying of LEGO pieces and models. Over time, the firm acknowledged these community innovations and developed its own services based on them (such as Design byME or Pick a Brick). Thus, the practice and communication innovations that the community constructed were translated and are being commercialized by the company. The following list exemplifies some of the current communication tools and conventions that the community has devised and their counterparts as they have been interpreted in the company’s offerings.

Exploring the Ecosystems and Principles of Community Innovation

•

•

•

•

•

•

Custom-made Image Galleries: hosted by individuals or small companies, members’ own sites and blogs, and galleries in third-party picture-sharing services (such as Flickr).They are used to create, share, and discuss images of members’ created LEGO models (unique creations) both in digital 3D renders and built with actual pieces. Forums and Web Sites: a variety of tools from Web sites and specialized forums used to document and discuss members’ own LEGO collections (owned pieces and sets, own creations). A marketplace consisting of “user shops” powered by off-the-shelf e-commerce software or hosted in existing services (such as eBay). The community actively swaps, sells, and buys official LEGO sets (new and used) as well as custom-made sets (from second-hand dissembled and vintage pieces). Wikis and Custom-made Web Databases: used to create inventories of available LEGO pieces and sets (what parts come bundled together) as well as to document LEGO products by creating unofficial catalogues (include out–of-production and collection sets). Tools to Create Virtual LEGO Models (3D Models, Animations, Building Instructions): this includes an open standard for LEGO CAD programs and several software solutions (e.g. LDraw). Video Sharing and Discussion Forums: hosted in third-party services (such as YouTube, Vimeo, etc.) to create, share, and discuss by-products that use LEGO products as inspiration. There is, for example, a strong genre of video productions made with stop-motion animations of LEGO figures. There is also a long-standing practice of appropriating LEGO bricks and figures

•

•

•

• •

•

•

to create sets for a variety of amateur digital film productions. Design byME™: a service allowing users to create a digital model using LEGO virtual bricks. Models can be uploaded to the site in order to share them with other users. Approved models can be bought and are shipped in a custom-made box and machine-generated assembly instructions (launched by the company in 2005 under the name LEGO Factory). Product Gallery: for all approved Design byMe models submitted. The service allows users to rate them and provides message boards (to discuss service related issues, not the models themselves), an online shop (connected to production facilities in Poland where custom-made boxes are handpicked and delivered); and basic user profiles. Creator™: a service for allowing LEGO users to document and share models. This is not restricted; any model can be shared here. Since the company is not “selling” the models, there are not as many copyright infringement filters as in the Design byMe. Other Tools: a model gallery and message board (same for the whole LEGO site) Pick a Brick™: a service allowing customers to buy brick packages (available around 2006) LEGO TV™: a media service to publish video and animation narratives featuring official and licensed LEGO characters. Digital Designer Software (LDD): allows users to create virtual models with digital LEGO bricks. It is linked to the Design byMe service (but only to a limited number of pieces) and Creator (freer).

Unlike the eCars – Now! Community, the LEGO community is more closely linked to the LEGO company. Also, the individual users within

227

Exploring the Ecosystems and Principles of Community Innovation

the community do not necessarily always have a common task. LEGO “creators” can make their own individual LEGO creations independent from the other LEGO “creators”. At other times, they join in active projects such as the joint documentation of an inventory of pieces and LEGO sets that exist, which helps them in their individual creative endeavors. Unlike the eCars – Now! community, the LEGO community is not always manifested in building something big together, but, rather, in connecting individual LEGO users and providing them with a peer group to share their creations with. This is different from the eCars – Now! community, which share a clearer common task. Thus, the LEGO community does not have such a strong need for synchronous communication channels. Because of the individual tasks, they also do not have as strong and continuous a need for collaboration. However, we can say that they do collaborate by sharing information about their own projects and, sometimes, around collective initiatives such as specific software tools (LDraw, Mindstorms software, etc.). Information-sharing and collective knowledge-building needs seem to be very strong. For this information-sharing, they have started to use Web 2.0 channels such as wikis, Flickr, and YouTube. Thus, they have constructed a set of tools for efficient communication and information-sharing about their own individual projects and the creative endeavors centred on the bricks.

COMMUNITY INNOVATION PRINCIPLES This section presents some of the basic tenets articulated by current conceptualizations of “user” and “open” innovation. We review them with a view to identify the ways in which they fail to account for some factors that are present in our cases. Amongst the existing frameworks of the literature that have been focusing more on the business perspective, two have been particularly

228

successful in articulating a coherent critique of linear and closed models while offering concrete management tools and policy suggestions. These approaches are usually grouped under the rubric of user innovation (Von Hippel, 1998, 2005) and open innovation (Chesborough, 2003, 2006, 2007).

From Manufacturer to User, What Else? User innovation (UI) literature has been key in establishing the fact that users are a significant source of innovation. This strand of research has contributed extensive empirical studies demonstrating the ways in which advanced users engage in modifications of a significant proportion of industrial and consumer products and why they do it. In establishing the conditions for the emergence of user innovations, these studies have demonstrated why a manufacturing-centred view is limited and outdated (Von Hippel, 1998, 2005). One of the most important applications of these insights is the “lead user” approach, which provides a systematic way of finding and incorporating into commercial innovation processes solutions figured out initially by advanced users (Franke et al., 2006). In this framework, “lead users” are considered better predictors of future market conditions than traditional market research indicators. Another important contribution is its insistence on product modifiability as an important variable to harness user input. This work, without a doubt, has important implications for the effective involvement of users, especially those who are in a position to play active roles as (co)-developers and, to some extent, even initiators of the process. However, in light of the examples, we identify some limitations: most of the applications of the approach remain at the individual user level; the role of communities, although discussed, has not been specifically articulated (West, 2009). Furthermore, in this approach, valuable user collaborators are mainly considered those “bright” and well-connected

Exploring the Ecosystems and Principles of Community Innovation

individuals that are usually technically proficient (Hysäälo, 2007). This is the product of a bias in innovations on “technology”, as only innovations that are embedded in manufactured products and have had economic impact are usually taken into account (Hyysälo, 2007). The approach easily dismisses other points of innovation, such as organizational arrangements or social practices7 that are quite visible in both the eCars-Now! and LEGO cases. Some of these limitations have consequences, for example, when drawing policy implications, organizing legislation, or thinking about funding schemas. The cases explored above not only point out users’ ability to innovate by themselves but also draw our attention to new forms of collective organization and collaborative practices that make possible other points of view and other forms to articulate innovation that might well be supported and expanded.

From Closed to Open; What Else? In strict terms, the open innovation (OI) approach describes an emerging distributed mode of innovation that, in contrast to the user innovation approach, focuses more on how companies can profit and gain competitive advantage by managing information flows more efficiently regardless of their source. Henry Chesborough (2003a) and his colleagues coined the term and have been developing this orientation. The argument posits that, in a world where knowledge is widely distributed, successful companies cannot rely only on their own research but, instead, should constantly search and incorporate, through buying or licensing, processes or inventions from other companies. Open innovation also suggests that internal inventions that cannot be used in the firm’s own business should be taken advantage of through licensing, joint ventures, and spin-offs outside the boundaries of the firm (Chesborough, 2003b). Even though the approach correctly identifies knowledge to be abundant, usually distributed outside the firm and of good quality, we could

also identify some limitations. The discussion has an implicit heroic innovator assumption (bright individuals), which leaves little room for discussing the social roots of innovation. The implications of open innovation discussions openly discuss the motivations (profit) for collaboration by raising the importance of the business model, but this puts too much emphasis on a particular type of definition of what counts as profit, which, in light of the cases reviewed, could be opened up more. Furthermore, OI concentrates analysis solely on the implications and benefits that these types of activities have for a particular way of organizing productive activities (a firm). As our cases suggest, the digital and business ecosystems are richer and more complicated than assumed by this discussion, when opportunities for meaningful work, fun, and economic profit can drive the activities and collaboration endeavors of more complex collectives.

Contrasting Principles In an insightful analysis of the situation, (Chesborough 2003) condensed the model of closed innovation through a set of 6 principles that, he argued, have implicitly guided this model. In contrast to them, he developed 6 new principles summarizing the emerging open innovation attitude and approach. In the following analysis, we use (Chesbrough’s 2003) initial comparison between closed (CI) and open innovation (OI) as a framework to propose community innovation (CoI) principles and contrast them with the existing CI, UI, and OI attitudes. The CoI principles are described from the point of view of an extended community, so, in our case, the “we” is not necessarily a company. As an example, we can see how a principle of open innovation exemplifies that “not all of the smart people work for us, so we must find and tap into the knowledge and expertise of bright individuals outside our company”, while our principles suggest that “not all our people are

229

Exploring the Ecosystems and Principles of Community Innovation

Table 1. Contrasting principles of CI and OI (from Chesborough, 2003a) expanded with user innovation (UI) and community innovation principles (CoI) (adapted and expanded from Botero, Vihavainen, & Karhu, 2009) €€€€€Closed Innovation (CI)

€€€€€Open Innovation (OI)

User Innovation (UI)

€€€€€Community Innovation (CoI)

€€€€€The smart people in our field work for us.

€€€€€Not all of the smart people work for us, so we must find and tap into the knowledge and expertise of bright individuals outside our company.

Lead users are the smartest and most important source of knowledge.

€€€€€Not all our people are “smart”, but they are connected and enthusiastic, and their contributions are valuable to us.

€€€€€To profit from R&D, we must discover, develop, and ship it ourselves.

€€€€€External R&D can create significant value; internal R&D is needed to claim some portion of that value.

R&D can be distributed, but it is mostly in lead users’ niches who are “scratching their own itches”.

€€€€€The more R&D is distributed, the better, as everyone can “scratch a different itch”.

€€€€€If we discover it ourselves, we will get it to market first.

€€€€€We don’t have to originate the research in order to profit from it.

Research needs to be discovered to make a profit from it.

€€€€€Value can be made in many ways: by services and other offerings, by users’ own creations, etc.

€€€€€If we are the first to commercialize an innovation, we will win.

€€€€€Building a better business model is better than getting to market first.

€€€€€We utilize existing business models and innovate new ones so that everybody can make profits of different kinds.

€€€€€If we create the most and best ideas in the industry, we will win.

€€€€€If we make the best use of internal and external ideas, we will win.

€€€€€Solving existing problems eventually benefits everybody. Also, it can just be made for fun!

We should control our intellectual property (IP) so that our competitors don’t profit from our ideas.

€€€€€We should profit from others’ use of our IP, and we should buy others’ IP whenever it advances our own business model.

‘smart’, but they are well-connected and enthusiastic, and their contributions are valuable to us”. Frank and Shah (2002) have indicated how, under certain conditions, innovation activities within user communities can act as the “functional equivalent of the multi-person innovation project team” inside an organization. Furthermore, in discussing how diverse groups of individuals have successfully collaborated on projects, driven by a diverse set of motivations or social signals rather than market prices or managerial commands, (Benkler 2002) suggested that a new mode of production has emerged. He referred to it as “commons-based peer-production” (Benkler, 2002). While the case studies in this chapter do not fully comply with the conceptualizations of either (Franke and Shah, 2003) or (Benkler, 2002),

230

Free revealing is important; it is not practical to worry about IP too much.

€€€€€We can openly share our IP, utilize existing open works, and not waste our resources on IP management.

they do exemplify certain attitudes that should be recognized and supported more. We are particularly interested in highlighting several insights. First, innovation communities are diverse and heterogeneous, not necessarily composed of bright individuals but, rather, committed and wellconnected collectives. Second, inflows and outflows of information and ideas are not determined only by a-priori ideas of what counts as the “boundary” of a firm or a community. Third, there is a need to leave room for broader conceptualizations of what count as profit and motivations and to account for the types of practices of enjoyment, commitment, and free play that our cases also suggest. Fourth, communities rely on assistance from fellow community members for both shared and individual projects; there is ample sharing of

Exploring the Ecosystems and Principles of Community Innovation

information, and that requires particular infrastructures at the technical and social levels.

CONCLUSION In this chapter, we have explored two different cases of community innovation from three different perspectives. First, we illustrated the Digital Business Ecosystems that have evolved between and around the communities and companies involved in these cases. Second, we analyzed the communication tools used inside the communities and between the various stakeholders in the ecosystem and the ways in which they have been configured to suit particular contexts. Third, we suggested a set of different innovation principles that explore alternative approaches and attitudes toward innovations. Based on our exploration on the two cases, we want to underline the following observations. •

•

Traditionally, the idea of diffusion of innovation has been used to account for the diffusion from producers and companies to users, or to communities that mostly have been considered to embody reactive adoption roles. However, in our cases, we can see it happen in the opposite direction. Communities are important loci of innovation that shouldn’t be overlooked. There is no silver bullet or any one-sizefits-all groupware that would be suitable for everyone’s diverse needs. However, there is a large pool of separate tools available around the Internet, though constructing a communication toolbox from the myriad technologies available today is a dynamic innovation process itself. This innovation process is essential for the community to interact efficiently and to negotiate a place for themselves between grassroots innovation and corporate control.

•

Today’s media and computer-supported community innovation can be a very complex ecosystem containing various actors with variable goals and motives. It is valuable for the community innovation process that users with variable interaction levels and with different motives are given chance to contribute to the community.

We recognize that community innovation is not a new phenomenon; however, it is increasingly made visible by the new information and communication technologies and infrastructures and ways of working together that they enable. The case studies presented here illustrate some of the potentials and challenges that exist when such a phenomenon is recognized. A better understanding of these processes has important implications not only for the design and development of open and inclusive social media applications but also for our understanding of the role that these infrastructures play in choreographing increasingly distributed design and innovation process that move beyond media production and consumption into other areas as well.

REFERENCES Baecker, R. M., Grudin, J., Buxton, W. A. S., & Greenberg, S. (1995). Readings in humancomputer interaction: Towards the year 2000. San Francisco, CA: Morgan Kaufmann Publishers, Inc. Beloe, S. (2006, August). Lego: Beyond the dark age. Radar: The diversity issue, 3 8-11. Bender, J. (2010). LEGO: A love story. Hoboken, NJ: John Wiley and Sons. Bijker, W., Hughes, T. P., & Pinch, T. (1989). The social construction of technological systems: New directions in the sociology and history of technology. Boston, MA: The MIT Press.

231

Exploring the Ecosystems and Principles of Community Innovation

Botero, A., Vihavainen, S., & Karhu, K. (2009). From closed to open to what? An exploration on community innovation principles. In Proceedings of the 13th International MindTrek Conference: Everyday life in the ubiquitous era (pp. 198-201). Tampere, Finland: ACM. Callon, M. (2004). The role of hybrid communities and socio-technical arrangements in the participatory design. Journal of the Centre for Information studies, 5(3), 3-10. Chesbrough, H. W. (2003a, April 15). The era of open innovation. MIT Sloan Management Review, 44(3), 35–41. Chesbrough, H. W. (2003b). Open innovation: The new imperative for creating and profiting from technology. Boston, MA: Harvard Business School Press. Ellis, C. A., Gibbs, S. J., & Rein, G. (1991). Groupware: Some issues and experiences. Communications of the ACM, 34(1), 58. doi:10.1145/99977.99987 Franke, N., & Shah, S. K. (2003). How communities support innovative activities: An exploration of assistance and sharing among end-users. Research Policy, 32, 157–178. doi:10.1016/ S0048-7333(02)00006-9 Franke, N., Von Hippel, E., & Schreier, M. (2006). Finding commercially attractive user innovations: A test of lead-user theory. Journal of Product Innovation Management, 23(4), 301–315. doi:10.1111/j.1540-5885.2006.00203.x Gay, J. (Ed.). (2002). Free software, free society: Selected essays of Richard M. Stallman. Boston, MA: Free Software Foundation. Godin, B. (2006). The linear model of innovation: The historical construction of an analytical framework. Science, Technology & Human Values, 31(6), 639–667. doi:10.1177/0162243906291865

232

Greenbaum, J. M., & Kyng, M. (Eds.). (1991). Design at work: Cooperative design of computer systems (1st ed.). New York, NY: Lawrence Erlbaum Associates. Grudin, J. (1994). Groupware and social dynamics: Eight challenges for developers. Communications of the ACM, 37(1), 92–105. doi:10.1145/175222.175230 Hyysalo, S. (2007). User innovation, design space, and everyday practices: Rodeo Kayaking case revisited. In Proceedings of the Nordic Consumer Policy Research Conference (pp. 1542-1558). Helsinki, Finland: Nordic Forum for Consumer Research. Jenkins, H. (2006). Convergence culture: Where old and new media collide. New York, NY: NYU Press. Johansen, R. (1988). Groupware: Computer support for business teams. New York, NY: The Free Press. Karhu, K., Botero, A., Vihavainen, S., Tang, T., & Hämäläinen, M. (2009). A digital ecosystem for boosting user-driven service business. In Proceedings of the International Conference on Management of Emergent Digital EcoSystems (pp. 246-253). Lyon France: ACM. Lautsi, V. (2009, August 29). Sähköautojen läpinmurtoon pitkä matka. Helsingin Sanomat, D4. Helsinki, Finland. LDraw community. (2008). LDraw history. Retrieved from http://www.ldraw.org /Article11.html Lehman, T. (1997). LUGNET plan. Retrieved August from http://www.lugnet.com /admin/plan/ Moore, J. F. (1993). Predators and prey: A new ecology of competition. Harvard Business Review, 71, 75.

Exploring the Ecosystems and Principles of Community Innovation

Moore, J. F. (1996). The death of competition: Leadership and strategy in the age of business ecosystems. Harper Business. Nachira, F., Nicolai, A., Dini, P., Rivera Lèon, L., & Le Louarn, M. (2007). Digital business ecosystems. European Commission. Nielsen, J. (2006). Participation inequality: Lurkers vs. contributors in Internet communities. Retrieved from http://www.useit.com/alertbox / participation_inequality.html Normann, R., & Ramírez, R. (1993, August). From value chain to value constellation: Designing interactive strategy. Harvard Business Review, 71(4), 65–77. Piller, F., Schubert, P., Koch, M., & Möslein, K. (2005). Overcoming mass confusion: Collaborative customer co-design in online communities. Journal of Computer-Mediated Communication, 10(4). Pinch, T. (1996). The social construction of technology: A review In Fox, R. (Ed.), Technological change: Methods and themes in the history of technology (pp. 17–35). Amsterdam: Harwood Academic Publishers. Prahalad, C. K., & Ramaswamy, V. (2004). The future of competition: Co-creating unique value with customers. Boston, MA: Harvard Business School Press. Prospero, M. A. (2007, December). Brick by brick, Lego’s new building blocks. Fast Company. 98. Retrieved from http://www.fastcompany.com/ magazine/98/dispatch.html Räsänen, J. (2009, October 2). eCars – Now!: Open development – Balancing between the material and immaterial worlds. Keynote presented at Mindtrek 2009, Tampere, Finland. Rogers, E. M. (1995). Diffusion of innovations (4th ed.). New York, NY: Free Press.

Sähköautot – Nyt! (2010). S. Sähköautot Retrieved from http://www.sahkoautot.fi/ Severa, P. SA – blogi. (2010). Severa sponsors electric car venture eCars – Now! Retrieved from http://blog.severa.com/ archives/234 Seybold, P. B. (2006). Outside innovation: How your customers will co-design your company’s future. Collins Business. Thrift, N. (2006). Re-inventing invention: New tendencies in capitalist commodification. Economy and Society, 35(2), 279–306. doi:10.1080/03085140600635755 Tuomi, I. (2003). Networks of innovation: Change and meaning in the age of the Internet. New York, NY: Oxford University Press. von Hippel, E. (1988). The sources of innovation. New York. USA: NY Oxford University Press. von Hippel, E. (2005). Democratizing innovation. Boston, MA: The MIT Press. West, J. (2009). Policy challenges of open, cumulative, and user innovation. Washington University Journal of Law & Policy, 30, 17–41. Yin, R. K. (2002). Case study research: Design and methods (3rd ed.). Thousand Oaks, CA: Sage Publications, Inc.

KEY TERMS AND DEFINITIONS Business Ecosystem: The business ecosystem layer of DBE; the ecosystem of companies, goods, and services. Community Innovation (CI): Collaborative innovation activities undertaken by collectives comprising people and the digital ecosystems that surround them. Computer-Supported Cooperative Work (CSCW): Research tradition that explores the ways in which computer-assisted coordinated

233

Exploring the Ecosystems and Principles of Community Innovation

activity is carried out by groups of collaborating individuals and how to best support it. Digital Business Ecosystem (DBE): Socioeconomic development catalyzed by ICTs emphasizing the co-evolution between the business ecosystem and its partial digital representation: the digital ecosystem. Digital Ecosystem: The digital ecosystem layer of DBE; the technical infrastructure that is used to connect to the services and information over the Internet and to enable networked transactions. Groupware: Computer-based software and media systems that support groups of individuals to achieve common tasks and goals. Open Innovation (OI): A distributed mode of innovation focusing on how companies can profit and gain competitive advantage by managing information flows more efficiently regardless of their source. User Innovation (UI): Innovation processes where the sources of innovation can be located in user niches rather than in manufacturers’ or suppliers’ activities.

234

ENDNOTES 1



2



3

4



5



6



7



Any trademarks and product or brand names referenced in this document are the property of their respective owners. eCars – Now! logo by Joona Kallio See the history of one of the software packages, called LDraw, at: http://www.ldraw. org/Article11.html. In this case, we have particularly followed the LUGNET hub (http://www.lugnet.com/), which describes itself as an international fan-created Lego user group network. “Computer-supported cooperative work or CSCW is computer-assisted coordinated activity carried out by groups of collaborating individuals” (Baecker, 1995, p. 141). “Computer-based systems that support groups of people engaged in a common task (or goal) and that provide an interface to a shared environment” (Ellis, 1991). Like the ones discussed by, e.g. Tuomi (2003).

235

Chapter 13

Supporting Local Connections with Online Communities Sanna Malinen Tampere University of Technology, Finland Tytti Virjo Tampere University of Technology, Finland Sari Kujala Tampere University of Technology, Finland

ABSTRACT Online communities have become popular among geographically distributed users of the internet. However, there is growing interest in using online communities to support social interaction in geographicallybased communities too. In this chapter, we study the value of online sociability and the role of local networking in two different online social internet sites. We present the results of a survey carried out among members of Finnish Facebook groups, and complement the results with interviews for users of a local online service for people living in the surroundings of the city of Helsinki. The goal of this study was to investigate how online groups and services with local content connect with real-life networks and sociability, or whether they remain separated. The results show that Facebook is used mainly for nourishing existing friendships online and less for meeting or looking for information on new people. However, Facebook groups are often connected to real-life activities and places, thus local connections and networks play an important role in the use of Facebook. For the users of the local online service My City, the strong local identity experienced and attachment to the place of residence were important motivators for active participation and the creation of content. DOI: 10.4018/978-1-60960-774-6.ch013

Copyright © 2012, IGI Global. Copying or distributing in print or electronic forms without written permission of IGI Global is prohibited.

Supporting Local Connections with Online Communities

INTRODUCTION During the last decade, various user communities and groups of internet have become an important part of people’s communication practices. In recent years, social networking sites and web services that lean on user-generated content, often referred as Social Media or Web 2.0., have been attracting users of the internet. Boyd and Ellison (2007) define social networking sites as online services that allow individuals to construct visible profiles, articulate a list of connections with other users, and view and traverse these lists of connections. Online communities provide the users opportunities for sharing experiences and giving and receiving peer support, as well as serving as information sources (Joinson 2008; MaloneyKrichmar & Preece 2002; Millen & Patterson 2003; Wellman & Gulia 1999). Altogether, both types of online services support people’s sociability and enhance their social connectivity. They allow people to socialize in many different ways, to rate, comment, and exchange information, as well as to share personal experiences with others. The experienced sense of community is known to have many positive outcomes in people’s lives, such as loyalty, willingness to help others, and commitment to community activities (McMillian & Chavis 1986). The concept of the ‘sense of virtual community’ (SOVC) refers to the emotional bond between the members of online groups and community-oriented behaviors that emerge from social processes and behaviors (Blanchard & Markus 2002). Online communities can be used to increase the social capital and cooperation of community members, and thus there is also a growing interest in using online communities to support and facilitate the social interaction of members of particular geographically-based communities (King & Brown 2007; Millen & Patterson 2002). Local networks and place of residence contribute to the person’s experienced sense of community as well. Term ‘locality’ refers to ‘spaces’ and ‘places’, which are closely related concepts.

236

Place has been defined as a space endowed with a meaning; people tend to have a positive emotional bond to familiar places, and this psychological relation between people and their environment is referred to as place attachment (Hummon 1992; Low & Altman 1992; Lewicka 2008). A person’s awareness of the history of a place intensifies their attachment to it, and vice versa; people attached to a place are known to express more interest in its past (Lewicka 2008). We suggest that people’s personal ties to their local surroundings can be reinforced by online communities. Online social networking and forming virtual groups with people who share the same interests can promote a sense of community in a face-to-face context, and thereby increase wellbeing. In this study, we suggest the term ‘sense of locality’ for describing people’s relation to their physical environment and its social surroundings. Sense of locality is thus a broader concept, including the social context, the geographical area, and a person’s emotional relation to these. Sense of locality refers to awareness of local issues, people and places, and a sense of belonging to one’s local environment. In the present study, we examine how the sense of locality and well-being can be improved and local connections reinforced by providing people with an opportunity to build social networks and get to know and discuss with people living in the same geographical area. We propose that users may be motivated to generate and share information concerning local issues since they are given the role of experts. Previous research (Millen & Patterson 2003; Pinkett 2003) suggests that participation in a local online community can increase the community’s social capital, which is manifested in the expansion of social networks and people’s increased awareness of community resources. Locality and sociability in online context are studied in two different cases. In the first case study, we conducted a survey in order to explore the value of local aspects of a social networking

Supporting Local Connections with Online Communities

service among the Finnish Facebook members. In addition, we investigate what motivates people to join in local Facebook groups and networks. In the second case, we focus on sociability and participation among the users of a Finnish location-oriented online service, My City (http:// omakaupunki.hs.fi/). We aim to find out how much online communities support local activities and relationships in real-life, or whether the sense of community remains only virtual.

BACKGROUND Social Networking on Facebook Facebook.com is clearly the most popular social networking site, with more than 500 million active users worldwide (www.facebook.com). Facebook allows users to create a personal profile, share content, communicate, and network with other users. As distinct from other online communities, on Facebook it is recommended to use real names instead of imaginary virtual identities, which leads to networking and socializing mostly with people familiar from the offline environment. Facebook launched outside the US in 2006 and quickly became popular in Finland in the autumn of 2007. According to the statistics, the current number of registered Finnish Facebook users is over 1,8 million (http://www.facebakers.com), and Facebook is currently the second most popular internet site in Finland (www.alexa.com). Facebook started as a geographically-based campus community, with its members sharing mainly local offline connections (Ellison et al. 2006; Joinson 2008). Compared to other social networking services, Facebook’s primary distinction is that it is based on existing offline networks, initially university communities, and later on other types of offline communities as well (Lampe et al. 2006). Because of its history, previous research suggests that this combination of offline and online communities

may have created some unique forms of use in Facebook (Joinson 2008; Lampe et al 2007). Previous studies on Facebook indicate that it is mostly used for maintaining previous relationships, and more precisely, seeing how friends are doing (Lampe et al. 2006). Facebook is used especially to refresh and maintain long-distance friendships, and it can also help to maintain relations and keeping in touch with them as people are moving from one offline community to another (Joinson 2008; Ellison et al 2006). In addition to maintaining existing social networks, Facebook is used for ‘social searching’, that is, investigating and finding out information about people familiar from an offline context, and it can serve as a ‘surveillance tool’, as it enables its users to track others’ actions and interests (Lampe et al. 2006). However, Facebook seems to be used less for ‘social browsing’, that is, using it for getting new contacts, sometimes with the aim of offline interaction, and more for increasing awareness and learn more about people who are already familiar from an offline context (Joinson 2008, Lampe et al. 2006). Previous studies indicate, that Facebook is used in a way which is contrary to the popular view of how online social networking sites are used, since the users are more interested in maintaining their existing connections than searching for new ones (Ellison et al. 2006). Nowadays, using Facebook has become popular among adults in the working life as well. Since the previous research concentrates mostly on college students, it is interesting to find out how other groups of people are using Facebook, and whether it differs somehow from the styles and purposes of use of students. Regardless of its international expansion, there are some characteristics left in Facebook that are related to geographical location, since members have to choose a Network (global, national, or specific university) to join when they register. A nationwide network is a rather extensive context to create a sense of community in, even in such a small country as Finland. Facebook encour-

237

Supporting Local Connections with Online Communities

ages its members to create groups and join them freely, which has led to a vast spectrum of different types of groups, e.g. geographical, political, and humorous ones. Therefore, group activities have an essential role when studying locality and sociability in Facebook. Despite the large size and popularity of groups, they often seem to remain rather inactive. The available Facebook group operations – posting a wall comment, pictures, or videos – facilitate discussions and chatting. Observation of group activities reveals that communication and content production are usually on quite an irregular basis. In this study we aim to find out the motivations people have to join and participate in Facebook groups, particularly if they do not serve as a communication channel. Facebook is not designed for supporting local communities, but what is the role of locality in Facebook groups? Do members wish they could have more activity and local contacts if these were better supported in the service? Or are there some other reasons for establishing and joining in Facebook groups? Furthermore, we are interested to find out how people are using Facebook, is it used for networking with new people or rather for maintaining existing contacts? Do the users communicate with people living nearby or with those who are physically distant?

The research data was collected in June-August 2008. The first case consists of two stages; at first, a brief online observation of activities on the Finnish Facebook groups was carried out in order to understand the popularity of the groups that are somehow connected to local context, and the themes that are discussed in the groups. After the observation period, an electronic survey was conducted among the members of Finnish Facebook groups and networks. Through the survey, we aimed to shed light on the various uses of Facebook, and identify different user groups among the participants. Especially, we wanted to find out what role do locality and local connections play in the use of Facebook. The second case study that consists of qualitative user interviews for the registered users of My City was performed in order to complement the survey results. As opposite to Facebook, My City is a local online service that is aimed for place-specific information, and intended for people who either live or visit in Helsinki area. By studying a geographically oriented internet site, we wanted to gain a better understanding of the motivations of the active users for producing content and sharing information with others on a local online service.

Method

CASE A: FACEBOOK SURVEY

This study adopts a multiple-case research design strategy (Dubé & Paré 2003; Yin 1994). Multiple case studies were performed to understand the influence of variability in a context and to gain more general research results than single cases (Yin 1994). The cases represent two diverse online services, which enhance the generalizability of the results (Eisenhardt 1989). Our research approach combines quantitative and qualitative data; quantitative data was collected in order to gain a statistical picture and generalized results of how people use Facebook and socialize on it, whereas qualitative interviews will provide an insight into motivations and reasons of the users.

Online Observation of Facebook Groups

238

According to the observations on Facebook, a large number of popular groups are based on location in one way or another. Their local connections occur in various ways; groups have been created in relation to former and current places of residence, from nationwide to a certain neighborhood or quarter, geographical areas of work, study and leisure activities, public events, and local services, such as shops, restaurants, and nightclubs. Some of the groups stand for a real-life group or association and use Facebook for communicating and distributing

Supporting Local Connections with Online Communities

information, but some are Facebook-originated groups seeking real-life activities, such as a local recycling circle. Being a member of these groups indicates a motivation to strengthen one’s sense of locality. In addition, it may also suggest that the members have a need to expand their social surroundings and networks in order to socialize with people with similar local connections and get to know them better.

Conducting the Survey Because of the renovation of the user interface in Facebook during the summer of 2008 the Network pages were no longer available, and we were not able to recruit respondents via the Facebook Finland Network, as we had planned. Instead, an invitation to take part in the survey was sent to the members of the “just for fun” group Great Facebook Race Finland (approx. 51,000 members), the group of people who previously lived in the town of Kuopio (Ex-kuopiolaiset, 1000 members), and the Facebook friend networks (approx. 100 persons) of the research team. Overall, 240 responses were received. Joinson’s study (2008) of Facebook uses and gratifications was used as a starting point when the questionnaire was designed. In Joinson’s twophase study 137 users first described how they use Facebook, and on the basis of these findings he identified 46 different uses and gratifications. In order to analyze the findings further, he conducted a survey in which respondents were asked to rate the 46 uses and gratifications using a 7-point Likert scale, and identified seven factors based on their evaluations. The online survey of the this study included basic demographic questions (e.g. age, gender, occupation, and household size), as well as questions concerning the use of Facebook in general (time spent on it each week, number of friends, privacy settings, activities, participation). The participants were also asked to rate the importance and value of a number of Facebook features and uses, using a 7-point interval scale. The scales were anchored

at 1 (not at all important) and 7 (very important). The questions concerning the importance, value, and motivations for use were similar to the factors identified by Joinson (2008) in order to enable comparisons to be made with the results. People’s motivations for creating and joining groups were asked through open questions. The survey was first piloted by ten Facebook users, and the questions were revised so as to be more unambiguous on the basis of the feedback from the pilote tests. The survey was open for over one month, from 9th July until the end of August. SPSS Statistics 17.0 was used for the statistical analysis of the quantitative data. In addition, content analysis was carried out for the qualitative data gained from open survey questions and user interviews.

Participants The survey was completed by 240 Finnish Facebook users. 153 of them were females (64%) and 87 males (36%). Their mean age was 31 years (range 13-65 years). Contrary to earlier Facebook studies (Joinson 2008; Lampe et al. 2006), the majority of the sample (67%) reported employment as their main occupation and only 21% were students. 5% of the participants were on parental leave and the rest of the participants (7%) responded that they were unemployed, on a pension, or ‘other’. 70% of the respondents had been registered with Facebook for 6-12 months and 21% for over a year. The majority (86%) were active users, visiting the site more than once a week, and 61% visited it daily or several times per day. In spite of the large number of visits, 86% reported spending less than 5 hours weekly in Facebook. Most of the respondents had quite a large number of friends linked to their Facebook profile: 70% had 51 or more and 39% had over 100 friends. As in Joinson’s (2008), study a clear majority of the respondents (74%) reported making their profile more private, and only a small minority (3%) more open. A fifth of the respondents (21%) reported having kept the default settings. 239

Supporting Local Connections with Online Communities

Results The Role of Groups in Facebook Respondents were asked to choose their personal reasons for joining a group from a list of 15 motivations based on the observations, and the most popular reasons seem to be somehow connected to their local context and existing social networks. As summarized in Table 1, as many as 75% of the respondents mentioned having joined a group that was related to their former place of residence, and 41% chose the group’s relation to their current place of residence as one of the reasons. 5% of the respondents wanted to get to know new people by joining groups. The high amount of mentions of joining a group for its relation to a former place of residence is partly explained by the recruitment procedure, but it is also congruent with previous research (Joinson 2008) indicating that finding old school friends is one common purpose for using Facebook. Other,

non location-oriented reasons for joining groups were: “To support a good cause” (60%), “The name of the group was funny” (40%), and “Just a way to spend my time” (8%). 23% (n = 56) of the respondents reported having created a Facebook group. They were asked more precisely what purpose the group was created for, and it seems that Facebook groups are mostly intended for friends and people with the same interests, or they were support and admiration groups. According to the content analysis of the answers to the open questions, the most common reasons for creating a group were: 1. I wanted to communicate and share content privately with certain people (22 mentions), 2. There was a need for a fast and easy information channel (12 mentions), and 3. Just for fun (7 mentions). The descriptions of groups and reasons for creating them reveal that Facebook groups are more often for a small circle of friends than open groups for everyone to join.

Table 1. The most common reasons given for joining a Facebook group Reason to join a group

% of users (N=240)

The group is related to my former place of residence

75

To support a good cause

60

The group is related to my hobby

57

The group is related to my current place of residence

41

The name of the group is funny

40

It is an inside group of my friends

39

It is a group of a society/organization that I belong to

36

The group is related to an event, e.g. music festival

34

The group is related to my job

34

The group is related to my studies

29

The group was established by a company whose customer I am

13

Just a way to spend my time

8

For some other reason

6

Getting to know new people

5

The group has a large number of members

3

(reasons relating to local or offline context marked in bold)

240

Supporting Local Connections with Online Communities

It seems that as group members, the respondents are rather active. In order to investigate their activity in the groups we asked how they communicated in them. The most common way to communicate in a group was to write something on its wall. Half of the respondents (53%) reported having written on the wall of a group, and 26% had added other content, such as pictures, videos, or links, for the whole group to see. On the basis of their responses, most of the groups they belonged to were private or limited to a small group of people. Therefore, the rather high number of wall messages may indicate activities in closed or hidden groups, since the type of group where the activity took place was not asked about. Since we were interested in knowing how online communities overlap with people’s offline activities, they were asked if they had participated in any real-life event or meeting organized by a Facebook group. The results show that 31% of the respondents had taken part in a real-life event or meeting of groups. We also asked the respondents if they had arranged any group meetings in real life and what kind of activities they were. They reported having met other members at concerts, games, parties or at the regular meetings of the

society or club they belonged to. Those who did not want to meet other group members felt that there was not really a purpose for doing that, for example, because a group was based on a one-off occasion or a joke that would not connect people in the long run. Since the actual amount of Facebook group activities in real life is not known, it is difficult to say whether it is the users’ unwillingness to participate or the lack of real-life activities that slows down Facebook groups in developing a stronger sense of locality in their members.

The Attraction of Social Features in Facebook We wanted to find out what the participants value about the use of Facebook. They were asked to respond on the Likert scale from 1 (= not at all interesting) to 7 (= very interesting) how attractive different socializing features of Facebook are. The respondents’ evaluations of the attraction of the social features of Facebook are presented in Table 2. As shown in the table, the most attractive features were “Looking at the profiles of people I know” (mean 5.62), “Looking at the pictures

Table 2. Attraction of social features of Facebook On a scale of 1-7, how interesting are the following features of Facebook to you personally?

Mean

SD

Looking at the profiles of people you know

5.62

1.25

Looking at the pictures uploaded by others

5.34

1.32

Seeing what other people have put as their status

4.69

1.64

Browsing your friends’ friends

4.47

1.55

Updating your own status

4.44

1.82

Uploading and sharing your own pictures

4.25

1.81

Editing your own profile

4.07

1.63

Being a member of groups

3.70

1.55

Organizing or joining real-life events

3.63

1.97

Following the news feed

3.63

1.86

Using advanced search to look for specific types of people

3.33

1.89

Looking at the profiles of people you don’t know

3.13

1.57

241

Supporting Local Connections with Online Communities

uploaded by other users” (mean 5.34), and “Seeing what people have put as their status” (mean 4.69). On the basis of the results, it seems clear that the most important function of Facebook is to enable people to follow the lives of their friends and other familiar people, whereas it was not used so much for social searching or browsing, and more specifically, for finding information on new people. The deviation in the answers on the 7-point scale was rather high, and only presenting the means thus hides interesting information; here, they serve as an initial glimpse of the order of importance of the features. The feature “looking at the profiles of people you know” divided opinions least of all, and was clearly the most attractive social feature of Facebook. The evaluations of the personal value of social features to the respondents are presented in Table 3. Compared to the attraction evaluations, there was less deviation in the scale, that is, respondents shared their opinions more. The first three uses, following the lives of friends, keeping in touch with people far away, and using Facebook as the only communication channel with certain people, were assessed as important, and meeting new people was quite unanimously evaluated as not so important. Interestingly, supporting work-related networks was rated as not important by an overwhelming majority. This is especially interesting

bearing in mind that 67% of our sample reported employment as their main occupation and many of the respondents had joined job-related groups. When asked more specifically about getting to know new people through Facebook, a third of the respondents (34%) reported having done it. Furthermore, 32% of those who had met new people via Facebook had also met their new acquaintances in real life, 41% of them had not met them but would be interested in doing so, and 27% preferred to only meet online. This further indicates that Facebook is used mostly to support real-life friendships, although meeting new people also has a role in Facebook use, and many users would like to become acquainted with new people. Further study of the effect of the demographic variables in meeting new people on Facebook reveals that men are slightly more positive about making new contacts. 14% of the men and 8% of the women reported having met new people via Facebook. In addition, in this group, 16% of the men had met their new Facebook friends face to face, whereas 8% of the women reported having done that. Furthermore, 3% of the men and 12% of the women reported not being interested in meeting their new Facebook friends in real life. However, these gender differences are not statistically significant. When compared with age, there were no noticeable differences between

Table 3. Personal value of Facebook uses On a scale of 1-7, how important are the following uses of Facebook to you personally? Following the lives of friends and seeing how they are doing

Mean 5.45

SD 1.39

Keeping in touch with people living far from you

5.41

1.47

Keeping in touch with people you don’t communicate with otherwise

5.31

1.48

Reconnecting with old friends you’ve lost contact with

5.26

1.32

Keeping in touch with the friends you meet face to face as well

4.49

1.60

Deepening acquaintanceships into friendships

3.55

1.54

Self-expression and telling about your own life

3.47

1.72

Maintaining important networks from working life

2.93

1.68

Meeting new people

2.76

1.68

242

Supporting Local Connections with Online Communities

Table 4. Factor 1 Factor 1: Connection with friends

Loading

Keeping in touch with people you don’t communicate with otherwise

.856

Keeping in touch with people living far from you

.786

Reconnecting with old friends you’ve lost contact with

.690

Following the lives of friends and seeing how they are doing

.574

age and whether they had met new people, or in their willingness to meet new Facebook friends face to face. The respondents were also asked to select the topics of information or help they would like to receive from online social networks. The most important topics were cultural events (67%), hobbies and leisure activities (57%), information related to one’s neighborhood (56%), peer support from people in a similar life situation (55%), and traveling and resorts (52%). The less interesting topics were peer-support in health issues (28%) and product recommendations (30%). Thus, it seems that information related to offline activities and place of residence were considered as the most interesting content of Facebook and other online social networking sites.

The Uses of Facebook In order to investigate different uses of Facebook in greater depth, factor analysis was conducted for variables that measure the attraction of the social features and personal value of Facebook. The initial factor analysis (rotation method Varimax with Kaiser normalization, extraction method Generalized Least Squares) yielded five factors

with eigenvalues over 1, explaining 63.4% of the variance. 18 of 21 variables had significant loadings (>0.4), and one of them (Keeping in touch with the friends you meet face to face as well) had similar loadings in two factors, and hence it was left out of the further analysis. Three items that did not load in any factors were also excluded from the analysis, they were: Maintaining working-life networks, Organizing and joining real-life events, and Being in the groups. The factor analysis (rotation method Varimax with Kaiser normalization, extraction method Generalized Least Squares) yielded four factors with eigenvalues over 1, explaining 65.33% of the variance. 16 variables had significant loadings (>0.4), only one variable (Following the news feed) did not load significantly in any factor. The first factor (Table 4) contains four items related to ‘Connection with friends’. These items clearly comprise keeping contact and reconnecting with old friends who either live far away or with whom keeping in touch has otherwise been difficult. This factor also includes following the lives of friends on Facebook. The second factor (Table 5) consists of three items that relate to more self-centered motivations to use Facebook, in addition to following other

Table 5. Factor 2 Factor 2: Self-expression and following new events

Loading

Updating your own status

.857

Seeing what other people have put as their status

.682

Self-expression and describing your own life

.591

Editing your own profile

.533

243

Supporting Local Connections with Online Communities

Table 6. Factor 3 Factor 3: Social investigation

Loading

Looking at the pictures uploaded by others

.706

Looking at the profiles of people you know

.659

Browsing your friends’ friends

.540

Uploading and sharing your own pictures

.500

people’s status updates. This factor is named ‘Self-expression and following new events’ and it includes updating one’s status on Facebook and following other users’ updates, telling about one’s life, and editing one’s profile. These items emphasize Facebook as a tool for telling about one’s life through the personal user profile, and in one’s own words, as well as active following of how others are, and what have they been doing. Factor 3 (Table 6) includes three uses that are somewhat related to social searching and social browsing as identified by Lampe et al (2006). In addition, this factor includes uploading and sharing one’s own pictures. Factor three is named ‘Social investigation’, that is, finding information about other people by browsing their Facebook profiles, exploring their social networks, and viewing pictures they have uploaded to Facebook. Social investigation is focused on both friends and unknown people. Furthermore, it seems that those who are interested in social investigation are also interested in uploading and sharing their own pictures for other users to browse, which may indicate of interests for becoming investigated by others. In the fourth factor (Table 7) are the items that are related to ‘Meeting new people’, which all

indicate of interests for social browsing (Lampe et al 2006), that is, expanding existing networks with new people on Facebook. Users who are interested in finding new people, deepening their existing contacts into friendships, and browsing profiles of unknown people on Facebook are also interested in using search feature in order to find certain kinds of people. The four factors were compared to user demographics and a T-test with independent samples shows that there was a significant difference between men and women in their rankings of Factor 1 ‘Connection with friends’ (t = 4,08, df = 238, p<.001). Females (mean = 5.59, SD = 1.16) scored higher in ‘Connection with friends’ than males (mean = 4.96, SD =1.13). In the rankings of the other four factors there were no statistically significant differences depending on gender. As the four factors were compared with the age variable and the correlations between them were studied, a positive correlation between age and Factor 4, ‘Meeting new people’, was found (r = .200, p<.01). This indicates that older users were more interested in meeting new people and expanding their networks on Facebook than younger ones.

Table 7. Factor 4 Factor 4: Meeting new people

Loading

Meeting new people

.869

Deepening acquaintanceships into friendships

.530

Using advanced search to look for specific types of people

.526

Looking at the profiles of people you don’t know

.471

244

Supporting Local Connections with Online Communities

CASE B: USER STUDY OF A LOCAL ONLINE SERVICE Online Service for Sharing Local Information My City, a local online service provided by Finland’s leading newspaper, Helsingin Sanomat, was launched in 2007. My City serves as a database and a search engine providing information on services and activities in the area of the 11 municipalities in the Helsinki metropolitan area. The community features include the creation of content on favorite places, service recommendations, reviewing and rating the attractions of the area, and creating and participating in interest groups, the topics of which include e.g. nature conservation, families with children, and dog parks. While Facebook is a tool for communication and self-presentation, My City is profiled more as an advanced online map complemented with user-generated content. The most popular content of the service, according to its log data, is presented in Table 8. At the time of the study, My City was a rather new service, and as the table below indicates, people mostly used it for finding local information instead of social networking purposes. As active users play an important role for the success of Social Media sites that are leaning on

user-generated content, we wanted to find out what motivates them to participate in and create content actively on the local online service. Furthermore, we wanted to investigate if their sense of locality can be reinforced by participation. To gather data for exploring our research questions and contribute to the development of the local community at an early stage, three semi-structured user interviews were carried out. With these qualitative interviews we wanted to complement the quantitative data that was gathered from the Facebook survey. The interviewees were recruited via the administrator of the service, and selected on the basis of their activity in content creation and group participation. All three participants had added content to the service, e.g. pictures or reviews of their favorite places, and were members of interest groups on My City, which were related to hobbies and pets, and were very similar to the ones on Facebook. The interviewees were males aged 18, 54, and 56 years.

The Results of the User Interviews Motivation for Participation The interviewees considered the service primarily as a databank and a useful source of local infor-

Table 8. The most popular content of My City The most viewed and searched content on My City (during one month) 1.

restaurants

2.

flea markets

3.

technical inspection station

4.

bars and pubs

5.

bicycle garages

6.

cafes

7.

Italian

8.

festival places

9.

saunas

10.

Friends of the Baltic Sea (a group)

245

Supporting Local Connections with Online Communities

mation. They reported using the service mostly to take a look at the weather or local news, or to search for a particular place or service. For all of the interviewees, the motivation to register with the service was their desire to produce content and share it openly. Instead of expecting external rewards for their contributions, they emphasized that sharing itself is motivating enough. On the contrary, one interviewee thought that rewarding users from content production may even make it feel compulsory and thereby less enjoyable. In the study of wikipedians, that is persons who actively create content to Wikipedia.org, Nov (2007) found out that Wikipedia’s free and collaborative nature was an important motivation to contribute. Our interviewees also felt a similar commitment to the community and wanted to promote its goals by taking part in its development, as the phrases of the respondents quoted below show. “I am motivated by the feeling that I am part of the community, and I can give people the information they need.” (male, 56) “Incentives and competitions may encourage [people] to produce more content for the service, but they can also make it obligatory.” (male, 18) Even though the service was mostly used as a databank, locality played an important role for them. The desire to learn more about one’s place of residence was reported as an important motive for using My City. The service enables its users to present their own expertise about local information they considered useful. According to the user interviews, the strong local identity experienced and attachment to one’s own city of residence were further motivators for active participation: “This city is very important to me, I was born here and I am interested in everything that is related to Helsinki.” (male, 54)

246

Joining Groups According to the previous research (MaloneyKrichmar & Preece 2002; Ridings & Gefen 2004) social support and friendship are the most important reasons for joining online groups, especially those dealing with personal interests, such as hobbies or recreation. Although My city is currently mainly used as a local database, the users reported having an interest in social networking and offline group activities, as well as meeting new people with similar interests and exchanging useful information with them, e.g. about which is the best dog park in the neighborhood. The users felt that a local internet service should provide them with an opportunity for general discussions with other people from the neighborhood and also more in-depth interaction, e.g. with private messages, according to the situation. They expected the threshold for participation in it to be low and the interaction rather informal. As one interviewee argues, informal communication is supporting the formation of community: “Making comments is actually not a discussion. Of course it is informative and efficient, you have a question – you’ll get an answer and there’s nothing pointless in it. But when it comes to a social network service, the pointless chatting is definitely a part of it.” (male, 18) Regarding social networking and meeting new people, the interviewees were slightly suspicious. Even though there was some interest in extending one’s networks with other members of local online groups, some privacy concerns regarding internet sites that are connected to real life context came up in interviews. The users wanted to connect with people with the same interests but at the same time maintain their privacy. Combining the virtual and real worlds also raised some questions and they wanted to be able to decide whom they would meet face to face.

Supporting Local Connections with Online Communities

“I would like to invite people to groups so that I could discuss things with them, and especially to find more active users - those who are there to produce content with a more serious attitude.” (male, 18) “If you are too close with them it may become a burden. People may show up at your front door any time.” (male, 54) The interviewees preferred anonymity and used nicknames instead of revealing personal information about them. However, knowing some personal information, e.g. age, gender, or place of residence, about the other users was considered relevant, especially when the reliability of content created by others was evaluated. Some details of personal information provide an idea of what kind of person is behind the nickname and therefore help to compare comments and ratings made by others with one’s own opinions. The user study of My City indicates that in general, the interviewees wanted to be anonymous in the local online service, but at the same time they were open to new contacts and had some expectations of social networking. They were interested in meeting other users offline, especially in connection with hobbies or other joint activities. All the interviewees were active users of the internet and familiar with social networking services, they also participated in several online groups related to hobbies or groups of friends. Internet sites that support local connections and activities were considered interesting, especially if they can provide useful information and contacts with local people. The results of the interviews show that the experienced sense of locality can motivate to use the service as well as to produce and share content on it.

CONCLUSION The results of the survey of 240 Finnish Facebook users are congruent with earlier studies (Ellison et al. 2006; Joinson 2008; Lampe et al. 2006) and show that the main purpose for using Facebook is to maintain and solidify existing relationships. Locality also plays a role in Facebook; for example, 41% of the respondents wanted to join groups related to their current place of living or other local social spheres, and 56% of users were interested in information related to their neighborhood. In addition, 32% of the respondents had already met new people through Facebook, although meeting new people was not evaluated as being an important purpose of using Facebook. Clearly, Facebook is a tool for social interaction and sharing content and information with friends or people that have similar interests. Different uses of Facebook identified in this study show that there are different user groups in Facebook with different needs concerning self-expression, social networking and content-sharing. Facebook is a good way of staying in contact with friends that you cannot meet so often otherwise. It also provides a channel for investigating other people and extending one’s social networks. In addition, people are interested in events, other people, and hobbies in the local context and looking for information about them from the internet. Based on the results, Facebook groups are utilized in local activities, and they seem to connect with real-life networks and events. Previous research of Facebook has focused mostly on students; when Facebook is used by young people in local communities such as on a university campus, its users seem to be willing to keep an open profile and meet new people. The older users in our survey are probably more established and with higher positions and they have more risks related to self-disclosure. However, according to our results there were no significant differences in the relation to privacy aspects depending on age or gender.

247

Supporting Local Connections with Online Communities

The results show that people do have interests in connecting local issues with their online networks and Facebook seems to be suitable for this purpose, partly because the majority of Facebook contacts are familiar from offline context. However, likewise in some earlier studies (Joinson 2008; Lampe et al 2006) also in the present study there are Facebook users who want to use it for meeting new people, and in order to that, search for interesting people and investigate others’ profiles. Thus, it is important to bear in mind that not all the users have same uses and preferences, for example, concerning their willingness to use it as a surveillance tool or for re-connection with old friends. The results of the Facebook study reveal that its users are interested in discussing and sharing local issues on their online networks. Within the second case of this study, My City, we wanted to enhance our understanding of what is the meaning of the locality for users. In My City the users communicate through creating location-based content, such as photos or reviews of their favorite places in the city area, for everyone to see and utilize. The interviews show that the most important reasons for participating in a local online community are willingness to help other people, share one’s expertise on local issues, and networking with other people with the same interests. At least in a city environment, people do not want to disclose themselves to anybody, but the need for connecting to others happens through similar hobbies and interests. The term ‘community social capital’ refers to the extent to which the members of a community can work and learn together, and ‘community cultural capital’ stands for the various forms of knowledge, skills, abilities, and interests that have particular relevance or value within a community (Pinkett 2003). We assume that the geographical context can facilitate the integration between the online and physical worlds. A local online service can be used as a tool for bringing residents, organizations, and businesses closer to each other

248

in a neighborhood. It can also be used to activate community members in producing information and content. An increase in the social and cultural capital of a community increases people’s awareness of local issues, their sense of locality, and therefore people’s capacity to improve the conditions of their neighborhood. We suggest that people are interested in finding others with similar interests and activities from their neighborhood, and online services can be utilized in that. Our results can be modified into implications for supporting the sense of locality by design. People may need some intimacy in a city environment. However, to support the sense of locality by online communities, people need to know who they are sharing the neighborhood with. Thus, location-based online communities should enable and encourage their users to provide user profiles that describe their identity, life situation, hobbies, and interests. User profiles that can be browsed, light content to share, and statements to make through e.g. group membership or status information could make it easier to get acquainted. Private messages and participating in online discussions with other residents of the area may create a context for real-world meetings, and later, face-to-face-meetings help to build trust in the online environment as well (Millen & Patterson 2002; 2003). Online communities and social networking sites provide many opportunities for supporting locality and encouraging people to participate in the activities of their surroundings. We suggest that these user-generated services and contents have positive outcomes as they can promote the experience of a sense of locality and enable their users to network with other people in the neighborhood.

ACKNOWLEDGMENT We wish to thank our colleagues, researcher Hannu Soronen for his advice in the analysis and Jarno

Supporting Local Connections with Online Communities

Ojala for assisting in this study. We would also like to thank the research partners of the project “PROFCOM – Product-Internationalization with Firm-Hosted Online Communities” for the successful collaboration, and especially Lassi Kurkijärvi from Sanoma Group for enabling this study.

REFERENCES Alexa (n.d.). Alexa. Retrieved August 11, 2010 from http://www.alexa.com. Blanchard, A., & Markus, M. L. (2002). Sense of virtual community – Maintaining the experience of belonging. In Proceedings of the 35th Annual Hawaii International Conference on System Sciences, 2002, HICSS. 3566-3575. Boyd, D., & Ellison, N. (2007). Social network sites: Definition, history, and scholarship. Journal of Computer-Mediated Communication, 13(1). doi:10.1111/j.1083-6101.2007.00393.x Dubé, & Paré, G. (2003). Rigor in information systems positivist case research: Current practices, trends, and recommendations. Management Information Systems Quarterly, 27(4), 597–636. Eisenhardt, K. M. (1989). Building theories from case study research. Academy of Management Review, 14(4), 532–550. Ellison, N., Steinfield, C., & Lampe, C. (2006). The Role of Facebook. In International Communication Association. Dresden: Spatially Bounded Online Social Networks and Social Capital. Facebakers (n.d.). Facebakers. Retrieved August, 11, 2010 from http://www.facebakers.com/ countries-with-facebook/FI/ Facebook (n.d.). Facebook. Retrieved August 11, 2010 from http://www.facebook.com/ press/info. php?statistics

Hummon, D. M. (1992). Community attachment: Local sentiment and sense of place. In Altman, I., & Low, S. M. (Eds.), Place Attachment (pp. 253–278). New York, NY: Plenum Press. Joinson, A. N. (2008) ‘Looking at’, ‘Looking up’ or ‘Keeping up with’ People? Motives and Uses of Facebook. In Proceedings of CHI 2008, Florence, Italy. 1027-1036. King, S., & Brown, P. (2007). Fix my street or else: Using the Internet to voice local public service concerns. In Proceedings of the 1st international Conference on theory and Practice of Electronic Governance 2007, ICEGOV ‘07, Macao, China, 72-80. Lampe, C., Ellison, N., & Steinfield, C. (2006). A Face(book) in the Crowd: Social Searching vs. Social Browsing. In Proceedings of ACM Special Interest Group on Computer-Supported Cooperative Work, ACM Press 167-170. Lampe, C. A., Ellison, N., & Steinfield, C. (2007). A familiar face(book): profile elements as signals in an online social network. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. San Jose, CA. 435-444. Lewicka, M. (2008). Place attachment, place identity, and place memory: Restoring the forgotten city past. Journal of Environmental Psychology, 28(3), 209–231. doi:10.1016/j.jenvp.2008.02.001 Low, S. M., & Altman, I. (1992). Place attachment: A conceptual inquiry. In Altman, I., & Low, S. M. (Eds.), Place Attachment (pp. 1–12). New York, NY: Plenum Press. Maloney-Krichmar, D., & Preece, J. (2002). The meaning of an online health community in the lives of its members: Roles, relationships and group dynamics. International Symposium on Technology and Society, 2002, ISTAS’02, 20-27.

249

Supporting Local Connections with Online Communities

McMillian, D. W., & Chavis, D. M. (1986). Sense of community: A definition and theory. Journal of Community Psychology, 14, 6–23. doi:10.1002/1520-6629(198601)14:1<6::AIDJCOP2290140103>3.0.CO;2-I Millen, D. R., & Patterson, J. F. 2002. Stimulating social engagement in a community network. In Proceedings of the 2002 ACM Conference on Computer Supported Cooperative Work. CSCW’02, 2002, New Orleans, LA. 306-313. Millen, D. R., & Patterson, J. F. 2003. Identity disclosure and the creation of social capital. In CHI ‘03 Extended Abstracts on Human Factors in Computing Systems 2003, Ft. Lauderdale, FL. 720-721. Nov, O. (2007). What motivates Wikipedians? Communications of the ACM, 50(11), 60–64. doi:10.1145/1297797.1297798

250

Pinkett, R. (2003). Community technology and community building: Early results from the Creating Community Connections project. The Information Society, 19, 365–379. doi:10.1080/714044684 Ridings, C., & Gefen, D. (2004). Virtual community attraction: why people hang out online? Journal of Computer-Mediated Communication, 10(1). Wellman, B., & Gulia, M. (1999). The network basis of social support: A network is more than the sum of its ties. In Wellman, B. (Ed.), Networks in the Global Village. Boulder, CO: Westview Press. Yin, R. K. (1994). Case Study Research, Design and Methods (2nd ed.). Thousand Oaks, CA: Sage Publications.

251

Chapter 14

P2P SCCM:

Service-Oriented Community Coordinated Multimedia over P2P and Experience on Multimedia Annotation Service Development Jiehan Zhou University of Oulu, Finland Mika Rautiainen University of Oulu, Finland Zhonghong Ou Aalto University, Finland Mika Ylianttila University of Oulu, Finland

ABSTRACT Peer-to-Peer Service-oriented Community Coordinated Multimedia (SCCM) is envisioned as a novel paradigm in which the user consumes multiple media through requesting multimedia-intensive Web services via diversity display devices, converged networks, and heterogeneous platforms within a virtual, open and collaborative community in this chapter. A generic P2P SCCM scenario is created and examined first. A SCCM model is designed with the adoption of the service orientation approach and principles. A tunneled hierarchical P2P model is designed for improving performance of service lookup and session setup. Next, performance analysis is presented with the average number of service lookup hops in the tunneled hierarchical P2P model. Finally, a prototype service implementation is presented with the design of content annotation service and application on face detection.

DOI: 10.4018/978-1-60960-774-6.ch014

Copyright © 2012, IGI Global. Copying or distributing in print or electronic forms without written permission of IGI Global is prohibited.

P2P SCCM

INTRODUCTION User experiences emerge from the active participation in events or activities and lead to the accumulation of knowledge, skill and enjoyment. This is made possible by ever-growing amount of networked multimedia content (e.g., video, audio) and multimedia-intensive services (e.g., multimedia searching, annotation) together with growing number of mobile users and deep penetration of broadband Internet connections. Collaborative use of multimedia content and multimedia-intensive services empowers users to experience the real world and share it with other users. The above emerging technical phenomena is generalized with a term ‘Community Coordinated Multimedia’ (briefly, CCM) and is characterized with Web accessibility, Web service-driven, mobility, equal participativity, etc. (Zhou, Rautiainen, & Ylianttila, 2008a). The shift towards CCM-driven user experience is manifested by several web services that have become popular in recent years, such as Wikipedia, Flickr, YouTube, and Joost. However, they are more emphasizing on community membership management and multimedia sharing, and less in the support of multimedia-intensive services such as multimedia analysis and annotation. There are amounts of efforts on generally addressing the issues of content annotation (Hansen, 2006) and retrieval (Croft, 1993; Rautiainen, 2006). There are also tools widely developed and applied for assisting users in the task of extracting, annotating, retrieving and mining multimedia content such as GATE with unicode-based text for supporting multilingual information extraction (Damljanovic, Tablan, & Bontcheva, 2008), ANVIL with frame-accurate, hierarchical multilayered annotation for multimodal dialogue (Kipp, 2004), and IBM VideoAnnEx annotation tool with MPEG-7 metadata (Lin, 2002). However, these tools are heavy weighted integrated multimedia

252

processing software systems. Meanwhile they are not Web services-based. Face detectors are studied for implementing our initial prototype, i.e., service-oriented face detection. The literature on cascade-structured detectors for faces and other objects are reviewed in (Wu, Brubaker, Mullin, & Rehg, 2008). A comprehensive review of other face detection approaches and techniques can be found in (Yang, Kriegman, & Ahuja, 2002). The cascade-structured face detectors such as Viola and Jones’ (Viola & Jones, 2004) and Wu et al.’s (Wu, Brubaker, Mullin, & Rehg, 2008) are mainly used in our experiments for test comparisons. In addition, there are many software libraries, examples of QuickTime for Java (Hoffert et al., 1992)(Chris Adamson, 2005), OpenCV (Gary Bradski, 2000), Java Media Framework (JMF) (Sullivan, Winzeler, Deagen, & Brown, 1998), open source MPlayer, that enable audio, video and other time-based media to be added to Java applications and provide various readymade functions for multimedia functionality and image processing, including face detection. In this chapter we propose a Peer-to-Peer (P2P) SCCM solution and present an application for face detection by combining service-oriented computing and face detection techniques to tackle the emerging requirements characterized by the CCM-driven user experience, such as Web accessibility, discoverability, and community driven collaboration. On the one hand, service-orientation system design and technology exhibits strong potential in leveraging and reinforcing CCM system development in coordination, composiblity, discoverability, extensibility and agility of multimedia consumption via diversity executing environments. Existing Web services technologies, i.e., Web Service Description Language (WSDL) (W3C-WSDL, 2005), Simple Object Access Protocol (SOAP) (SOAP, 2003), and Universal Description Discovery and Integration (UDDI) (UDDI, 2004) are widely used to actualize service-oriented solutions. On the other hand,

P2P SCCM

the vision of community coordinated multimedia is very compatible with P2P technology. P2P technology has potential to become a powerful enabler that harnesses Web services to promote collaborative and coordinated multimedia computing and communication with its advantages of scalability, fault tolerance, dynamic networking, and collaboration support (Ylianttila et al., 2008). This chapter introduces a model for P2P service-oriented community coordinated multimedia, which is a product of the ongoing EUREKA ITEA 2 CAM4Homeproject1 that focuses on creating a metadata-enabled content delivery and service framework. There are three contributions in this chapter. First, we analyze P2P SCCM paradigm for meeting emerging networked CCM requirements and design service-oriented CCM model. Second, we propose a tunneled hierarchical P2P model for improving the service lookup performance. Third, we make a theoretical performance analysis of the tunneled hierarchical P2P model and develop a prototype for face detection with the comparison of various cascade face classifiers, and present a preliminary comparison results.

DEFINITIONS Multimedia represents a synchronized presentation of bundled media types, such as text, graphic, image, audio, video, and animation. CCM (Community Coordinated Multimedia) system maintains a virtual community for the consumption of CCM multimedia elements, i.e. content generated by end users and content from professional multimedia provider (e.g., Video on Demand), namely End-user Subscribed Service (ESS). The consumption involves a series of interrelated multimedia intensive processes such as content creation, aggregation, annotation, ESS publish, etc., which are encapsulated into Web services in the context of SCCM, namely mul-

timedia intensive services, briefly multimedia services. The encapsulating process is called modeling multimedia application as a web service, which is a novel approach, consisting of analyzing, modeling, and implementing conventional multimedia applications in line with emerging Web services. The approach aims to regard a conventional multimedia application as a hosted service accessible to customers across the Internet. By partitioning the conventional heavyweight multimedia applications into lightweight pieces and providing them as Web services, multimedia services take obvious advantages of mobile device access, heterogeneous network convergence, and system scalabilities, and alleviates the customer’s burden of multimedia system cost, maintenance, ongoing operation, and support. User is a person who has a personal profile managed by the CCM System. They are the actors who interact with SCCM multimedia services for generating, processing and consumption of CCM multimedia elements. Service-oriented CCM (SCCM) refers to an architectural view of modeling CCM system which is analyzed and designed in compliant with principles of service orientation modeling (e.g. service discoverability, service autonomy, service composability) and implemented by utilizing Web services technology. P2P SCCM leverages the roles of requestors, providers, and registry center in conventional SOA architecture as service peers, which are assumed to publish its interface and access information to other peers, search for operations and resources exposed by other peers, and intermediate communication between peers. This is contrary to traditional three roles-based SOA model. However, P2P SCCM is not based on pure P2P architectures. Instead it uses super-peers as intermediary peers to act as rendezvous, namely rendezvous peers, which is responsible for service indexing and identity mapping.

253

P2P SCCM

USE SCENARIO AND EVALUATION FOR THE SERVICE-ORIENTED COMMUNITY COORDINATED MULTIMEDIA PARADIGM P2P SCCM Scenario P2P SCCM envisions that user experiences are enriched and extended by the collaborative consumption of multimedia services with the interplay of two key enabling technologies of web services and P2P technology. A brief description of the P2P SCCM paradigm (Figure 1) is as follows (Zhou, 2008b) (1) a user Bob records a video clip consisting of a real world event, say a named person or Figure 1. P2P SCCM scenario example

254

object. (2) a content analyzing service is requested for assisting the video content understanding. (3) a content annotation service is requested for the video content description specific to demanding detection levels (e.g., frames detection for the low level detection, face detection for the middle level detection, context detection for the high level detection). (4) Bob verifies the detected concepts, and optionally inserts descriptive value to the video content. (5) Bob publishes the produced content to the community network, where the peers of the network are notified with the aggregated video content. (6) content searching and filtering services are requested for facilitating pushing and pulling the multimedia content to the peers. (7) an

P2P SCCM

interested peer (Alice) is notified by the updated content. She requests a content download service and downloads the content to a local terminal via community network. (8) Alice creates comments on the content produced by Bob by requesting a content commenting service. (9) Alice selects Charlie and delivers her commented content to Charlie. (10) Charlie requests a content viewing service facilitating consuming the content and follows the bundled reference to a forum that contains discussions related to the multimedia content.

Scenario Evaluation Examining the above scenario, P2P SCCM is characterized by the following key features: •

•

Networked digital content and multimedia services. Content files containing audio, video, data or anything in digital format are very common in CCM. Multimedia services dealing with multimedia computing such as content annotation are networked and available by service discovery. Large user base. CCM usually has a large user base for sharing multimedia content and distributed multimedia computing purpose. The user base is usually organized in

•

•

•

terms of specific interest ties and membership management. Diverse connectivity between service peers. CCM uses diverse computer networks, Internet, and cellular networks for delivering the user content and multimedia services. Large network of collaborative nodes. CCM consists of large collaborative multimedia service nodes, which live on the edge of the network and can be composed for accomplishing a complex task. No distinction between clients and servers. CCM multimedia services do not have the distinct notion of clients or servers. The CCM multimedia service nodes usually simultaneously function as both “clients” and “servers” to the other nodes on the network.

SERVICE-ORIENTED CCM DESIGN CCM Service Layers A three primary service layers model was addressed in (Erl, 2005). To provide a CCM automation solution, we abstract two CCM service layers, i.e., CCM business service and application

Figure 2. CCM service modeling layers

255

P2P SCCM

service layers as Figure 2. This layered structure alleviates us to build services that take CCM business, application, and agility into considerations. Within this model, the CCM business service represents the most fundamental CCM building block, encapsulating a distinct set of multimedia business logic within a well-defined functional boundary and bringing the representation of corporate business models into the web services arena. CCM business services are considered fully autonomous and frequently expected to participate in service compositions. It is more than likely that CCM business services will act as controllers to compose available CCM application services to execute their business logic. The CCM application service layer establishes the ground level foundation that exists to express technology specific CCM multimedia processing functionality. CCM application services are responsible for representing technology and application logic. Their purpose is to provide reusable functions related to processing CCM multimedia elements within new or legacy application environments. CCM application services have the following common features: (1) expose multimedia functionality within a specific processing context; (2) draw upon available resources

within a given platform; (3) they are generic and reusable to achieve point-to-point integration with other application services.

CCM Service Design Continually, modeling CCM services requires defining a set of service candidates and their logical contexts. A high level service-oriented modeling process was presented in (Erl, 2005). Its customized procedure (i.e. decompose CCM process, identify CCM business services, and identify CCM application services) is applied to identify CCM services. Table 1 presents the refined CCM services by using task-centric service modeling method. Task-centric service modeling derives services from the sources of business process management and analysis of process workflow. Typical sources include use-case models and process description. A good example of this method is Business Processing Management Language (BPML) process modeling (Chang, 2005)(Andrews, Curbera, & Dholakia, 2003). The specification of CCM business and application services is given in the following sub-sections respectively.

Table 1. Services identified in service-oriented CCM Application service layer ESS-awareness

Business service layer

Application service layer Contentawareness

ESS publish

Discover service

Content creation

ESS guide

Registry service

Content annotation

ESS notification

User profiling

Content aggregation

Eventing

Content retrieval

Messaging ESS authorization ESS delivery ESS viewing

256

Searching

Content collection

Streaming

Content sharing

Downloading Adaptation

Content viewing

Annotation

Content moderation

P2P SCCM

CCM Application Service Specification CCM application services are categorized into two groups of content-driven and ESS driven application services as Table 3. Respectively, CCM is designed for supporting both content-driven and ESS-driven application development.

TUNNELED HIERARCHICAL P2P MODEL In P2P SCCM scenario (Figure 1), multimedia services are distributed in P2P overlay networks. P2P SCCM leverages the roles of requestors, providers, and registry centre in conventional SOA architecture as service peers. Those multimedia service peers are assumed to publish its interface and access information to other peers, search for operations and resources exposed by other peers, and intermediate communication between peers. There are many alternatives and design options to building P2P systems. Five common topologies are star, bus, ring, hierarchical, and

mesh (Robert Flenner, Michael Abbott, Toufic Boubez, & et al., September 22, 2002). Hybrid P2P architectures are now widely adopted to improve important performance characteristics of P2P systems, such as service lookup and content management. For the same reason, a tunneled hierarchical P2P model is designed as Figure 3. The tunneled hierarchical P2P network topology resembles a hybrid tree. The service nodes (i.e., rendezvous service in Figure 1) above each peer node (i.e., peer service in Figure 1) act as a central point of control for peer nodes directly below. The rendezvous service node duplicates all the resource indices and maps relationships between IP addresses and node Identifiers. The tunneled hierarchical P2P model (THM) is designed based on the following assumptions: •

•

The first overlay consists of concrete multimedia services; the second and third overlay consists of rendezvous services which manage peer service registries and indices. The whole set of peer services is categorized into three groups according to their

Table 2. CCM business service specification Name

Specification

Registry

Service is utilized by registering other services so they can be discovered and composed.

Discovery

This service provides means to discover and utilize other services that are registered to SCCM service platform

User profiling

This service provides means for CCM applications and services to manage community and user contacts, and deliver personalized mutlimedia over different networks (e.g., IPTV, Web, mobile, P2P).

Eventing service

This service provides means to raise and listen to events occurring in the CCM system.

Messaging service

This service provides means to send messages to a predetermined target or multiple targets in CCM system.

Searching service

This service provides means to search CCM elements (i.e., user generated conent or ESS) by matching the query.

Streaming service

This service provides means to establish a content stream between two nodes regardless the network they reside in.

Downloading service

This service provides means to enable a content download between two nodes regardless the network they reside in.

Adaptation service

This service provides means to process CCM multimedia elements in order to adapt the content according to specific multimedia consumption devices.

Annotation service

This service provides means to automatically or semi-automatically extract metadata from the content or manually insert new metadata for it.

257

P2P SCCM

Table 3. CCM application service specification Name

Content-driven CCM

ESS-driven CCM

Specification Content creation

The end user uploads content to the CCM system by importing content.

Content annotation

The end user adds extra information to an uploaded content.

Content aggregation

The end user integrates external content to the aggregating content, e.g., importing a piece of music into a clip of video, etc.

Content retrieval

The end user searches content database for a given purpose.

Content collection

The end user collects and saves the retrieved result.

Content sharing

The end user shares content with his contacts.

Content viewing

The end user views content.

Content moderation

The CCM administrator supervises content and makes sure that content is safe, legal, and non-offensive.

ESS publish

The service provider develops or registers a service to the CCM system.

ESS guide

The service guide provides the end user with information on the available services.

ESS notification

The CCM system informs the end user the upcoming service.

ESS authorization

This service provides means of protecting services delivered within CCM.

ESS delivery

The content stream is distributed over variant networks, e.g. DVB-H or IP.

ESS viewing

With this service, the end user can play, forward, backward ESS content.

Figure 3. A tunneled hierarchical P2P model

popularity, i.e., ‘hot service’ with a highest popularity, ‘warm service’ with a medium popularity, and ‘cold service’ with a lowest popularity. The frequency of service lookup is used for evaluating service popularity. In this way, from the top down, in the vertical perspective, all the nodes from different sub-overlays constitute multiple trees rooted by the nodes in the upmost sub-overlay. From the horizontal

258

perspective, there are also multiple sub-suboverlays (denoted as SSOs) in each sub-overlay, according to different locations or operators which prevent them from communicating with each other directly. In the tunneled hierarchical P2P model, multimedia service request is generally handled in the local overlay with a priority, if no proper result is found then the request will be forwarded to the upper level overlay until a proper result is found or a failure response is returned. The

P2P SCCM

Table 4. Notations and description in performance analysis Notation

Description

P(n/H)

The probability of one successful lookup through n SSOs, n=1,2,3, in the condition of ‘hot service’.

P(n/W)

The probability of one successful lookup through n SSOs, n=1,2,3, in the condition of ‘warm service’.

P(n/C)

The probability of one successful lookup through n SSOs, n=1,2,3, in the condition of ‘cold service’.

EH

The average number of SSOs for one successful lookup of ‘hot service’.

EW

The average number of SSOs for one successful lookup of ‘warm service’.

EC

The average number of SSOs for one successful lookup of ‘cold service’.

NTHM

The average number of SSOs for one successful lookup in the tunneled hiearchical P2P model.

ETHM

The average number of hops for one successful lookup in the tunneled hiearchical P2P model.

DTHM

The average session setup delay in the number of hops in the tunneled hiearchical P2P model.

o(k)

Higher-order infinitesimal with respect to k.

above service lookup mechanism is improved with repsect to the requested service popularities and through buillding up vertical tunnels between lower and upper overlays. When looking up ‘hot services’, the service lookup procedure goes as the usual. When looking up ‘warm services’ and ‘cold services’, vertical tunnels are built up connecting lower overlay and upper overlay directly for speeding service lookup. Vertical tunnels are built connecting the first and second overlay for ‘warm services’, while connecting the first and third overlay for ‘hot services’. Examples are tunnel 1 for ‘warm services’ lookup and tunnel 2 for ‘cold services’ lookup in Figure 3. Detailed description of the tunneled P2P hierarchy is given in (Ou, Zhou, Harjula & Ylianttila, 2008).

PERFORMANCE ANALYSIS OF THE TUNNELED HIERARCHICAL P2P MODEL Notations and Description In this section, a theoretical performance analysis of the tunneled hiearchical P2P model is given. Firstly, we put forward some assumptions to make the performance analysis much clearer (Table 4).

1. Each SSO in each sub-overlay equally has k nodes. The third sub-overlay has only one SSO, the second sub-overlay has k SSOs, while the first sub-overlay has k2 SSOs. Thus, the total number of nodes in the third sub-overlay is k, while the second suboverlay k2, and the first sub-overlay k3. 2. Each node in the first sub-overlay provides one service. Each service has equal probability to be distributed in each node in the first sub-overlay. The nodes from the second and third sub-overlays just duplicate the service indices and mapping relationships provided by the nodes from the first sub-overlay. They do not provide services themselves. 3. The average lookup hops in each SSO with k nodes is log(k) according to Chord (Stoica et al., 2003). 4. PH, PW, PC stands for the probability of ‘hot service’, ‘warm service’ and ‘cold service’ respectively. Thus, 0<=PC<=PW<=PH<=1 and PH+PW+PC=1.

Average Number of Lookup Hops in the Tunneled Hierarchical P2P Model In the tunneled hierarchical P2P model, ‘warm service’ can be forwarded to the second suboverlay directly while ‘cold service’ forwarded to

259

P2P SCCM

the third sub-overlay directly. Thus the number of SSOs for one successful lookup of ‘warm service’ is at most two, while for ‘cold service’, one SSO, located in the third sub-overlay, is adequate to find the proper service as it has all the resource indices and mapping relationships backup for the nodes from the lower sub-overlays. Thus:  P (1 / H ) = k / k 3   2 3  P (2 / H ) = (k − k ) / k   3 2 3  P (3 / H ) = (k − k ) / k   N 3   2 E H = ∑ nxP (n / H ) = ∑ nxP (n / H ) = 3 − (k − 1) / k   n −1 n −1 

(1) For the ‘warm service’, at most two SSOs are enough for a successful lookup.  P (1 / W ) = k 2 / k 3   3 2 3  P (2 / W ) = (k − k ) / k    P (3 / H ) = (k 3 − k 2 ) / k 3   N 3   E H = ∑ nxP (n / W ) = ∑ nxP (n / W ) = 2 − 1 / k   n −1 n −1 

(2)

For the ‘cold service’, it is always one SSO for a successful discovery. Ec = 1

(3)

So the average number of hops for one successful lookup in the tunneled hiearchical P2P model is:    ETHM      = [ + + ] log( ) P xE P xE P xE x k  H H W W C C    2  = [1 + PW + 2PH − ((k − 1) / k )xPH − (1 / k )xPW ]x log(k )     = [2 + PH − PC + o(k )]x log(k )    

(4)

Equation 4 shows that the average number of service lookup hops is decided by the difference between PH and PC.. Given PH=0.5, PW=0.3,

260

PC=0.2, then ETHM=2.3log(k), while in the flat architecture introduced in (Stoica, Morris et al. 2003) with k3 peers, the average number of hops is log(k3)=3log(k). So the average number of hops of the tunneled hierarchical model can be reduced in comparison with traditional P2P models.

Content Annotation Service Prototype This section presents our experience on content annotation service prototype development. One of the most significant challenges facing the community coordinated multimedia service development is the augmentation of semantically rich descriptions to the multimedia content. Due to its time consuming nature, structured annotation has not become popular in contemporary prevalent multimedia services. Most of the available video services, for example YouTube and Flickr, rely on community created lightweight metadata, such as tags, free text, and user comments as the elaboration of available data. However, according to (Lassila & Hendler, 2007), the evolution of Web services will lead towards better structured semantic data. As the amount of multimedia increases on the web, there is a demand for services that would alleviate the creation of more structured metadata descriptions for the available content. This section introduces a content annotation service prototype and incorporates the application of face detection into the service structure.

Service Design and Prototype Our prototype service development starts with the design of content annotation service. In the SCCM service framework, the content annotation service is specified in the business layer. It operates by receiving a content annotation request from a service client, interacting with video structure extraction and face detection services. The focus of the content annotation service design is to create semi-automation to the annotation process and

P2P SCCM

distribute the computation over the Web. Due to the inherited ambiguities in automatic content understanding methodology, user based verification of annotation results are also offered. Following presents an example how the content annotation business logic is realized by encapsulating a face detection service. Figure 4 presents the details of service grouping for content annotation and service activity sequences for handling face detection. SOAP is used for parameter and video data, as well as transmission of MPEG-7 metadata between services. The MPEG-7 metadata contains the video structural decomposition, and any results from content annotation using XML based MPEG-7 elements (Manjunath, Salembier & Sikora, 2002). NetBeans 6.1 (NetBeans, 2009) is selected to as an integrated Java development environment (IDE) for the development and deployment of SCCM services. The interaction sequence between services for face detection is described as the following:

1. The service client sends a request of video annotation to the AnnotateVideoFile service. 2. The AnnotateVideoFile service invokes the ExtractVideoStructure service to download the video file. 3. The AnnotateVideoFile service invokes the ExtractVideoStructure service to extract the video file structure and update the MPEG-7 metadata file. 4. The ExtractVideoStructure service invokes the AnnotateVideoFile service to validate the structure information. 5. The AnnotateVideoFile service invokes the DetectFace service to download video file and detect face information. 6. The AnnotateVideoFile service invokes the DetectFace service to download the MPEG-7 metadata and update the file. 7. T h e D e t e c t F a c e s e r v i c e i n v o k e s AnnotateVideoFile servce to verify annotation results.

Figure 4. Face detection service deployment sequence

261

P2P SCCM

8. The AnnotateVideoFile service presents the service client with the annotation results and asks for a verification. The snapshot (Figure 5) demonstrates the Web user interface with the content annotation of video clips. The snapshot (Figure 6) demonstrates the Web user interface with the content annotation of images. By accessing our developed content annotation service, the user can do people detection from image or video files.

Application on Face Detection To develop the content annotation service, we start with face detection prototype implementation. Many algorithms and frameworks contribute face detection. In our initial experiment, we focus on cascade-structured algorithms offered by ViolaJones embedded in OpenCV (P. Viola & M. J. Figure 5. User interface with video annotation

Jones, 2004) and Wu’s (Jianxin Wu et al., 2008). Cascade classifiers are used for rare event detections, like face detection. A cascade face detector uses a sequence of node classifiers to distinguish faces from non-faces. The main advantage is its processing speed. A cascade detector can detect faces in real time. Our tested cascades include Asymboost, Adaboost, and Forward Feature Selection (Jianxin Wu et al., 2008). Moreover we modify Wu’s face classifiers and make some comparisons between Viola-Jones’ and Wu’s sample code for face detection focusing on false negative detection (i.e., a face that is not detected), false positive detection (i.e., a face detected where there is no face) and detection time. The OpenCV provides implementation of the Viola and Jones’ method, which uses haar-like features to make weak classifiers and combines them into a strong classifier. Wu’s face detector also uses cascades to build strong classifiers from weak classifiers. Both face detection methods use an integral image for rapid feature detection. Wu’s method determines that two detection rectangles are considered as Figure 6. The user interface with image annotation

262

P2P SCCM

detection of the same area if both of them overlap each other by 60%. With our modification we determine the detections to be the same if either of them overlapped the other by over 60%. This is also done in OpenCV face detection. but it is not clear what are the boundaries to consider two detections to be the same. This takes into account the situations where there are made notably smaller detections inside a bigger one. In the tests both detectors were used in a way that detections with no overlaps are discarded. Here the description of all OpenCV classifiers in Table 5 can be found in the XML files of the cascades which can be located at \data\haarcascades when you install OpenCV. Ada is abbreviation for Adaboost; Asym for Asymboost; FDA for Fisher’s Discriminant Analysis; LAC for Linear Asymmetric Classifier; and FFS for Forward Feature Selection. The test results (see Table 5, 6, and 7 and Figure 7) are obtained in Windows XP. Some tests

are also made in the Linux environment which gave some alternative results most notably with the detection speed. Wu’s face detector also presents a Forward Feature Selection classifier that improves face detection speed, but performs not as good as other classifiers. Our modified Wu’s method produces slightly better results and does not cause notable false negative detection with close range faces. Regarding suitability for service implementation, Wu’s algorithm is capable of producing slightly better detection results (3 false negatives, 1 false positive vs. 4 false negatives, 1 false positive) but the processing speed for the images is over 10 times more than that of Viola and Jones’ implementation in OpenCV. However in the Linux environment we noticed an increase in detection time while we were able to optimize the Wu’s code for better results. In our quick tests the OpenCV is only two or three times faster than the Wu’s code. Service deployment using OpenCV

Table 5. Face detection in OpenCV’s detector (Viola and Jones’ method) Classifier

False Negatives

False Positives

Detection Time (ms)

Frontalface_default

4

2

349.752

Frontalface_alt

6

1

300.563

Frontalface_alt2

6

0

279.01

Frontalface_alt_tree

10

0

260.836

False Negatives

False Positives

Detection Time(ms)

4

1

4496

Table 6. Face detection in original Wu’s detector Classifier cascade_Ada cascade_Ada_FDA

5

3

4316

cascade_Ada_LAC

5

1

4316

cascade_Asym

5

1

4535

cascade_Asym_FDA

4

2

4582

cascade_Asym_LAC

6

3

4707

cascade_FFS

8

0

2080

cascade_FFS_FDA

7

2

4019

cascade_FFS_LAC

9

3

3784

263

P2P SCCM

Table 7. Face detection in modified Wu’s detector Classifier

False Negatives

False Positives

Detection Time (ms)

cascade_Ada

3

1

4434

cascade_Ada_FDA

3

1

4331

cascade_Ada_LAC

5

0

4332

cascade_Asym

4

1

4567

cascade_Asym_FDA

3

1

4597

cascade_Asym_LAC

4

0

4691

cascade_FFS

8

0

2033

cascade_FFS_FDA

7

2

4050

cascade_FFS_LAC

9

1

3816

Figure 7. Experiments comparing cascade performances among three methods

264

P2P SCCM

implementation of Viola and Jones’ face detection is possible for both Windows and Linux operating systems, which makes it an attractive choice. Finally, in order to wrap face detector applications to the actual service, following features are required: (1) a queue system for servicing multiple requests, (2) MPEG-7 encapsulation of annotation metadata for inter-service communication, (3) SOAP messaging component for interservice orchestration (4) video data transmission management.

CONCLUSION P2P Service-oriented Community Coordinated Multimedia presents the user a novel paradigm of consuming multiple media through requesting multimedia-intensive Web services via diversity display devices, converged networks, and heterogeneous platforms within a virtual, open and collaborative community. To reach the paradigm, this chapter incorporates technologies of service-oriented computing, multimedia computing and P2P communication into CCM design. It starts with the analysis of CCM requirements and the technical features of service-orientation approach and P2P technology. A general P2P SCCM scenario is created and examined. Then, a service-oriented CCM model is designed mainly focusing on identifying and specifying CCM business services and application services. A tunneled hierarchical P2P model is described for improving service framework performance on service lookup. A service prototype is described with the case studies of AnnotateVideoFile service and face detection applications. The performance analysis of the tunneled hierarchical P2P model is presented with the average number of hops for one successful service lookup. Through adopting service-oriented computing, multimedia computing and P2P technology, the P2P SCCM model takes a few of advantages in advanced multimedia computing (e.g., content sharing and management,

collaborative multimedia computing, and pervasive multimedia computing) and accommodates characteristics of service composibility, discovery, legacy system leverage, etc. The future tasks are identified in realizing P2P SCCM, including: 1. Continuation of content annotation and face detection service development according to web service design patterns. 2. Take the service composition into consideration with content annotation applications development. The Java Business Integration (JBI) runtime environment and BPEL designer (NetBeans IDE,) provides services for executing service composition. 3. Comparing and optimizing face detection classifiers by using more image samples, and applying optimized cascade structured classifiers to multiple object detections, not only face detection. 4. Make a further system performance analysis based on the tunneled hierarchical P2P model by comparing average number of service lookup hops and service session setup delay to the other P2P models. 5. Design and develop SCCM P2P framework based on JXTA.

ACKNOWLEDGMENT This work is being carried out in the EUREKA IETA 2 CAM4Home project funded by the Finnish Funding Agency for Technology and Innovation (Tekes) and the project of SOPSCC (Pervasive Service Computing: A Solution Based on Web Services), funded in the Ubiquitous Computing and Diversity of Communication (MOTIVE) program by the Academy of Finland. Many thanks are given to colleagues of Jouni Sarvanko, Arto Heikkinen for the multimedia service prototype development.

265

P2P SCCM

REFERENCES Adamson, C. (2005). QuickTime for java: A developer’s notebook. Sebastopol, CA: O’Reilly Media, Inc. Andrews, T., Curbera, F., & Dholakia, H. (2003). Business process execution language for web services version 1.1. Retrieved from http://www-128. ibm.com/developerworks/library/specification/ ws-bpel/. Bradski. G. (2000). The OpenCV library. Dr. Dobb’s Journal, Computer Security, Chang, J. F. (2005). Business process management systems: Strategy and implementation. Boca Raton, FL: Auerbach. doi:10.1201/9781420031362 Croft, W. B. (1993). Knowledge-based and statistical approaches to text retrieval. IEEE Intelligent Systems and their Applications, 8(2), 8-12. Damljanovic, D., Tablan, V., & Bontcheva, K. (2008). A text-based query interface to owl ontologies. In: 6th Language Resources and Evaluation Conference (LREC). Hansen, F. A. (2006). Ubiquitous annotation systems: Technologies and challenges. HYPERTEXT ‘06: In Proceedings of the Seventeenth Conference on Hypertext and Hypermedia, Odense, Denmark. 121-132. Hoffert, E., Krueger, M., Mighdoll, L., Mills, M., Cohen, J., Camplejohn, D., et al. (1992). QuickTime: An extensible standard for digital multimedia. In Compcon Spring ‘92. Thirty-Seventh IEEE Computer Society International Conference, Digest of Papers, pp. 15 - 20 Jiehan, Z., Mika, R., & Mika, Y. (2008b). P2P SCCM: Service-oriented Community Coordinated Multimedia: modeling multimedia applications as Web services and experience. In IEEE Proceedings of Asia-Pacific Services Computing Conference (IEEE APSCC 2008), December 9-12, 2008, Yilan, Taiwan, PP. 145-149.

266

Lassila, O., & Hendler, J. (2007). (n.d.). Embracing “Web 3.0” IEEE Internet Computing, 11(3), 90–93. doi:10.1109/MIC.2007.52 Lin Ching-Yung. (2002). Universal MPEG content access using compressed-domain system stream editing techniques in Proc. IEEE Int. Conf. Multimedia & Expo, 2002, (2), 73–76. Manjunath, B. S., Salembier, P., & Thomas, S. (2002). Introduction to MPEG-7: Multimedia Content Description Interface (Eds.). New York, NY: John Wiley & Sons, Inc. Michael, K. (2004). Gesture generation by imitation - from human behavior to computer character animation. (PhD thesis), Boca Raton, FL: Dissertation.com. NetBeans IDE. (2009). Developer Guide to the BPEL Designer. Retrieved from http://www. netbeans.org/kb/61/soa/bpel-guide-overview. html#ggceg. Ou, Z., Zhou, J., Harjula, E., & Ylianttila, M. (2008). Truncated Pyramid Peer-to-Peer SIP Architecture with Vertical Tunneling Model. IEEE Global communications conference, 30 Nov. -4 Dec. 2008, New Orleans, LA, 1-5. Rautiainen, M. (2006). Content-based search and browsing in semantic multimedia retrieval. Faculty of Technology. Sean, S. Winzeler. L, Deagen, J., & Brown, D. (1998). Programming with the java media framework. New York. NY: John Wiley & Sons, Inc. SOAP. W3C. (2003). SOAP version 1.2 part 1: Messaging framework. Retrieved from http://www.w3.org/TR/2003/REC-soap12part1-20030624/ Stoica, R., Morris, D., Liben-Nowell, D. R., Karger, M. F., & Kaashoek, F. Dabek & H. Balakrishnan (2003). Chord: a scalable peer-to-peer lookup protocol for Internet applications,” Networking, IEEE/ACM Transactions on, 11, 17-32

P2P SCCM

Thomas, E. (2005). Service-oriented architecture (SOA): Concepts, technology, and design. Upper Saddle River, NJ: Prentice Hall. UDDI. (2004). UDDI version 3.0.2. Retrieved from http://www.Oasis-Open.org/committees/ uddi-spec/doc/spec/v3/uddi-v3.0.2-20041019. Htm. Viola, P., & Jones, M. J. (2004). Robust realtime face detection. International Journal of Computer Vision, 57(2), 137–154. doi:10.1023/ B:VISI.0000013087.49260.fb W3C-WSDL. (2005). WSDL: Web services description language (WSDL) 1.1. Retrieved from http://www.w3.org/TR/wsdl Wu, Jianxin, Brubaker, S. C., Mullin, M. D., & Rehg, J. M. (2008). Fast asymmetric learning for cascade face detection. IEEE Transactions on Pattern Analysis and Machine Intelligence, 30(3), 369–382. doi:10.1109/TPAMI.2007.1181 Yang, Ming-Hsuan, Kriegman, D. J., & Ahuja, N. (2002). Detecting faces in images: A survey. IEEE Transactions on Pattern Analysis and Machine Intelligence, 24(1), 34–58. doi:10.1109/34.982883 Ylianttila, M., Harjula, E., Koskela, T., & Sauvola, J. (2008). Analytical model for mobile P2P data management systems (J. Sauvola Trans.). In Proceedings of 5th IEEE Consumer Communications and Networking Conference. pp. 1186 – 1190. Zhou, J., Rautiainen, M., & Ylianttila, M. (2008a). Community coordinated multimedia: Converging content-driven and service-driven models. In IEEE Proceedings of 2008 IEEE International Conference on Multimedia & Expo (ICME 2008), June 23-26, 2008, Hannover, Germany, pp.365-368.

KEY TERMS AND DEFINITIONS Community Coordinated Multimedia (CCM): system maintains a virtual community for the consumption of CCM multimedia elements, i.e. content generated by end users and content from professional multimedia provider (e.g., Video on Demand). Multimedia: represents a synchronized presentation of bundled media types, such as text, graphic, image, audio, video, and animation. Multimedia Intensive Services: briefly multimedia services are interrelated multimedia intensive processes such as content creation, aggregation, annotation, etc., which are encapsulated into Web services. The approach aims to regard a conventional multimedia application as a hosted service accessible to customers across the Internet. By partitioning the conventional heavyweight multimedia applications into lightweight pieces and providing them as Web services, multimedia services take obvious advantages of mobile device access, heterogeneous network convergence, and system scalabilities, and alleviates the customer’s burden of multimedia system cost, maintenance, ongoing operation, and support. P2P SCCM: leverages the roles of requestors, providers, and registry center in conventional SOA architecture as service peers, which are assumed to publish its interface and access information to other peers, search for operations and resources exposed by other peers, and intermediate communication between peers. This is contrary to traditional three roles-based SOA model. Service-Oriented CCM (SCCM): refers to an architectural view of modeling CCM system which is analyzed and designed in compliant with principles of service orientation modeling (e.g. service discoverability, service autonomy, service composability) and implemented by utilizing Web services technology.

267

P2P SCCM

Web Service: is an autonomous, standardsbased component whose public interfaces are defined and described using XML (W3C) and that supports interoperable machine-to-machine interaction over a network using mainly Webbased standards.

268

ENDNOTE 1



http://www.cam4home-itea.org/

248

Chapter 15

Unravelling Intellectual Property in a Specialist Social Networking Site Sal Humphreys University of Adelaide, Australia

ABSTRACT This chapter examines how the complexity of motivations and practices found in a specialist social networking site intersect with the institutions of intellectual property. The popular niche or specialist social networking site (SNS) called Ravelry, which caters to knitters, crocheters and spinners, is used as a case study. In this site people use, buy, sell, give away, and consume in a mixed economy that can be characterised as a ‘social network market’(Potts et al., 2008). In a co-creative social networking site we find not only a multidirectional and multi-authored process of co-production, but also a concatenation of amateurs, semi-professionals and professionals occupying multiple roles in gifting economies, reputation economies, monetised charitable economies and full commercial economies.

INTRODUCTION Co-creative online social environments have many emergent qualities – they are evolving and fluid as new possibilities are encountered and developed (Banks and Humphreys, 2008). Intellectual property (IP) on the other hand is a legal instituDOI: 10.4018/978-1-60960-774-6.ch015

tion that is complicated, unwieldy, and based on industrial-style production and distribution models that are reasonably fixed, linear and inflexible in character (Benkler, 2006). The users of online social networking sites find themselves having to come to grips with the complexity of IP law in order to participate fully in the environment and its markets. For those that engage with it, IP is proving to be overwhelming and confusing. It

Copyright © 2012, IGI Global. Copying or distributing in print or electronic forms without written permission of IGI Global is prohibited.

Unravelling Intellectual Property in a Specialist Social Networking Site

may ultimately stifle the entrepreneurial gambits of some, and chill the creative efforts of others, without necessarily effectively protecting the creative work of those whom it is designed to benefit. This chapter explores conversations between users on a number of bulletin boards within Ravelry that shed light on attitudes, practices and issues that arise in the negotiation of IP law. It shows the differing interests professionals, semiprofessionals and amateurs have and highlights the co-existence of an emergent ‘norms-based’ IP System (Fauchart and von Hippel, 2006) alongside the law-based IP system. This combination of implicit and explicit rule systems is a reflection of the overall structure of the site which is one based on both social and commercial exchange economies. It reveals the intersection of these differing systems as contested and abrasive rather than seamlessly compatible. Analysis of discussions demonstrates that the discourse of the legal IP system is being used in the service of creating norms within the community that in some cases work to valorise commercial gain over social gain, and which can overstate the actual reach of the law. Thus the chapter explores the negotiation of IP law and norms by a mixed group of professionals, amateurs and semi-professionals in an environment where the affordances of the internet and social networking sites have created a great deal of uncertainty.

RAVELRY In this chapter an analysis of the discourses and practices around intellectual property is carried out on the specialist social networking site Ravelry. This site is for knitters, spinners and crocheters1 and has attracted over 600 000 users (in the two and a half years to March 20102), many of whom are very active contributors to the site. Individuals have their own profile areas on the site where they are able to upload photos of the items they are knitting, with details of the yarn they used and

the patterns they followed and any modifying for their own purposes. These details are aggregated for other users to find, and linked variously to commercial and non-commercial sites where the patterns or yarn can be found (sometimes for sale, sometimes for free in the case of patterns) both on and off the Ravelry site. Local yarn stores and libraries are also linked (with maps and contact details). Searches allow the user to browse photos of the multiple versions of a pattern that have been knitted by other users, thus allowing for them to see how the pattern knits up in different yarns, sizes, colours and variations/ modifications. Sometimes there are hundreds of finished versions of a particular pattern available for viewing. Comments about patterns and yarns are made, alerting people to their pitfalls or joys, there is a ‘favourites’ system which generates searchable popularity metrics in all available categories and so on. Much of the data available about the patterns and yarns has been previously available elsewhere on the net, but the aggregation of the data into one very user-friendly searchable database which draws on user-generated content has proved immensely popular. Designers are able to upload their patterns to either sell or give away, with a PayPal payment system in operation within the site. They can link to their own websites if they already have an online payment mechanism available there. Advertising is also available on the site, with both commercial retailers and individual designers paying for ads on the site. Ravelry also has very active discussion boards about not only all things associated with yarn, knitting and crocheting, but also about politics, tv shows, special interests and just about anything else imaginable. These boards are surprisingly well populated. By June 2008, a little over a year after the public beta test began, there had been over 5 million posts to the boards. Several of the boards maintain a reasonably concentrated focus on intellectual property – one, named ‘copyright matters’ has numerous threads dedicated to trying to unravel the complexity of

249

Unravelling Intellectual Property in a Specialist Social Networking Site

intellectual property as it relates to knitting and patterns and cross-jurisdictional confusions. Another board for Ravelry ‘shopkeepers’ regularly deals with copyright and licensing issues confronting the designers who sell through the site. The designers’ boards (which have both professional and amateur designers) are also a frequent source of debate about IP. The ease of publication, the ease of copying and the enormous reach of the distribution network have changed the characteristics of flows of information for this niche community. This study interrogates these changes and their impacts on existent cultures and on innovation potentials. Understanding the role of intellectual property as a commercialising mechanism can be useful in the analysis of interactions between sociocultural networks and commercial networks. Like modders in games communities (Banks, 2007, Postigo, 2008), participants have a variety of motivations, rationales and justifications for their stances in relation to commercial and noncommercial distribution of their work, and employ various discourses in their discussions and arguments on IP issues. What becomes clear in the analysis of this site’s discussion boards is that even with highly motivated participants who are trying to ‘do the right thing’, the legal quagmire that is intellectual property law defeats many. This body of law is clearly inadequate to the task of catering to mixed economies and diverse motivations as well as the changed distribution channels available. One effect is that micro-businesses which could be the source of so much innovation, and which can provide a pathway for designers and innovators into more commercial production, can founder on the rocks of copyright, licensing and patent law. If a designer with a few patterns wants to sell on Ravelry they have an opportunity previously unavailable to them to market and distribute their designs commercially. Going to the expense of seeking legal advice on copyright and on licensing across global jurisdictions is a strong disincentive for some.

250

The other effect analysed in this study is the way the discourses of copyright and licensing are drawn upon to support a valorisation of commerce over gifting. IP can be seen to be used in the service of a shift from gifting to commercial culture as the dominant mode of exchange. However the success of such a shift is by no means a given, and the outcomes are still emergent. In this paper I give some preliminary analysis derived from the discussion boards of the site.

INTELLECTUAL PROPERTY AND NETWORKED PRODUCTION When intellectual property law and in particular copyright law was introduced in the US it was designed as a balancing mechanism between the needs of individual authors for recompense for their work, and the understanding that culture and the production of new knowledge proceed through the building on, and the continual reshaping and extension of, existing works. As initially conceived, copyright allowed for a limited period of time during which an author or authors could charge for the use of their work before it became part of the public domain – available for other creators and innovators to use and build upon. Creativity and authorship were thus conceived of as partially fed by what had gone before (See for instance Bettig, 1992, Boyle, 2002). Appropriation is a major part of what drives cultural innovation. Thus, historically, copyright was intended not as a ‘…property right but a policy that balanced the interests of authors, publishers and readers’ (Coombe, 2003:1). It was an incentive framework that recognised that all new work builds on existent works and therefore only grants a limited monopoly right as a kind of necessary evil in a market economy. It could be seen as an attempt to facilitate the intersection between commercial and cultural imperatives. Over time the balance between authors’ rights and the public domain has been eschewed in

Unravelling Intellectual Property in a Specialist Social Networking Site

favour of an increase in the rights of authors and a shrinking of the public domain. The original copyright period of 7 years (with the possibility of a further 7 year extension) has now extended to the lifetime of the author plus 70 years in the US (Sonny Bono Copyright Extension Act of 1998), with similar trends elsewhere. This shift, through which the role of the author and the role of the public domain more generally have gradually been reconceptualised, is important when considering the emergent forms of authorship, co-creation and collaboration that we are witnessing in networked environments (Vaidhyanathan, 2004). While legal institutions (and parliaments under pressure from industry bodies) have moved to place the emphasis on the singularity and importance of authorship, the processes enabled through networking technologies have tended to bring to the fore the processes of collaboration, reuse, recombination, and co-creation. As Coombe notes: Emphasis upon intellectual property tends to exalt originality rather than creative variations, singular authors rather than multiple interpreters, canonical works rather than social texts, and to privilege a moment of inscription over the process of ongoing appropriation. (Coombe, 2003:2) Broadly speaking, in global policy terms, we have seen a move to include intellectual property as part of trade regimes rather than as part of cultural policy (Frow, 2000, Drahos and Braithwaite, 2002). It is a shift which favours the viewing of all creative production as property, much like stock, to be traded and owned (Smiers, 2002). This discursive practice serves to minimise any consideration of characteristics such as cultural value and the value of affect and networks in production. It is a powerful force in defining what matters and what doesn’t. Digital media and in particular online digital media have presented a series of challenges to the maintenance of IP conventions and laws (Lessig, 1999, 2004). The first and most public

of challenges has been centred on the changed distribution and reproduction affordances offered by the new media forms. Thus we have seen court battles over copying and distribution by users of copyrighted works. Napster is a salient example (Rimmer, 2001). But it is necessary also to consider how ease of publication has engendered a whole new cohort of authors who publish outside of conventional publishing institutions. These authors tend to have little access to legal advice about their rights and obligations as authors, just as domestic users of networked media have little access to legal advice about their use, reproduction and distribution of existent media. Intellectual property and its conventions are built on an industrial era model of publication, where authors mostly needed to pass through the gate-keeping institutions of publishing houses in order to publish and distribute their work. Thus production, distribution and reproduction within this system are regulated through mechanisms which assume a very linear process from author to consumer through a publisher. The networked environments of the internet undermine this process and also its conventions, in part through the circumvention of publishers. This applies most obviously and most publicly to reproduction and distribution on the internet. Less attention has been given to the fact that authors are now able to self-publish and distribute widely without recourse to publishing houses. Cultural production in the form of mash-ups, modding and other co-creative activities all present challenges as the consumers become more noticeably producers and innovators. Fauchart and von Hippel (2006) explore the existence of what they term ‘norms-based’ intellectual property systems. These are systems for the regulation of intellectual property that exist outside, or instead of, law-based systems. They exist in communities which have established a set of social norms that they use and enforce through social regulation. Although Fauchart and von Hippel are concerned to identify the features of such systems that are separate from a law-based

251

Unravelling Intellectual Property in a Specialist Social Networking Site

system, they also point to the existence of ‘mixed’ systems that incorporate both. “Law and norms do not generally operate in separate spheres. Rather, they typically operate either to support or subvert each other.”(Rai, 1999, cited in Fauchart and von Hippel, 2006). In their work on flows of information (about recipes and techniques) among haute cuisine chefs in France, whose recipes are not covered by copyright, they demonstrate that the flow of information is highly regulated by three main norms established within that community. These norms are about direct copying, rules for sharing of information based on trust and expectations of reciprocity, and attribution. They also look at the social mechanisms used to enforce these norms, which in the case of chefs includes a system where transgressors are publicly shamed, access to information flows is cut, and reputation damaged. Reputation is seen as having a direct relationship to economic success with this community. They point out that such systems only work if the member of the group targeted for censure actually cares about these things. Thus for the system to be effective it must have ‘buy in’ from its members. Those for whom such censure makes no difference will obviously be immune to the system and its norms. Knitters and knitting pattern designers will clearly have a different set of norms, but the basic principles of establishing socially-based norms, which fulfill agreed upon functions and also having mechanisms of enforcement will hold in this environment as well. This norms based system may either serve similar functions to law-based systems or provide alternative functions. Using the framework of a norms-based system in this analysis lends intelligibility to the notion that there are regulating mechanisms beyond the law at work and allows that they may be socially derived. In my analysis of the discussion boards of Ravelry it will become clear that there is not yet a clearly defined set of norms in place within this community, but there is an attempt of sorts to overcome the inadequacies of the legal system

252

through imposing a socially-based set of norms. Exactly what the norms are however, in whose interests they operate, and whether they encourage innovation, is not settled.

MARKETS AND MECHANISMS WITHIN SPECIALIST SOCIAL NETWORKING SITES An online networked production environment changes the mechanisms for production, publication and distribution that the legal institution of IP presumes. A social networking site, unlike the broader internet, gives a more focused view of these changed processes, with the relations between not only people, but objects, made more explicit and visibly articulated. Boyd and Ellison offer the following definition of social network sites as: Web-based services that allow individuals to (1) construct a public or semi-public profile within a bounded system, (2) articulate a list of other users with whom they share a connection, and (3) view and traverse their list of connections and those made by others within the system. (Boyd and Ellison, 2008: 211) To this definition, in the context of a specialist SNS such as Ravelry we can add that the environment also makes it possible to view and traverse the connections between objects – it allows users to witness the variations, modifications and derivations that emerge, as well as to track the use and reuse of objects such as patterns. There are four aspects or qualities of SNS sites that throw into sharp relief the debates about IPin new media environments. Each of these features can be seen more broadly on the net, but within the confines of an SNS environment the qualities of aggregation and visibility bring them into sharp relief. The first quality is that of the lowered barriers to publication. The ability of small scale

Unravelling Intellectual Property in a Specialist Social Networking Site

designers, both amateurs and those who wish to sell their work, to publish without having to deal with the gate-keeping mechanisms of the large conventional publishing houses has resulted in a flood of self-published content. This also implies a large cohort of authors not literate with copyright or IP more generally, and unaware of what their rights and obligations as publishers might be. Secondly, the ease of digital copying is a quality found more widely on the net, but has particular implications within a specialist or niche SNS. For instance, while knitting patterns have been available on the internet for some time, the concentration of patterns available within this specialist SNS environment is new. The site’s capacity and preparedness to make information more accessible, and to aggregate that information for ease of use by participants, has heightened the awareness of how easy it is to copy and how little users understand about copyright. Thirdly, the global distribution enabled by the internet is again brought into sharp focus within the specialist SNS. Self-publishing designers have never had such easy access to such an enormous distribution network. Previously if designers published through a publishing house they would very rarely have had to deal with the cross-jurisdictional issues of IP – this would be handled by the publishing house legal team. Now, on top of needing to come to terms with the issues in their home jurisdiction – challenging enough – they need to understand what their rights and obligations are in multiple jurisdictions across the globe, which are visible and present in the site. There are designers from many countries selling their work to users in many other countries through the site, and each has to come to terms with the variations in law, as well as the variations in cultural practices that emerge through such a site. These three first elements all have as an underlying factor the issue of scale. The fourth and final aspect of specialist SNSs I want to identify as representing a significant shift and therefore a challenge to conventional

institutions is the co-existence within the same environment of amateur, semi-professional and professional users. While Leadbeater and Miller describe some of this phenomenon in their work on the pro-am revolution, they fail to problematise the friction that this turn toward the harnessing of amateur inputs into professional contexts creates (Leadbeater and Miller, 2004). Benkler (2006) has discussed at length the differing structures, motivations and benefits of market and ‘non-market’ networks. Social and gifting economies rely on motivations quite different from commercial economies. People operating within a non-market economy tend to be motivated by different rewards – nonfinancial rewards such as social status, the intrinsic reward inherent in some creative activities, the idea of giving back to the community to which the user belongs and so on. Fauchart and von Hippel’s exploration of the norms-based IP system among chefs is a testament to this alternative set of rewards. Benkler also points to the many efficiencies of being able to tap into such a large pool of talent and know-how – what Pierre Levy (Levy, 2001) would call collective intelligence – and the different organisational structures generated by these arrangements. What is perhaps more interesting than this non-market/market dichotomy are the hybrid market environments where there is no such clear distinction between the social and commercial economies – where instead they co-exist in the same space, and where some people occupy different positions over time within the same markets. It’s possible to identify environments where people both give away and sell their content (Banks, 2007, Banks and Humphreys, 2008). People negotiate and occupy positions within social, reputational, gifting, and commercial economies – sometimes simultaneously, sometimes sequentially. In this complex environment, social networks matter as much as commercial or financial networks and each shapes the other to some extent. Potts et al (2008) suggest that economists and policy makers need to understand creative industries in

253

Unravelling Intellectual Property in a Specialist Social Networking Site

terms of these social network markets. Their work provides a useful start in trying to understand the mechanisms at work in social network markets, and the ways in which social networks influence both production and consumption within creative industries. Sites where the economic model relies on user-generated content, and where that usergenerated content is not necessarily financially remunerated provide a good starting point for analysis (see Humphreys, (2009) for an analysis of Ravelry as a social network market). This is also where the model of norms-based intellectual property provides further illumination, as it can be used to conceptualise regulation based in systems that might value something other than commercial recompense, using enforcement outside of the legal systems that usually serve the interests of commerce. Different Specialist Social Networking Sites use different business models to generate income. Some provide combination shareware plus premium services (for instance Cellar Tracker), some are run by commercial enterprises as a brand promotion exercise (for instance SkiSpace), some rely on advertising and so on. Some allow and encourage users to create their own commercial ventures within their site and others prohibit commercial activity. Digital games are a prime example of modding communities being encouraged in their content creation and innovations and yet are often prohibited from doing so for financial gain (The Sims is illustrative). The strength of the social networks generated within a site, and the willingness with which users will contribute their own content to databases obviously drives the economic success of these sites for the owners, but also for those users who are engaged in commercial activities. The mechanisms which coordinate the range of activities within a specialist SNS could be considered emergent, rather than fixed, at this stage. How these emergent forms articulate a relationship with existent forms is of key interest. Attention should be paid to an institution such as intellectual prop-

254

erty and its ‘fit’ with new practices – how it shapes the practices of users, how it is being reformed by them in turn, and whether some new mechanism of coordination may be emerging. If exchange is not all commercial, how can a mechanism of property, designed to coordinate ownership in a commercial landscape, adapt? If the number of copyright holders has exploded into millions who access distribution without recourse to an established publishing house and legal experts, what is to be done about the behemoth of copyright law that is incomprehensible to all but the most specialised legal minds? Norms-based systems are a rational response to such unintelligibility, but are only really effective if community norms can be largely agreed upon. This study attempts to describe the social encounter with a legal institution, rather than the institution itself. It will also become clear from the analysis below that the discourse of intellectual property is mobilised in the service of various self-interested arguments on the part of users – it can be used in the service of cultural norming, mobilised in an attempt to enforce a particular ethics or to valorise particular behaviors over others. It is thus important to understand it not only as a series of legal mechanisms, but also as a site of ideological contestation.

RAVELRY AND THE PARTICULAR CHARACTERISTICS OF ITS SPECIALTY AREA The particularities of each specialist SNS will obviously impinge on the kinds of practices that emerge from it. A specialist SNS is tapping into preexistent cultures and networks which will bring to the site their own traditions and peculiarities. I want to detail here some of the characteristics of the knitting and spinning culture that exist outside the SNS and that are brought into the site by users, recognising the ways in which existent forms affect emergent forms.

Unravelling Intellectual Property in a Specialist Social Networking Site

Knitting and spinning are crafts that have a long history and many long-standing traditions. While knitting in earlier centuries was often done as piecework, by women who had access to few means of economic independence, or who contributed to the meagre income of a poverty stricken family, the advent of machine looms and knitting machines (Abrams, 2006) has meant that hand knitting has become less of a stable source of income and more of a craft with a place in a gifting economy. This is not to deny that there are commercial outlets for hand-knitting, but to suggest that knitting itself is done less for economic gain now. There is still a strong utilitarian function to knitting, but also a strong culture of gifting. It is mostly women who knit and it is mostly for themselves, their friends and family. It should be noted then, that Ravelry is a gendered site. Male knitters are certainly present but the majority of users are women and this gendered quality may impact on the nature of discussions, and the associated cultures of sharing, co-operation or competitiveness. There is also a strong practice of knitting for charity, whereby garments are knitted for sale for charity – thus bridging the non-commercial/commercial dichotomy in an interesting way. Motivations may be firmly within the social economy, but outcomes reside in both the social and monetary economies. Secondly knitting results in ‘utilitarian’ items – objects which under US law cannot be copyrighted. Thus in the US, patterns – the written instructions for making a garment – are protected by copyright, but the knitted garment usually will not be, and can be legitimately sold in most cases, without reference to the designer. This is not the case in other jurisdictions such as the UK and Australia where three dimensional objects will in some circumstances be protected by copyright and may also be protected by registration as a ‘design’. This protection restricts the making, distribution and commercial use of such objects. As with all things in the legal arena there are exceptions and nuances to these broad statements, but it is im-

portant to have a passing understanding of some of the particular cross-jurisdictional problems confronting this community. These aspects of the legal complexity are really pretty confusing to the average knitting amateur wanting to participate and contribute to the site and the culture. The constant conflation of the written pattern with the knitted object by most knitters is obviously problematic given these legal details. Thirdly, it seems that with paper-based publications for knitting, there has been an under-theradar economy of sharing patterns with friends, family and local knitting groups. This small-scale practice of breaching copyright becomes visible and problematic in the large-scale social distribution networks of internet. (There are parallels here obviously with the sharing of music, where people used to make cassettes of their records for friends but the scale of mp3 sharing put the practice in a new light). Fourthly, knitters modify patterns all the time. They create derivative works, they amend patterns, and they design their own garments all the time. It is standard practice within the craft to take a pattern and use it to your own ends. It is common to see a garment you like and make up your own pattern based on what you saw – a kind of reverse engineering. This really only becomes problematic in the online publishing environment, where knitters are always sharing their tips on how to modify a pattern, or are writing down and sharing the pattern they made up, based on what they saw somewhere else. This is part of how innovation occurs – it is a vital part of the process of experimentation and research that goes into the creative activity of design and innovation. There is an ongoing discussion on the site, that is reignited regularly, about what constitutes an original work and what constitutes a copy, a modification or amendment. There is absolutely no agreement about this, and sadly the legal framework provides no certainty either. However intellectual property and copyright are used in discussions that seek to prevent people publish-

255

Unravelling Intellectual Property in a Specialist Social Networking Site

ing their designs. It is interesting to notice how this particular environment not only provides affordances for new practices, but also reshapes cultural etiquette.

I want to turn now to an analysis of the discussion boards on Ravelry and how these discussions play out. At some stage it would be good to track the actual use of objects through the site – to trace the lines of inspiration and modification, of use and re-use. This would require a level of access to the database not available for this research. Some observations can be made, but given the vast size of the site and database, it would not be possible to claim empirical coverage of the whole site without some data mining tools and access to the database.

the study doesn’t make a claim to represent all users of Ravelry and their attitudes to and practices around IP. Rather it explores the explicit conversations held by those most concerned with issues around IP in the hope of understanding the articulations of old institutional forms with new assemblages afforded by the internet. Some of these articulations are specific to the site and to knitters, but some are more generalisable and arise from structural changes that are more widespread. What is presented here is an analysis of a single thread of a discussion board, chosen for its representativeness and supplemented with postings from other threads where this particular thread missed a salient issue. I have chosen this method of demonstrating through a single example in the hopes of maintaining some clarity to the discussion, but also to illustrate that a single thread often covered enormous amounts of territory and conveyed many different attitudes and understandings.

Method of Analysis

Overview

There are over 6000 discussion boards on Ravelry and this study doesn’t purport to represent a comprehensive survey of them. For a period of 9 months I followed discussions on four main boards where discussions about intellectual property arose most often. These were the “Copyright Matters” forum, the “Designers” forum, “Patterns” forum and “Designers and crafters working together” forum. I kept tabs on a variety of other boards as well. I also read through many archived conversations on these boards that pre-dated my study. The main boards were checked a number of times a week and new discussions concerning IP were marked using the online software markup program Diigo. Comments were tagged into categories generated through a grounded research methodology. A discourse analysis was performed to identify the characteristics of differing attitudes toward intellectual property, community values, commercial values, reputation and so on. As such,

What the discussion boards reveal is a community where there is neither consensus over what constitutes legal behaviour, nor over what constitutes ethical behaviour in relation to the publication and commercial exploitation of patterns. Thus both law-based and norms-based systems are contested: the law-based systems due in part to their complexity and impenetrability; the norms-based systems due to varying degrees of investment in them by differently motivated users – often distinguished by their status as amateurs, semi-professionals or professionals. The discussions about copyright and intellectual property in Ravelry often follow similar trajectories. This is not to say the contributions come from the same people all the time, but the same ideas and arguments arise, and the same lack of consensus emerges in almost all of these threads. On one discussion board it was suggested that a resource page about copyright be put together and

THE RAVELRY DISCUSSION BOARDS

256

Unravelling Intellectual Property in a Specialist Social Networking Site

a small pop-up pointer to the page be provided to users at various points in their use of Ravelry, in an effort to avoid these repeated discussions. One response to this suggestion was a graphic based on the Microsoft ‘help’ paperclip graphic with text which reads: It looks like you’re starting a copyright discussion. You might also enjoy: bashing your head against walls; squeezing lemon juice into your own eyeballs. (Poster One3, Patterns forum, March 2009) I want to start this analysis by using one particular thread as illustrative. I chose this thread because it highlights quite a range of issues within the one discussion, but each of these issues can be found in many other places in other threads as well.

The Question The original poster asked a question about the following situation. A local bagpipe band had a woman who knit kilt ‘hose’ (stockings) for them. She died, leaving no pattern for the hose. The band approached the local yarn store and asked if they could work out what it would cost to get someone to knit the same hose. They left a sample. The poster had received the sample and worked out how to knit up a copy for them and what the costing would be. Now she wanted to know whether she could legally publish a pattern that she had written down from this process and if she could sell it:

FYI, I have no idea if the woman used a commercial pattern for these socks, or whether she just made it up as she went along. (Poster 2, Designer forum, July 2008) There is a recognition here that legal and ethical issues are not necessarily the same, although this distinction is not always acknowledged and some posters will conflate the two. It is an indication that there are both explicit and implicit rules regulating this issue.

Confusion about Commercial Gain and Copyright There were many replies, starting with: I think it makes a difference whether you are trying to sell the pattern or publish it for free. (Poster 3, Designer forum, July 2008) This initial response is basically probably incorrect but does demonstrate what seems to be a common misconception that many people hold about copyright. This is the idea that as long as you don’t make a commercial gain from a copy it’s ok to use it however you want. This misconception is also found in peoples’ practices with music. It posits copyright as akin to the Creative Commons non-commercial licence4 in its underpinning logic.

The Difference Between Patterns and Objects

So the question is…if I wanted to publish the pattern I worked up, does copyright law restrict me from doing so? Anyone have any copyright expertise in the area of publishing redacted patterns?

The reply to this post while correctly addressing some of this, fails to address whether it actually is a copy of a written pattern (it’s not), and in some ways muddies the waters further.

And heck, if you want to weigh in on whether you think I’m morally corrupt for thinking of doing this, feel free.:)

Legally it doesn’t make any difference at all. If using the patt is a copyright infringement, then publishing it for free still impacts the actual copyright holder as it can undercut sales for them, besides which they have the right to determine

257

Unravelling Intellectual Property in a Specialist Social Networking Site

under what conditions their pattern is distributed. If the copyright holder has registered the copyright and sues, then someone posting the patt for free is still liable for damages. (Poster 4, Designer forum, July 2008) Registering patterns for copyright is not a requirement in most jurisdictions – it is an automatic right. Following this we get to the ‘it’s complicated’ response. Did you reconstruct the sock w/o seeing the original pattern? In that case, it is not copyright that is involved at all. Copyright has to do with copying or reproducing the pattern, not the original item. The answer of whether it is legal depends on where you live and where you are doing the work and maybe where you are selling it. In the US, it is perfectly legal to re-engineer an item w/o seeing any of the original instructions. … In other countries, apparently it is illegal to copy an item that way. Caveat: I am not a lawyer nor do I play one on the internet. (Poster 5, Designer forum, July 2008) This answer covers the legal ground reasonably well. It doesn’t give a definitive answer (because there isn’t one).

The Confusions of Cross-Jurisdictional Issues It raises the cross-jurisdictional issues in play and gives advice about US law. One poster responds to the statements about US copyright law with: I was responding specifically to “In the US, it is perfectly legal to re-engineer an item w/o seeing any of the original instructions.” which just isn’t always true. (Poster 6, Designer forum, July 2008)

258

The original poster is Canadian however and there is no Canadian-specific response forthcoming in this particular discussion. There is confusion in most discussions of cross-jurisdictional issues about which law would apply – when you have buyers in one jurisdiction and sellers in another, performing transactions through a site possibly based in a third jurisdiction (if both buyer and seller are outside the US).

Differences Between Copyright, Design and Patents At this point the previous poster introduces further complexity by inserting design and patent law into the conversation. Actually, some clothing designers protect even their /designs/ and don’t allow imitations to be made. If you’re not familiar with the hullaballoo in the cloth diapering world, someone patented a type of cloth diaper called a pocket diaper and now no one is legally able to sell their version of a pocket diaper no matter how many modifications they’ve made, whether they’ve even ever seen one of the actual patented diapers, or even if they came up with the idea all on their own without ever even hearing of this other pocket diaper. (Poster 6, Designer forum, July 2008) The conversation continues on this for a few posts, with any clarity rapidly disappearing into the quagmire of different areas of intellectual property law. That is patent law and is a completely different issue. You can also trademark your design which is yet another set of laws. I was just pointing out that this is not a copyright issue. Patents and trademarks cost a lot of money to file. (Poster 5, Designer forum, July 2008)

Unravelling Intellectual Property in a Specialist Social Networking Site

Attribution The upshot of the discussion is the original poster decides she won’t publish the pattern on the site, although legally it would seem she was well within her rights to do so. The advice of various posters to go and talk to the family of the dead knitter and find out where the pattern came from, to go and talk to a lawyer and to track down the pipers’ regiment and see if they knew where the pattern came from – all so that she could attribute the design if possible – inserted a moral discourse into the discussion. We can see that there is a slippage into a norms-based framework. However a norms-based system requires a tacit acceptance of the norms. Given the diversity of users, such a consensus has yet to be (and may never be) reached.

Copying Objects not Patterns (Again) At this point another poster joins the conversation by saying she has a similar problem/question, in that she saw a photo of an actress wearing a jacket and developed her own pattern from the photo. Now she wants to publish it but isn’t sure of the legality. She says: I don’t know if it’s legal to publish that patter [sic] for free, i don’t want to sell it (it will be illegal). what do you think about that? (Poster 7, Designer forum, July 2008) This is yet again the indication that people think that selling patterns may cross some legal line, but they are not so sure if sharing for free will. The first response says: If you developed a pattern soley[sic] from seeing a finished garment, that is your pattern, even if it’s not all that “original”. And if you changed some things, it’s definitely yours. You can publish it for free or sale, you would not be breaking any copyright laws. (Poster 8, Designer forum, July 2008)

One of the site mechanisms available on Ravelry is the ability to indicate whether you agree or disagree with a post. In relation to the above response there were an even number of agree and disagree.

Conflicting Attitudes to Commerce and Gifting At this point the discussion swerves into some strong moral discussion and highlights the uneasy co-existence of the social, reputational and financial economies within the same environment. I know that the flattery of having knitters wanting your patterns can feel really good, but it’s wise to step back and take a breath to think things through first before committing. To me the bigger question is why bother spending all the time, effort and sometimes money to put out a complex free pattern? People will be just as demanding that it be accurate, well-photographed, sized from 0-60, test-knit, etc, as if they had paid for it, so you might as well at least have that. … So sure, go ahead and practice your reverse-engineering skills; those are extremely valuable things to develop. Knit all the things you want for yourself or as gifts while you hone those techniques. And then put the time into creating something really original, something that you don’t have these worries about. (Poster 4, Designer forum, July 2008) There are a number of interesting aspects to this post. Firstly it highlights the amount of work that goes into developing a pattern. The conversation in later posts picks up on this and mounts the argument that designers who give their work away are under-valuing themselves and their time. There is indeed a long-standing line of thought that creative workers constantly undervalue their work and time and should push harder for proper remuneration for their creative endeavours. In this discourse, value is couched in monetary terms. There is very little challenge

259

Unravelling Intellectual Property in a Specialist Social Networking Site

made to this assumption – it specifically pours scorn on the idea that a designer might find flattery (a form of social reward) enough incentive or reward for publishing. No-one comments in this particular thread although in another, similar, thread someone says: I’ve been knitting for a long time, part of the KnitList community for more than a decade (can’t remember!). The “I made this and wanted to share it with you” feeling, especially during the holiday pattern exchanges, is great. But there’s a different feeling in the air now, less of a share, more of a “mine!” (Poster 9, Designer forum, July 2008) Another aspect touched upon here and further elaborated upon in other threads is that selfpublishing comes with some disadvantages, one of which is responding to requests for support from users of the pattern, who are apparently quite demanding. When publishing through a conventional publisher, pattern support is handled most often by the technical staff of the magazine. Some designers have commented however, that they are so easily contacted through the web now that they end up doing direct support anyway, even if the pattern is not self-published.

Defining Originality The final interesting aspect to the post is the reference to designing “something really original” as if that were an unproblematic concept around which there was a measure of consensus. The long threads debating what constitutes a modification and what constitutes an original piece of design attest to this lack of consensus. A later post in response to the opinion that publishing this pattern would be entirely legal also wheels in a strong piece of cultural norming: This is not a good habit for serious designers to get into. I don’t know if you (the poster of the original thread) have intentions of going profes-

260

sional or not, but if you are or think you might ever, steer clear of this. As has been mentioned before, what is legal and what is ethical are not always the same. This industry is not a large one, and the choices you make now will affect how your work is seen. (Poster 10, Designer forum, July 2008) What is implied here is the reputation economy and its power to determine future success. This is a characteristic also identified by Fauchart and von Hippel in the way that French chef’s norms work. The original poster is well aware of this economy as she posts: I wanted to distinguish between the “can” and the “should” because I didn’t want a situation where it turned out I was legally okay to publish the pattern, and did, and then discovered after the fact that people thought I was kinda scummy to have done so. (Poster 2, Designer forum, July 2008) Again we see the articulation of both tacit and explicit rules co-existing. The thread then moves briefly to the idea of originality and the use of ‘stitch dictionaries’ – which describe the basic techniques for creating the different kinds of fabric produced through knitting. There is little consensus here as to whether it is legitimate to copy the basic formula from a stitch dictionary into a pattern. For the most part it is thought to be fine, as these are seen as part of the traditional tools of knitting, without an obvious original author. There are some exceptions of course, where new stitch patterns are developed, but for the most part these stitch ‘recipes’ are seen as tools or ingredients, there to be combined in new ways to create new forms. There are people who question whether they should publish a pattern that uses a stitch pattern from a stitch dictionary. … There are a lot of opinions about the extent to which people should rely on

Unravelling Intellectual Property in a Specialist Social Networking Site

these resources, but as a general answer, you may use both as part of your design process and neither is considered unethical. So while I agree with the sentiment overall, I think it’s good when we have these discussions so that in the cases when someone wishes to do something entirely acceptable, they don’t limit themselves unnecessarily. (Poster 11, Designer forum, July 2008) Here is some acknowledgement that the moral and ethical discourses of some posters may be having an undesirable effect on new designers, discouraging them from publishing. An exchange in a different thread affirmed this line of thinking: IMO, there are many people who think they have rights they simply aren’t allowed under US law and it leads to bullying which really ticks me off. (Poster 12, Patterns forum, July 2008) Bingo. It’s one thing to assert your legal rights. It’s another thing to use misunderstanding of copyright to drive competitors out of business. (Poster 13, Patterns forum, July 2008) Others are less flexible. This post from a different thread typifies a particular strand of anticopying discourse: Copyright “rights” are unable, at present, to prevent people from “misappropriating” the design elements of another; they do exist to protect the “written words”. There is an ethical element which obviously the law does not, at least in the US, address yet. I think it would behoove people to consider this: If I do “take” the design of someone else and rewrite it in my own words, and simply change a stitch pattern, then publish it, HOW WOULD I REACT if someone did the same thing to me? Would I be

happy about it? Or would I scream and holler and be upset that someone “took” my design? (Poster 14, copyright matters forum, April 2008) The use of heavily loaded terms like ‘misappropriating’ (in law this is the language used for theft and fraud) makes clear the judgement being passed on people seen to be ‘copying’ designs. This sentiment is echoed in various other threads as well.

Professionalism The original thread discussion moves into a short consideration of what constitutes a ‘professional’ designer, in which the idea of ‘professional standards’ is raised and where it is implied that ‘professional ethics’ would mean that publishing a pattern made from a copy of someone else’s work should not be contemplated. Asked for a definition of professional, this reply is offered: As you are selling designs … you probably fill the bill at least by practice. There are however designers that are at the same capacity that pretty much do it just for the fun of it. It is a hobby for them. There are behaviors that are common to “real professionals”, (some of which “real professionals” do not execute well). This is where ethical standards and so on kick in. (Poster 10, Designer forum, July 2008) More broadly this issue of professionalism is at the core of some of the biggest clashes that occur around expectations and behaviors within this SNS. Whereas prior to internet environments, and prior to aggregating sites such as this SNS, professional designers operated within a reasonably well defined sphere of interaction with publishing houses with standard practices, the enabling of easy self-publishing and the creation of this SNS which allows for easy access and the ability for users to find those self-publications, has meant

261

Unravelling Intellectual Property in a Specialist Social Networking Site

that the number of ‘professionals’ (people who make money from their designs) has significantly increased, but also that professionals now operate in a mixed environment with amateurs. And in fact it is probably possible to further differentiate, as the last poster did, between those who are trying to make a living from their designs and those who are only supplementing their income and who either are supported by partners or have their own day jobs. The competition for attention and for money has become much bigger. It is in this mixed group that we can see how the norms-based system is struggling in that the norms are still emergent and contested. The practices of amateurs – doing what they have done for a long time, only now more visibly and more widely, is seen by the amateurs as a wonderful boon – an expansion of resources at their fingertips. However the professionals can see the self-publishing free pattern market as direct competition and as undermining their ability to make money. They mount a strong argument for valuing designers’ work through monetary payment. What’s more, they sometimes exert pressure on other designers to stop giving away patterns. This post, from a different thread, holds a strong admonishment from a professional: Where is the respect for your selves as a designer! … I hope I don’t offend anyone, but hey designers, please respect the work you’re doing, you are doing a hell of a job and you deserve to get compensated for your work. (Poster 15, Designers and Crafters working together forum, April, 2008) In particular some designers see the posting of free patterns that look similar to their own as a threat and this engenders many long debates attempting to define originality and who can claim it. The legal framework does not help in this argument very much, as the law is unclear, cumbersome, and inconsistent across jurisdictions.

262

Thus we get the twinning of legal discourse with a moral discourse about what should be done, which works heavily on the assumption that the monetary economy should trump the social and gifting economies.

Mixed Practices A complicating aspect to these observations is that many of the designers who sell patterns also share some patterns for free. Some have a rationale of sharing free patterns as a marketing strategy, drawing people into a familiarity with their work that will build their reputation and lead to financial rewards in the longer term. Others have followed a trajectory of starting out by publishing their novice designs for free, but beginning to charge money as they become more confident, and again, as their reputation is built. Interestingly some publish ‘simple’ patterns for free and charge for their more complex patterns. Thus even some of the ‘professional’ designers straddle the social, gifting and financial economies. The rationales just described give some coherence to what might look like inconsistent behaviour. Similarly, while many consumers of patterns will gratefully take the free patterns, many voice a desire to pay designers for their hard work as well. Thus there are often compliments to designers on their work, accompanied by a willingness to pay for design, and a clearly stated desire to encourage designers in their professional gambits.

The Limits of Conversation The important thing to finally point out is that despite these very long, very thoughtful explorations of the intellectual property and copyright issues, the reach or audience of the discussions is limited. Thus in a site with over 600 thousand users, most of the threads in the “Copyright Matters” forum studied here were read by a maximum of about 400 readers. Some threads in the “Patterns” forum that relate to copyright have been read by over

Unravelling Intellectual Property in a Specialist Social Networking Site

3000 users, but this is still quite a small proportion of the site population. Thus while some designers are working out the issues among themselves, the vast majority of users are probably mostly oblivious to such conversations. Copyright comes up as a peripheral issue in various other forums, so it is not completely off the radar, but it is probably only a major concern for a small percentage of Ravelry users.

WHAT GOVERNS CONSUMER CHOICES If copyright and intellectual property more generally are mechanisms to regulate exchange in the commercial market and to balance the rights of authors, consumers and publishers, are they doing the job efficiently in new environments such as the one described here? Is the commercial market and its logic the system that is driving the exchanges that take place here, or do the other systems – those based in social exchange, reputation and gifting – change the logic to the point where intellectual property should no longer be the mechanism through which exchange is organised and regulated? A search of most popular patterns is revealing in that there is a spread of free and for-fee patterns. In a scan of the top 10 ‘new and popular’ patterns, 7 of the 10 were free. In the ‘most popular’ in different categories of patterns (not based on being recent) again there is a spread. In the ‘baby’ section 8 of the top 20 patterns are free, 11 of the top 20 bags are free, 11 of the top 20 blankets, 6 of the top 20 cardigans are free, 6 of the top 20 pullovers and so on. The users do not seem to be primarily basing their decisions on price. Other mechanisms are driving their choices. Potts et al (2008) would suggest it is not just taste, but the choices of others that influence an individual’s choices in a social network market and this would seem to be one explanation for how users make decisions about patterns to knit. Reputation comes

into play alongside taste and design, as does the ability to gain attention in this space. Sorting mechanisms such as the ‘most recent and popular’ and the ordering of search results by popularity contribute to a snowball effect around particular patterns. If users are not price-sensitive about patterns, then the worry professional designers have about being undercut by what they perceive to be ‘knock-off’ copies of their patterns offered for free might be ill-founded. However more research is needed to tease apart the particular mechanisms of choice in play. If the incentives and rewards for both consumption and production in this space do indeed span the different economies, it points to the increasing irrelevancy of copyright as an incentive and reward mechanism. What emerges from this complex environment may have other mechanisms at its heart. Reputation is extremely important here, and can be built through a number of means. Sometimes this is derived from older publishing environments. Hence, a designer who has published in one of the big knitting magazines has an easier time establishing credibility and legitimacy in this environment. However the pathways for new designers to build reputation have just been opened up considerably through the new online specialist SNS environment.

CONCLUSION Copyright and intellectual property are not completely irrelevant as mechanisms of regulation, but there is a problem if most people have very poor literacy with a key regulatory mechanism, and in fact even most trained legal people admit to uncertainty over it. If a simple request for clarification generates a thread with over 80 posts and almost inevitably ends with “you need to consult an attorney”, then there is a problem. While discussion leads to some literacy, in fact the general effect seems to be off-putting rather than encouraging people to learn more. In the resulting confusions

263

Unravelling Intellectual Property in a Specialist Social Networking Site

moral rather than legal discourses seem to drive some of the decisions made to publish or not, even though discussions are couched around copyright. The frictions between amateurs and professionals, or the social and financial economies of a social network market, are played out on the field of IP.

Solutions The Ravelry site owners were in the process of designing some standard Creative Commons style licenses for designers to use if they chose, but acknowledged that these would not fulfil the needs of all designers in all jurisdictions. They partly hoped that the licenses would serve an educational function for buyers by making clear what they were able to do with the patterns they were either purchasing or being given for free. The Ravelry standard licenses have never eventuated, however. It is possible that the owners encountered the same legal nightmare the users have been dealing with and decided not to venture into this territory. The broader issues implied through this case study are about the need both for simplification and flexibility in intellectual property law and the need for developing new literacies in legal matters in communities of users. The use of IP as an ideological tool in the service of creating a largely commercial environment is interesting, but the success of this strategy is not by any means a foregone conclusion. Whether a norms-based system could emerge as a co-regulatory mechanism for IP within this community will depend on whether dominant norms can be established. Thus far the interests of the diverse range of users seem to mitigate against this, and the likelihood of getting effective ‘buy in’ from enough users to have a means of enforcing such a system seems some way off. However this study has shown the emergence of a group of issues and some common attitudes towards them. The sheer repetition involved in the conversations on the bulletin boards around the issues may indicate that over time a form of

264

community consensus may emerge. No simple solution suggests itself to the problem confronting the users of Ravelry or more widely the users of other specialist SNS sites. The sites themselves are still emergent social forms, and it seems clear that this is a transition period where norms are still being struggled over and no settled form has arisen yet. Law has always lagged behind social practice and in this case it is probably going to be some time before the vested interests of entrenched IP holders and institutions are superseded by newer regimes that more adequately reflect the needs of users.

ACKNOWLEDGMENT Thanks to Dr Mary Heath, Flinders University Law School, for valuable insights (although all legal commentary is my own and any mistakes in interpretation are mine alone). Thanks also to the two anonymous reviewers for helpful suggestions.

REFERENCES Abrams, L. (2006). Knitting, Autonomy and Identity: The Role of Hand-Knitting in the Construction of Women’s Sense of Self in an Island Community, Shetland, c. 1850–2000. Textile History, 37(2), 149–165. doi:10.1179/004049606x132078 Banks, J. (2007). Opening the production pipeline: unruly creators. In De Castell, S., & Jenson, J. (Eds.), Worlds in Play: International Perspectives on Digital Games Research. New York, NY: Peter Lang Press. Banks, J., & Humphreys, S. (2008). The Labour of User Co-Creation: Emerging Social Network Markets? Convergence, 14(4), 401–418. Benkler, Y. (2006). Wealth of Networks: how social production transforms markets and freedom. New Haven, Connecticut: Yale University Press.

Unravelling Intellectual Property in a Specialist Social Networking Site

Bettig, R. V. (1992). Critical perspectives on the history and philosophy of copyright. Critical Studies in Mass Communication, 9(2), 130–155. doi:10.1080/15295039209366821 Boyd, D., & Ellison, N. B. (2008). Social Network Sites: definition, history and scholarship. Journal of Computer-Mediated Communication, 13, 210–230. doi:10.1111/j.1083-6101.2007.00393.x Boyle, J. (2002). Fencing off Ideas: Enclosure & The Disappearance of the Public Domain. Daedalus, (Spring): 13–26. Coombe, R. J. (2003). Commodity Culture, Private Censorship, Branded Environments, and Global Trade Politics: Intellectual Property as a Topic of Law and Society Research. In Sarat, A. (Ed.), Companion Guide to Law and Society. Cambridge, MA: Basil Blackwell. Drahos, P., & Braithwaite, J. (Eds.). (2002). Information feudalism: who owns the knowledge economy?London, UK: Earthscan. Fauchart, E., & Von Hippel, E. A. (2006) NormsBased Intellectual Property Systems: The Case of French Chefs. MIT Sloan Research Paper No. 4576-06. Retrieved from http://ssrn.com/ abstract=881781 Frow, J. (2000). Public Domain and the New World Order in Knowledge. Social Semiotics, 10(2), 173–185. doi:10.1080/10350330050009416

Lessig, L. (1999). Code And Other Laws Of Cyberspace. New York, NY: Basic Books. Lessig, L. (2004). Free culture: how big media uses technology and the law to lock down culture and control creativity. New York, NY: Penguin Press. Levy, P. (2001). Collective Intelligence. Malden, MA: Blackwell. Postigo, H. (2008). Video Game Appropriation through Modifications: Attitudes Concerning Intellectual Property among Fans and Modders. Convergence: The International Journal of Research into New Media Technologies, 14(1), 59–74. doi:10.1177/1354856507084419 Potts, J., Cunningham, S., Hartley, J., & Omerod, P. (2008). Social network markets: a new definition of the creative industries. Journal of Cultural Economics, 32(3), 167–185. doi:10.1007/s10824008-9066-y Rimmer, M. (2001). Napster: infinite digital jukebox or pirate bazaar. Media International Australia, 98, 27–38. Smiers, J. (2002). The abolition of copyrights: better for artists, Third World countries and the public domain. In Towse, R. (Ed.), Copyright in the cultural industries. Cheltenham, UK: Edward Elgar Publishers. Vaidhyanathan, S. (2004). The Anarchist in the Library. New York, NY: Basic Books.

Humphreys, S. (2009). The economies within an online social network market. A case study of Ravelry. in Flew, T. (Ed.) Communication, Creativity and Global Citizenship: Refereed Proceedings of the Australian and New Zealand Communications Association Annual Conference. Brisbane.

WEBSITES

Leadbeater, C., & Miller, P. (2004). The Pro-Am Revolution. How enthusiasts are changing our economy and society. London, UK: Demos.

http://www.cellartracker.com

http://www.ravelry.com http://www.creativecommons.org.au

http://www.skispace.com

265

Unravelling Intellectual Property in a Specialist Social Networking Site

ENDNOTES 1



2



3



266

For the sake of brevity I will mostly refer to knitters for the rest of this chapter. Statistics sourced from http://www.ravelry. com/statistics/users on March 23 2010. According to this page there had been 151,966 users active in the past 7 days. The names of posters have been anonymised for the purposes of this research.

4



Creative Commons non-commercial attribution licence basically “Lets others copy, distribute, display, and perform your work — and derivative works based upon it — but for noncommercial purposes only.” (http:// creativecommons.org.au/licences, accessed March 31 2010)

288

About the Contributors

Artur Lugmayr describes himself as a creative thinker and his scientific work is situated between art and science. Starting from July 2009 he is full-professor for entertainment and media production management at the Department of Business Information Management and Logistics at the Tampere University of Technology (TUT): EMMi - Entertainment and Media Production Management (http:// webhotel2.tut.fi/emmi/web/). His vision can be expressed as to create media experiences on future emerging media technology platforms. He is the head and founder of the New AMbient MUltimedia (NAMU) research group at the Tampere University of Technology (Finland) which is part of the Finnish Academy Centre of Excellence of Signal Processing from 2006 to 2011 (http://namu.cs.tut.fi). He is holding a Dr.-Techn. degree from the Tampere University of Technology (TUT, Finland), and is currently engaged in Dr.-Arts studies at the School of Motion Pictures, TV and Production Design (UIAH, Helsinki). He chaired the ISO/IEC ad-hoc group “MPEG-21 in broadcasting”; won the NOKIA Award of 2003 with the text book “Digital interactive TV and Metadata” published by Springer-Verlag in 2004; representative of the Swan Lake Moving Image & Music Award (http://www.swan-lake-award.org/); board member of MindTrek (http://www.mindtrek.org), EU project proposal reviewer; invited key-note speaker for conferences; organizer and reviewer of several conferences; and has contributed one book chapter and written over 25 scientific publications. His passion in private life is to be a notorious digital film-maker. He is founder of the production company LugYmedia Inc. (http://www.lugy-media.tv). More about him on http://www.cs.tut.fi/~lartur. Heljä Franssila is a project manager and researcher in University of Tampere, Hypermedia Laboratory, Finland. Finding efficient and human-centered ways to couple the new information technologies and interaction styles with work practices and business processes is her central multidisciplinary research interest and motivation. At the moment she works in close research co-operation with several commercial enterprises to find and design meaningful and sustainable ways to support and energize diverse communities in working life with social media and web 2.0. Pertti Näränen (b. 1962 in Finland) is Senior Lecturer of media studies at Tampere University of Applied Sciences, School of Art and Media. His doctoral thesis title was “Digital television: Analyses on early history, challenges to media policy, and transformation of television” (2006, University of Tampere, dept. of Journalism and Mass Communication). The dissertation is available in electronic format at http://acta.uta.fi/teos.php?id=10822.Previously Mr. Näränen has worked as researcher, lecturer and journalist in print and radio media. He also has background in cinema studies and in various new media projects. Home page: http://www.tamk.fi/~narper/

About the Contributors

Olli Sotamaa completed his PhD at the Tampere University in spring 2009. The dissertation discussed player productivity among game cultures and player-centred design. Currently I’m heading a project called Games as Services (GaS). I’m also actively involved in teaching our masters students. His areas of competence are: digital popular culture, game cultures, player production, player-centred design. His current research interest is player-created content on game consoles, Game industry as cultural industry. More about Olli on https://153.1.6.41/hyper/henkilokunta/sotamaa_en.php. Jukka Vanhala is a professor at the Department of Electronics at Tampere University of Technology and the director of the Kankaanpää Research Unit of Wearable Technology. He received his MSc at the Software Engineering Laboratory in 1985, Licenciate of Technology at the Microelectronics Laboratory in 1990, and doctor of technology at the Electronics Laboratory in 1998, all at TUT. He has been appointed to the professorship of electronics at TUT in 1997 and to the docentship of interactive technology at Tampere University at 2005. His career also includes six years of work in the industry both in Finland (Tele Finland (Sonera), SoftPlan (Nokia), Instrumentation) and in USA (IBM). His expertise is in ambient intelligence, embedded systems and wearable technology. Zhiwen Yu is currently a professor and Ph.D. supervisorat the School of Computer Science, Northwestern Polytechnical University, P. R. China. He received his B.Eng, M.Eng and Ph.D. degree of Engineering in computer science and technology in 2000, 2003 and 2005 respectively from the Northwestern Polytechnical University. He has worked as a research fellow at the Academic Center for Computing and Media Studies, Kyoto University, Japan from Feb. 2007 to Jan. 2009, and a post-doctoral researcher at the Information Technology Center, Nagoya University, Japan in 2006-2007. He has been a visiting researcher at the Context-Aware Systems Department, Institute for Infocomm Research (I2R), Singapore from Sep. 2004 to May 2005. He has been an Alexander von Humboldt Fellow at the Mannheim University, Germany from Nov. 2009 to Oct. 2010. *** Abdelhak Attou is a software engineer working on telecommunication mobile applications for various SmartPhone platforms. He undertook a PhD in context aware service adaptation at the Centre for Communication Systems Research (CCSR) at the university of Surrey, where he was working on a project funded by the Mobile Virtual Centre of Excellence in mobile and personal commutations (MVCE) on ubiquitous services. Dr Abdelhak Attou got his BSc in computer science from the university of Manchester. Asta Bäck (M.Sc. Tech) works at VTT Technical Research Centre of Finland since 1983. She graduated with a M. Sc Tech in printing technology and graphic arts from the Helsinki University of Technology in 1983. She currently works as senior research scientist and leads the Networked media research team. Her team develops new media concepts and prototypes that combine personal media and social user interactions, and utilise user created metadata to improve and personalise media experiences. She managed two eLearning related European projects: CustomDP (2000-2003) and SELEAC (2004-2005). She was the project manager in VTT’s SOMED project (SOMED = Social media in the crossroads of physical, digital and virtual worlds) (2006-2008). She has more than 120 publications registered at the VTT publication register.

289

About the Contributors

Andrea Botero is a Doctor of Arts candidate in the Media Lab - School of Art and Design of the Aalto University where she also works as researcher. Her research interest lies in theoretical and practical implications of broad participation in creative design processes and how this relates to “innovation”. Her design work explores services, media formats and technologies for communities and their social practices. Alessandro Canossa works from the Center for Computer Games Research, IT University of Copenhagen. His work focuses on developing methods for locating, evaluating and designing for, patterns of behavior in computer game players. He is working with a range of game development companies to improve user-oriented testing in game development. Mark Doughty is a Principal Lecturer in the Lincoln School of Computer Science at the University of Lincoln. Mark was a co-author for the European Union Life Long Learning LEONARDO project ‘DREAD-ED’. Mark’s research activity is in the field of serious and social gaming and he has a number of previous publications in this and other fields. Anders Drachen is a Visiting Researcher at the Institute for Informatics, Copenhagen Business School, and consultant at Game Analytics Technologies. He collaborates with a range of game development companies to improve user-oriented testing in game development. Martin Ebner is Head of the Departiment Social Learning of the Computer and Information Services of Graz University of Technology and therefore he is responsible for all e-learning activities at this university. His research interests are especially in mobile and ubiquitous learning approaches and in the application of future technologies for teaching and learning. Martin taught a number of lectures on the topics Technology Enhanced Learning, Multimedia Information Systems and Social Aspects of information technology at the Institute of Information Systems Computer Media. He is member of different international committees in the area of e-Learning and conducts the e-Learning Blog: http:// elearningblog.tugraz.at. Victor Manuel García-Barrios is Professor for Computer Science and key researcher at School of Geoinformation, Carinthia University of Applied Sciences, in Villach, Austria, as well as Honorary Professor at Galileo University (Guatemala City, Guatemala). He has got a PhD in the field of Multipurpose Modelling Systems at Graz University of Technology (TUG), has been research assistant and project manager at TUG as well as lecturer at Graz University of Applied Sciences (Campus02). He has been responsible for and participated in many research projects, such as the research projects AdeLE (adaptive e- learning with eye-tracking), MISTRAL(semantic multimodal information management), and APOSDLE (process-oriented, self-directed, work- integrated learning). He has supervised more than 50 BSc and MSc theses in the fields of information and communication technology, and has published in the last years numerous papers in renowned international conferences and journals. His research areas and working activities include personalisation systems, technology-enhancing learning, multi-purpose user modelling, human-computer interaction as well as location-based services, geographical information systems, and decision-support systems.

290

About the Contributors

Alison Gazzard is currently a Post-Doctoral Research Fellow in New Media at the Research Institute for Media, Art and Design, University of Bedfordshire, UK. She holds a PhD from the School of Creative Arts, University of Hertfordshire, UK entitled ‘Paths, Players, Places: Towards an Understanding of Mazes and Spaces in Videogames’. She also holds an MA in 3D Computer Animation from the National Centre of Computer Animation at Bournemouth University. Her current research interests include paths, journeys and time in game spaces and location-based media, as well as understanding player:avatar relationships in virtual and real worlds and the spaces in-between. [http://www.boundedspace.org] [[email protected]] Nina Haferkamp is Research Associate at the Department of Communication at the University of Muenster, Germany. She received her Diplom (eqv. to M.Sc.) in Media Science (specialization: media psychology) in 2006 from the University of Cologne, Germany, and finished her Ph.D in Psychology in 2009 at the Department Social Psychology: Media and Communication at the University of DuisburgEssen, Germany. From 2008 to 2009, she worked as Research Assistant in the EU-funded project “DREAD ED – Disaster Readiness through Education” at the University of Duisburg-Essen. At her current job at the Department of Communication at the University of Muenster, she is involved in a DFG-funded young scholars’ network on privacy and Web 2.0. Her research interests include social aspects of Web 2.0 (in particular self-presentation on social networking sites) and computer-mediated communication. Sal Humphreys began work at the University of Adelaide in February 2009. She completed her PhD on online digital games at QUT in 2005 and subsequently was awarded a 3 year position as post-doctoral research fellow on an ARC linkage grant through QUT. Sal has previously taught at the University of South Australia in media studies, and has worked for various state and private organisations in South Australia in areas relating to online media. She was employed by Ratbag games development company for a year in 1998, and has worked as a freelance editor. Kimmo Karhu is a doctoral student and researcher in the Software Business and Engineering Institute at the Aalto University School of Science and Technology. He received a Master of Science degree in Computer Science from Helsinki University of Technology in 2002. Prior to joining the faculty, he worked in ICT industry for more than ten years in various positions, the latest as VP of Product Development in the biotechnology company Medicel. His research interests include digital business ecosystems and communities, and the ways in which these could be studied using mathematical modelling. Karhu is also interested in open source approaches and computer law in general. He has published several papers for international conferences. Ben Kirman is a Lecturer within the School of Computer Science and member of the Lincoln Social Computing (LiSC) Research Centre at the University of Lincoln. Ben’s research interests are in the field of Ludology (the study of games and play) and specifically the social aspects of gaming. Ben also has a strong interest in non-electronic gaming and social networking as applied to multiplayer gamers and players of different gaming styles.

291

About the Contributors

Hannu Korhonen is a senior researcher at Nokia Research Center, Finland. He has been working in the game research area since 2005, focusing on developing playability heuristics that are used with the expert review method. The method is used to evaluate mobile games on Nokia’s N-Gage platform and Hannu has been involved in playability assessment of mobile game titles such as Dance Fabulous, Snakes Subsonic, and Dick Dagger and the Fallen Idol. He has published several scientific articles on playability issues and other topics in mobile game research. He is also a Ph.D. student at the University of Tampere and his research topic is playability evaluations of mobile games with an expert review method. His other research interests include various areas of mobile interaction. Recently, he has been studying the user experience, specifically playful experiences, on mobile devices which applies game research knowledge to regular software design to make them more engaging, attractive, and most importantly, more playful for the users. Nicole Krämer is Professor for “Social Psychology – Media and Communication” at the University Duisburg-Essen since 2007. She finished her PhD in 2001 with a thesis on socio-emotional effects of nonverbal behavior and computer animation as a method in communication research. In the academic year 2003/2004 she was visiting scholar and visiting lecturer at the University of Cambridge, Faculty of Social and Political Sciences. In 2006 she received the venia legendi for psychology with a habilitation thesis on “Social effects of embodied conversational agents” at the University of Cologne. Her research interests include human-computer-interaction, social psychological aspects of web 2.0, nonverbal behaviour and computer supported instructional communication. She heads various EU- and DFG-funded research projects on virtual environments, computer supported instructional communication, humanagent/robot interaction and science communication via web 2.0. Sari Kujala is a professor of psychology at Tampere University of Technology, unit of HumanCentered Technology (IHTE). Her background is in psychology and cognitive science. In addition, she has received Ph.D. in human-computer interaction. Her research interests focus on user experience, value-centered design, and user involvement. Shaun Lawson is a Reader in the School of Computer Science at the University of Lincoln where he directs the Lincoln Social Computing (LiSC) Research Centre . He holds PhD and BEng degrees from Universities of Surrey and Newcastle respectively. He joined Lincoln as a Senior Lecturer in 2005 having previously being a Lecturer at Edinburgh Napier University since 2000. His main areas of research are focussed on interactions with, and through, web-based and mobile systems for social purposes; much of his recent work has a strong multi - disciplinary theme featuring innovative collaborations with, for instance, colleagues from health and life sciences. He has authored or co - authored over peer reviewed 50 publications which have accrued over 100 citations from academic colleagues and has held recent research grants worth over £1 million from sources such as EPSRC, the EU, Microsoft Research and UK charities. Ning Li received her BEng degree in Computer Science and Engineering department from Shandong University, China in 1997, and PhD degree in Computer Science from the University of Manchester, UK in 2002. She worked as a Research Fellow in the Centre for Communication System Research (CCSR) at the University of Surrey, UK. She was a Work Package Coordinator for the UK DTI-funded

292

About the Contributors

Ubiquitous Services research programme. Since 2009, She has been working as a Research Associate in the Knowledge Media Institute at the Open University, UK, on a number of EU-funded projects. Her research interests include Ubiquitous Computing, Semantic Web, Semantic Web Service and Knowledge Management. Conor Linehan is a Post-doctoral Researcher in the Lincoln Social Computing (LiSC) Research Centre at the University of Lincoln. He holds BA and PhD degrees in Psychology from the National University of Ireland, Maynooth. His research interests lie in the application of empirically proven psychological methods to the design of technology for education and behaviour change. He has a particular interest in the potential of computer games as teaching tools and was employed as a reasearch assistant on the DREAD-ED project with the joint responsibility for game design. Sanna Malinen is a researcher and a PhD student in usability at Tampere University of Technology (TUT), Unit of Human-centered Technology. She has been working as researcher, project manager and part-time lecturer at TUT since 2006. Her scientific background is in Social Psychology and she graduated from University of Tampere in 2000. Since 2000, Sanna has been working on several research projects that relate to communication technology, user studies, and design of mobile and internet services. In her current research project, she is investigating social media and online communities. In her dissertation, she is focusing on social practices on online user communities. Klaus Moessner is a Professor in the Mobile Communications Research Group, Centre for Communication Systems Research (CCSR) at the University of Surrey. Klaus earned his Dipl-Ing (FH) at the University of Applied Science in Offenburg, Germany, an MSc from Brunel University, UK and his PhD from the University of Surrey, UK. His main responsibilities within CCSR include conducting research, managing and co-ordinating different research contracts and projects, supervision of PhD and MSc students and teaching courses at postgraduate student level. He leads a research team on Software based systems and IP based internetworking technologies in CCSR. Klaus is responsible for the content and service adaptation work package in the Mobile VCE core 4 research programme “Ubiquitous Services”. Pirjo Näkki works as a research scientist at VTT Technical Research Centre of Finland since 2003. She graduated with a M.Sc. Tech in Information Networks from the Helsinki University of Technology in 2006 with Human-Centred Information Systems as a major subject. Currently she does her PhD on online methods for user driven innovation. Her work and studies relate to user-centred design, user experience and usability, social media and online collaboration. She is especially interested in open innovation and user participation in early product concept design. Currently she works in innovation and social media projects, in which she develops web-based methods for user involvement in design. Luisa Nigrelli works in ISTC CNR, managing European funded projects and taking care of technology transfer from the research context to the business environment. Her background is in foreing languages and educational psychology, she took her master degree at University of Pavia in new technologies for teaching and knowledge management strategy. She was visiting scholar at Indiana University Bloomington, in Education Psychology Department. Previously she worked in KPMG and in Telecom Italia, with a special focus on international business development, EU funded programs and CSR.

293

About the Contributors

Janne Paavilainen is a project manager at the Game Research Lab, University of Tampere, Finland. He has been involved in several games research projects focusing on casual, mobile, educational and social games, and he has published several articles on these domains. Janne’s interests in games research are in design and evaluation methods, especially in heuristics. Currently Janne is working in the SoPlay project studying games and play in social media. His current task is to develop design and evaluation heuristics for Facebook social games. Janne holds a master’s degree in economics and is currently planning his Ph.D. studies focusing on usability, playability and user experience in first-person shooter games. Hannamari Saarenpää is a game researcher from the Game Research Lab, University of Tampere, Finland. She has been working there since 2006 and during that time participated in several game-related projects (especially within the area of pervasive and educational games). Currently she is working in the YOUSAT project that focuses on creating new mobile technology for interactive nature-based tourism exploration. Her main focus and responsibility in this project, as well as the previous projects, has been evaluation of player experience and user-centered design. Hannamari holds a master’s degree in Interactive Technology and is currently planning her Ph.D. studies. Christian Safran is a lecturer and researcher at the Institute for Information Systems and Computer Media at the Graz University of Technology. He received a PhD in the field of collaborative tools for technology-enhanced learning and teaching and has participated in several research projects in the context of e-learning. He has supervised numerous student projects and master theses, and teaches programming lectures for students of computer science. His research interests include social software, online communities of practise, and the application of social media in learning. Massimiliano Schembri is a psychologist working as researcher at the Institute of Cognitive Sciences and Technologies (ISTC) of the Italian National Research Council (CNR). In the past years he has worked in basic research projects studying the interaction between learning and evolution. His research interest is applying the discoverings and contents of Artificial Life to improve and enrich the serious game development. Merat Shahidi attained his BEng in 2006 with a first class honours degree in Telecommunications Engineering, and MSc in Mobile & Personal Communications in 2007 at King’s College London. He is now with the Centre for Telecommunications Research, King’s College, as a PhD Research Student. He is working within the core 4 research programme “Ubiquitous Services” of the UK’s Virtual Centre of Excellence in Mobile & Personal Communications (Mobile VCE). Stefan Uhlmann received his MSc degree in Multimedia and Signal Processing from the Tampere University of Technology, Tampere, Finland in 2007, where he is currently working towards his doctoral degree. He authored or coauthored several scientific publications. In the world of ubiquitous multimedia, his research interests are within user profiling and personalization especially considering portability and profile reusability.

294

About the Contributors

Sari Vainikainen graduated with a M.Sc. Tech in Graphic Arts Technology from the Helsinki University of Technology in 1992. She works as research scientist at VTT Technical Research Centre of Finland since 1994. Currently she works at VTT Media Technologies knowledge centre and her expertise includes semantic web technologies, social media, web applications and publication processes. Her recent projects relate to semantic analysis, semantic user profiles and recommendations. She is interested in combining the features of social media with semantic web technologies in order to create more intelligent services. Recent projects: Next Media – Event Management (2009-2010), Crossmedia - Next Generation Editorial Systems (2008-2010), PHAROS (IST-45035)- Platform for search of audiovisual resources across online spaces (2006-2009), SOMED - Social media in the crossroads of physical, digital and virtual worlds (2006-2008), TÄKY - User generated metadata as meaning indicator and part of the user experience (2006-2008). Sami Vihavainen, M.Sc., is a researcher at Helsinki Institute for Information Technology HIIT where he is a member of Self Made Media research group. He is also a doctoral student at the Department of Human Centered Technology in Tampere University of Technology. During 2007-2008 Vihavainen acted as a Visiting research scholar at University of California Berkeley’s School of Information for 1,5 year. Sami’s research interests consist of understanding people’s use of media in their everyday lives and use that understanding for designing new media services. He has several publications on the implications of media technology to people’s everyday social interaction.

295

296

Index

A Aberrant Player 200, 203-204, 206-207, 209-212, 214 access networks and devices 165 Adaptation Decision Engine (ADE) 155-156, 159, 162 Adaptation Management Framework (AMF) 149, 151, 154-157, 159-164, 166 Adaptation Manager (AM) 59, 151, 154-157, 159162, 198, 214, 246, 279 Adaptation Mechanisms (ADMEs) 157-162, 165 Adaptation Ontology 151, 153-155, 159-162 Adaptation Operations (AO) 149-163 adaptation targets 165 AI characters 191, 198 Android 95, 101, 106-107 Appropriated Play 200-201, 203-206, 208-212, 214 Attention Profiling Markup Language (APML) 71, 86-87 augmented reality games 211

B Blended Learning 129 Blogs 71, 81, 135, 147, 198-199, 225-227, 233 Business Ecosystem 218, 220, 222-223, 233-234 business intelligence 6, 8, 22

C character-based games 6-8, 10, 22 collaborative e-Learning 95 Color Histogram 174-175, 178-179, 187 ColorsInMotion 171-173, 175-176, 178-179, 183186 Community Coordinated Multimedia (CCM) 251258, 260, 265-268 community-driven 222-223

Community Innovation (CI) 216, 218, 224-225, 228-233 community managers 3 company-controlled 222-223 Competitive Acting 188, 192, 194, 196-197 Computer Supported Collaborative Work (CSCW) 25, 218, 224, 233-234, 250 Content Adaptor (CA) 26, 44-48, 65, 93, 109, 126, 128-129, 151, 154, 156-157, 159-161, 185-186, 197-198, 231, 233, 249-250, 266 Context-Aware 89, 91-93, 165, 167-168 Contributive Aberrant Player 204, 206, 209-210, 214 Converged Media 188, 190, 196-197 Creative Commons non-commercial licence 278 Creator™ 227 Cross-System Personalization 66, 75-76, 87-90, 92-93 Cultural Logic 201, 204-207, 209, 214 cultural norming 275, 281 custom-made Image Galleries 227 custom-made Web Databases 227

D Design byME™ 227 Digital Business Ecosystem (DBE) 218, 220-223, 233-234 Digital Designer Software (LDD) 224, 227 Digital Drama 188-189, 191 Digital Ecosystem 220, 222, 232, 234 Digital Item Adaptation (DIA) 152-153, 159-160, 167-168 Diigo 277 discussion boards 226, 270-271, 273, 277 Disruptive Aberrant Player 204, 209, 212, 214 DREAD-ED project 112-114, 116, 121, 123-125

Index

E

H

eCars – Now! 216, 219, 221-222, 224-225, 227228, 233-234 Emergency Management 111-116, 118, 122-129 Entrepreneurs 222, 225 European Telecommunications Standards Institute (ETSI) 70 expert review method 29-31, 34-36, 41, 43

heads up displays (HUDs) 114 HEP heuristics 32 Heuristic Evaluation for Playability (HEP) 32, 35 HSB Color Space 176, 187 Human-Computer Interaction (HCI) 1-2, 5, 23-26, 28, 30, 32, 43-48, 90, 108, 112, 231

F Facebook 6, 67, 226, 235, 237-245, 247-249 Facebook Group 226, 238, 240-241 FFMPEG 160, 169 First-Person Shooters (FPS) 7, 209 First Responder Simulation and Training Environment (FiRSTE) 114 Flow 21, 119, 124, 188, 191, 205-206, 209, 213, 273 Formal Concept Analysis (FCA) 73, 89 Forums 45, 204, 207, 210, 223, 226-227, 232, 255, 277-284 Fuzzy Analytic Network (FAN) 73

G GAMECAST 190-193, 195, 198-199 game evaluation 4, 29, 31, 33-34, 47-49 game experience 16, 21-22, 30, 32, 36, 42, 49, 192, 196, 207 Game-Initiated Events (GIEs) 3, 8-9 Game Mechanics 11, 31-32, 35-36, 38, 49, 116, 118-121, 129, 211 game metrics data 2, 6, 23 gameplay metrics 3, 5-10, 13, 16, 18, 21-24, 27-28 game researchers 3, 30, 32, 34-35 gameworld 201-203, 206-208, 212 General public 116, 222-223, 225 geographical information system (GIS) 10, 14, 16, 99 Geo-Tagging 95-96, 100, 104, 109 Glitch 206-207, 214 Google Docs 97-98 Google Wave 97-98 Grapple User Modeling Framework (GUMF) 7071, 86 Group Decision-Making 112, 117, 125, 129 Groupware 31, 33, 43, 224-225, 231-232, 234

I Intellectual property (IP) 257, 269-277, 279, 283286 Interactive Visualization 171-172, 186-187 International RSS feed 226 Intrinsic Learning 115, 129 IP conventions 272 iPhone 95, 101, 106-107 IRC channel 225-226

K Kaiser normalization 243 Knowledge Base (KB) 131, 138-139, 150-151, 153154, 159, 266

L Learning Outcomes 111-112, 114-115, 117-119, 124-125, 129 LEGO and eCars creators 222 LEGO and eCars users 222 LEGO Digital Designer software 224 LEGO TV™ 227 ludus 200-201, 203-204, 206-208, 211, 214

M Mailing lists 225-226 Massive Multiplayer Online Role Playing Games (MMORPGs) 4, 7, 26, 189 Mediawiki 95, 99-101, 104, 109 Metcalfe’s Law 98 Metrics-Driven Development 6, 22 Microblogging 95-96, 104-110 Micropage 95, 105-106 Mobile Device (MD) 45, 73, 75-80, 84, 93, 97, 101, 103-105, 109, 185, 253, 267 mobile learning (m-Learning) 95-97, 99, 102, 104, 108-110 Mobile Virtual Center of Excellence (MVCE) 164, 167

297

Index

MPEG-7 152-153, 159-160, 167-168, 252, 261262, 265-266 MPEG-21 DIA 152-153, 159-160, 168 Multimedia Description Schemes (MDS) 84, 152153, 160 Multimedia Intensive Services 253, 267 My City 235, 237-238, 245-248

N Network Perspective 164-165 non-player characters (NPCs) 9, 30, 191, 194-195 norms-based framework 280 norms-based IP 274

O ontology 70-71, 76, 81-83, 86, 88-91, 130-136, 138-143, 145-147, 149-156, 159-162, 167, 169 Open Innovation (OI) 228-230, 232, 234 Our social metadata ontology (OSMO) 139, 141143, 146 OWL-S 152, 160, 169

P P2P SCCM 251, 253-255, 257, 265-266, 268 paidia 200-201, 204, 206-208, 212-213 Paper Prototyping 129 Participation-of-Many 190-191, 195 Peer-to-Peer (P2P) 251-255, 257-260, 265-268 personalized user profile 67, 88 Personal Shopping Assistant (PSA) 53 Pick a Brick™ 227 player behavior 1-8, 10, 13, 15, 22-23 player metrics 2-5, 28 PLAY heuristics 29, 31-38, 40-43, 45, 47-48 playtesting 10, 26, 28, 36, 47 Portable Personality 66, 76, 88, 90-93 Product Gallery 227 Purposeful Play 200-203, 208-210, 212, 214

R Ravelry 269-271, 273, 275-278, 280, 284-287 Real-Time Strategy (RTS) 6, 8, 10, 33 RGB Color Space 187 Role-Playing Games (RPGs) 3, 7-8, 114

S semantic tagging widget 132, 138-141, 143, 146

298

Semantic Web 70, 80, 86, 88, 141, 147, 150, 152153, 156, 166-169 Service Description Language (WSDL) 152, 160, 168-169, 252, 267 Service-oriented CCM (SCCM) 251-257, 261, 265266, 268 Service Provider (SP) 78-80, 84, 149, 154, 166 Severa project management software 226 Simplicity User Profile (SUP) 75 Slit Scan 173, 176-177, 181, 187 Smart Shopping-Aid (SSA) 54 social bookmarking 130-133, 136-137, 139, 146147 social networking site (SNS) 237, 269-270, 273275, 282, 284-285 social semantics 147 Specialist Social Networking Sites 269-270, 273, 275 Speed running 207 Spreading Activation algorithm 73 Square Enix Europe (SEE) 1, 10, 12, 22-23 Syntagmatic 201, 203 syntagms 200-205, 207-210

T tagging 89, 107, 130-136, 138-147 The document workshop (Paja) 226 Three-Phase Model (TPM) 58 Tilkut 130-131, 133-146, 148 TUGeoWiki 95-96, 98-108, 110 Turn-Based Games (TBG) 6

U Ubiquitous Services 53, 149-150, 162, 164-167, 169 Unified Resource Identifier (URI) 77, 79, 140 unified user context model (UUCM) 71, 75, 78 usability 2, 21-25, 29-36, 38-39, 44-49, 135, 166 user experience data 21 User-Initiated Events (UIEs) 3, 8, 28 User Innovation (UI) 30, 228-230, 232-234 User Perspective 162, 164-166 user study 35, 130, 133-134, 136, 147, 245, 247

V valorisation 271 Varimax 243 VideoAnalyzer 173, 175, 178

Index

Video Jockeying (VJing) 174, 184 Video Sharing and Discussion Forums 227 Video Space 171-173, 175, 179-184, 187 Videosphere 173, 185 Video Visualization 172-173, 185, 187 Virtual Environments (VEs) 3-4, 6-7, 23-24, 27, 64, 114, 188, 197, 279, 281 Virtual LEGO Models 226-227

W

web application 102, 133, 137, 142, 147 Web Document Conceptual Clustering (WebDCC) 73 Web Service Description Language (WSDL) 152, 160, 168-169, 252, 267 Web Services 96, 149, 152-153, 159-162, 166, 168169, 236, 251-254, 256, 260, 265-268 Web Translation Engine (WTE) 165 Wiki 95-110, 148, 169, 222, 225-226

Web 2.0 93, 95-96, 133, 147, 184, 216, 218, 222223, 228, 236

299